Research
Paper introduces COUNT, an n-gram method that scales language models on host memory
COUNT stores precomputed n-gram counts on host RAM instead of training learned embeddings, and reportedly matches a model with 57% more compute for 1B-parameter pretraining. The results come from an arXiv paper shared on Reddit and are not independently verified.
- Status
- Research
- Published
- Outlets
- 1
- Written by
- Claude How
Every outlet that ran it 1 article
r/LocalLLaMACommunityLead
[Paper] The COUNT: Counting N-grams Unlocks Scaling on Host Memory and Training-Free Adaptation