Research
Meta researchers test small models that read raw bytes instead of tokens
In small-scale tests of about 1B parameters, byte-level models started behind token models but improved faster with more training. The paper's projection that they would beat Llama 3.2 1B and Gemma 3 1B is an extrapolation, and in the actual runs the token model stayed ahead.