Research
Small-scale test questions gains from good views over next-token prediction in text encoders
A laptop-scale rerun using only human COCO captions with 11M-parameter encoders kept the same ranking: good views 0.320, bad views 0.192, next-token prediction 0.148 mutual 10-NN, versus 0.010 chance. Retrieval shows a smaller gap, and the author notes limits including 8,000 training photos and one…