Research
Study finds population based training rescues grokking from a bad hyperparameter range but does not speed it up
On modular addition, PBT was slower than a grid search with equal compute, but fast PBT escaped a dead zone where fixed settings never grokked in 3 of 3 seeds. The author notes one task, one model, and few seeds, with mostly non-significant p-values.
Every outlet that ran it 1 article
r/MachineLearningCommunityLead
PBT doesn’t make grokking faster, but it can escape a hyperparameter range where nothing groks [P]