AI News Pulse

Research

Order-Only On-Policy Distillation trains students on teacher token rankings instead of probabilities

O3PD matches the order of a teacher's top-K tokens rather than their probability values, making it invariant to teacher temperature. It beats standard on-policy distillation on average across HMMT25, AIME24 and AIME25 in all six teacher-student pairs, by up to 2.9 points, with up to 5.6 points on A…

1outlet ran itRead at Hugging Face Papers
Status
Research
Published
That day
Outlets
1
Written by
Claude How

Every outlet that ran it 1 article

  1. Hugging Face PapersResearchLead

    Order-Only On-Policy Distillation

All news