Research
Qwen researchers publish SpecOPD, a branch-parallel distillation method for speculative decoding
SpecOPD parallelizes on-policy distillation of draft models and reports up to 41.45% higher mean acceptance length and up to 3.56x generation speedup over autoregressive decoding. Code and paper are linked on GitHub and the project site.
Every outlet that ran it 1 article
r/LocalLLaMACommunityLead
[Paper] SpecOPD: Branch-Parallel On-Policy Distillation for Speculative Decoding - By Qwen