Live page ยท Day archive

Order-Only On-Policy Distillation

Read the original at HF Daily Papers

Summary

Order-Only On-Policy Distillation trains student models using teacher token rankings, outperforming standard methods on math benchmarks by up to 5.6 points.

Carried by: HF Daily Papers. First seen: .