Read the original at HF Daily Papers
Researchers introduce ReSAIL, a method that prevents performance collapse in iterative self-distillation by selecting informative steps and preserving teacher behavior, achieving a 22.5% average gain.
Carried by: HF Daily Papers. First seen: .