Live page · Day archive

On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation

Read the original at arXiv

Summary

Multi-teacher on-policy distillation consolidates capabilities from multiple teachers but exhibits incomplete generalization beyond the supervised distribution.

Carried by: arXiv, HF Daily Papers. First seen: .