Live page · Day archive

Latent-MOPD: Latent Multi-Teacher On-Policy Distillation

Read the original at arXiv

Summary

Researchers introduce Latent-MOPD, a method that trains language models using hidden states from multiple teachers. It outperforms baselines on nine benchmarks across math, code, and logic.

Carried by: arXiv, HF Daily Papers. First seen: .