Read the original at HF Daily Papers
Meta Expert proposes cross-layer expert sharing for Mixture-of-Experts models, reducing parameter redundancy and improving accuracy while matching Vanilla MoE performance using only 70 to 80 percent of the experts.
Carried by: HF Daily Papers. First seen: .