Live page · Day archive

LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

Read the original at HF Daily Papers

Summary

LoopFill is a training-free speculative decoding algorithm for looped transformers that achieves an average speedup of 5.85 times on Ouro-2.6B and 5.13 times on Huginn-3.5B.

Carried by: HF Daily Papers. First seen: .