Read the original at HF Daily Papers
TokenRouter is a serving system for token-level LLM routing that uses request-centric programming and model-centric execution to dispatch requests asynchronously across subservers.
Carried by: HF Daily Papers. First seen: .