Live page · Day archive

LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time

Read the original at arXiv

Summary

LowRAM compresses KV caches using low-rank keys and amortized values, achieving up to five times higher decode throughput with four times compression.

Carried by: HF Daily Papers. First seen: .