Find Similarity
发现相似性
Identifies long-range similarity in KV representations across extended contexts.
识别长上下文 KV 表征中的长程相似性。
ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
ResidualKV：面向高效长上下文推理的残差式 KV Cache 压缩
Previously titled DeltaKV. This overview follows arXiv v2 (September 30, 2026). Submitted to IEEE TPAMI; under review.
原名 DeltaKV。本页按 2026 年 9 月 30 日的 arXiv v2 更新。IEEE TPAMI 在投，尚未接收。
First author. KV storage reduced to as little as about 1/8 with near-lossless quality, using a reference-plus-residual representation. See the benchmark-specific figures below.
第一作者。通过参考 KV + 残差表示，在接近无损质量下将KV 缓存最低约为原来的 1/8；各实验设置的具体结果见下方。
ResidualKV studies long-range similarity in KV representations and compresses long-context LLM memory by storing quantized residuals relative to globally retrieved references instead of discarding tokens.
ResidualKV 研究 KV 表征中的长程相似性，通过将 KV 状态编码为相对全局检索参考的量化残差来压缩长上下文 LLM 显存，而不是直接丢弃 token。
Overview
论文概览
ResidualKV belongs to the efficient long-context LLM inference line. It addresses the memory pressure of KV cache by exploiting redundancy across distant context positions rather than pruning away context outright.
ResidualKV 属于高效长上下文 LLM 推理方向，针对 KV cache 显存压力问题，利用远距离上下文位置之间的冗余，而不是直接剪掉上下文。
Store Residuals
存储残差
Encodes quantized residuals relative to globally retrieved references to reduce memory.
将 KV 状态编码为相对全局检索参考的量化残差，以降低显存。
Improve Serving
提升服务效率
Targets near-lossless long-context inference with better throughput.
面向近无损的长上下文推理和更高吞吐。
Related Research Areas
相关研究方向
ResidualKV connects residual-based representation compression, KV-cache memory reduction, and sparse-first LLM serving systems such as SparseEngine.
ResidualKV 连接残差式表示压缩、KV-cache 显存优化，以及 SparseEngine 这类稀疏优先的大模型服务系统。
ResidualKV
KV cache compression
long-context LLMs
residual-based compression
long-range similarity
efficient inference
SparseEngine
Key Takeaways
核心要点
- Problem: long-context LLM inference makes KV cache a dominant memory bottleneck.
- 问题：长上下文 LLM 推理中，KV cache 会成为主要显存瓶颈。
- Idea: use long-range similarity to encode current KV states as quantized residuals relative to globally retrieved references.
- 核心想法：利用长程相似性，把当前 KV 状态编码为相对全局检索参考的量化残差。
- Evidence: arXiv v2 reports near-full-cache quality with 13%-16% KV storage and 30% attention computation on LongBench. Matched-budget multimodal experiments use 8%-10% storage and 10% computation; decoding speedups reach 1.5× with KV-cache quantization and 3.4× without it.
- 实验结果：arXiv v2 在 LongBench 上使用 13%-16% KV 存储与 30% 注意力计算，保持接近完整缓存的质量。在匹配预算的多模态实验中使用 8%-10% 存储与 10% 计算；启用和不启用 KV-cache 量化时，解码最高分别加速 1.5× 与 3.4×。
Resources
相关资源
Revised paper and implementation for ResidualKV; the Chinese write-up covers the earlier DeltaKV version.
ResidualKV 修订版论文与代码；中文解读对应早期 DeltaKV 版本。