[DeepSeek and Kimi Are Aggressively Reducing Cache Usage, NVIDIA Enables Cross-Model Cache Reuse] According to monitoring by Dongcha Beating, over the past two years, large model inference optimization has been focusing on reducing KV Cache. KV Cache stores intermediate computation results after a model processes the context, and the longer the content, the more memory and bandwidth it consumes. DeepSeek-V2, leveraging MLA, reduces KV Cache by 93.3% compared to DeepSeek 67B; Kimi Linear further reduces it by up to 75% compared to full MLA. Everyone is working to make long-context processing more cost-effective. However, these solutions face a significant limitation: cache is typically only reusable within the same model. When an agent switches from a smaller model to a larger one, the target model has to recompute tens of thousands or even hundreds of thousands of tokens from scratch. The more frequent the model routing, the harder it becomes to ignore this redundant computation. NVIDIA's latest paper aims to break down this barrier. The team discovered a clear linear relationship in the cache between different-sized models within the same family that share a matching KV structure. By calibrating with 500 segments of text, each containing 1024 tokens, they were able to derive a mapping that allows the KV Cache computed by one model to be transferred for use by another model. For example, when switching from Qwen3-14B to 32B, recomputing a 32K context would take about 7 seconds, but converting the cache takes only about 0.28 seconds—approximately 25 times faster. In other words, in the future, smaller models could even handle the "reading" phase first: processing long contexts into KV Cache. When greater computational power is needed, the cache can then be transferred to a larger model, allowing it to immediately start "thinking" and generating outputs. This could save even more computational resources during model routing. Currently, running simple tasks on smaller models primarily saves on generation costs. However, when switching to larger models, long contexts often need to be recomputed by the larger model. If cross-model KV Cache becomes mature, even this prefill step could be handled by smaller models. Agents could maintain context over time using cost-effective smaller models, only waking up larger models for more complex tasks—and without having to "reload" from scratch each time. [Original Article Link]