When an agentic AI system hands a task from a small model to a larger one—or passes it back down again—it pays a steep tax: the receiving model has to recompute the entire conversation from scratch, driving up compute costs and latency. This is a major bottleneck for enterprises building long-horizon, multi-LLM workflows. Nvidia researchers have a solution that sounds almost too simple to be true: simple linear algebra.
The researchers introduced a cross-model KV cache transfer technique that directly maps the prefilled KV cache from a source model into the target model. This technique aligns with real-world agentic applications where large contexts accumulate across many turns. For real-world AI applications, cross-model KV cache transfer can reduce compute costs and latency on long-running, multi-LLM workflows—and it does so with simple linear math, not an expensive deep learning model.
Experiments show that, on compatible model pairs, this linear mapping process runs 2.7 to 25 times faster than recomputing the conversation while retaining up to 98% of the target model’s standalone accuracy.
Why model swapping is so expensive
Examining how LLMs handle memory helps understand why multi-model workflows hit a performance wall in production. When an LLM receives a prompt, it must first execute the “prefill” stage, which is the initial forward pass that computes the keys and values for all input tokens and populates the Key-Value (KV) cache.
After that, it enters the “decode” phase, where it computes and generates the next tokens in the sequence. During this phase, the model reads from this KV cache to predict new tokens one by one, bypassing the need to re-evaluate the entire history of the conversation for each new token.
In multi-turn conversations or long-horizon agentic sessions, the context gradually becomes longer. Because the computational cost of the prefill stage scales directly with both model size and input length, processing these long sessions becomes increasingly expensive and introduces significant latency if the KV cache is invalidated.
This invalidation happens whenever the AI system tries to swap models mid-session, such as routing a complex reasoning step to a larger model or dropping to a smaller model to save costs. Because different LLMs have different architectures, they expect their cache inputs in different formats. As a result, any model switch forces the receiving model to repay the entire prefill cost from scratch to recompute the KV cache for the accumulated context.
Mapping memory with linear regression
The Nvidia researchers studied cross-model KV cache transfer to see how developers can transform the KV cache of one model into the expected format of another without running the prefill phase again.
If solved, cross-model KV cache transfer has benefits in both directions. Small-to-large model transfer upgrades the quality of the output. For example, a cheap, small model handles the routine parts of an agentic workflow but struggles with a complex reasoning problem, and you map the KV cache to a larger model and continue the process seamlessly.
Large-to-small model transfer reduces compute costs. A highly capable, large model might be used to unpack a massive, complex system prompt or synthesize a dense PDF at the start of a session. Once the heavy lifting is done, the session’s KV cache is mapped down to a smaller, more economical model to handle the rapid-fire, conversational turns that follow.
There have been previous efforts to solve the KV cache transfer problem, but they suffer from key limitations: the need for expensive gradient-based training or very strict architectural constraints.
The key finding of the Nvidia study is that cross-model KV cache is a significantly linear structure. This means the mapping can be done with simple algebra tricks and without the need for heavy neural network training. For example, when experimenting on KV cache transfer from a 14-billion parameter Qwen3 model to a 32-billion parameter version, the authors discovered that a simple linear regression mapping from one source layer to a target layer can recover 56% of the variance in the target’s keys and 32% of the variance in its values. When combining multiple source layers, those numbers climbed to 79% and 65% respectively.
For this initial study, the authors restricted their focus to within-family transfers, such as transitioning between different-sized models in the Qwen, Llama, or Ministral families. These models share tokenizers, training data DNA, and core architectural styles but differ in size and depth. However, the researchers note the technique could eventually be expanded to cross-family transfers, mismatched KV head counts, or hybrid architectures that blend standard attention with other memory mechanisms.
The implications for enterprise AI are significant. Organizations running multi-model agentic workflows can now do so with far less computational overhead. The humble linear regression—math that predates modern computing by over two centuries—turns out to be exactly what today’s most sophisticated AI systems needed.