Cache-to-Cache Communication Eliminates LLM Text Bottlenecks

When multiple large language models work together in a multi-agent system, they typically exchange information through text tokens. One model compresses its understanding into words, and the next model reads those words and reconstructs the meaning. This process, known as text-to-text communication, introduces inefficiencies that researchers have been working to eliminate. A breakthrough technique called Cache-to-Cache (C2C) communication, published at ICLR 2026, proposes a fundamentally different approach: letting models share internal representations directly through their key-value caches.

The Text Bottleneck in Multi-Model Systems

Modern AI systems increasingly rely on multiple LLMs working in concert. In collaborative setups, different models take on specialized roles, exchanging analyses and partial solutions. In routing configurations, an orchestrator switches between models with different capabilities and cost profiles as a conversation progresses. Both architectures share a common vulnerability: the text handoff.

Researchers from Tsinghua University and collaborating institutions identified three critical problems with text-based communication between models:

• Information loss — A model stores context as high-dimensional numerical states internally. To communicate through text, it must compress that rich representation into a sequence of discrete tokens. The receiving model then interprets those tokens and builds its own approximation of what the first model knew. This compression-decompression cycle inevitably degrades information.
• Semantic ambiguity — Text instructions can be imprecise. The researchers demonstrated this with a Coder model and a Writer model collaborating on an HTML document. The Coder understood that a paragraph tag marked a specific structural location, but its textual instruction failed to convey that precision. The Writer placed content in the wrong position.
• Computational overhead — The sending model must generate its message token by token, a sequential process that takes time. Those tokens then inflate the receiving model’s input context, requiring additional processing before it can respond. In systems with long contexts or frequent agent handoffs, this overhead compounds rapidly.

How Cache-to-Cache Works

C2C bypasses the text bottleneck by using a representation that LLMs already create during inference: the key-value cache. When a model processes a prompt, it runs a prefill stage that converts input tokens into internal key and value representations. These cached states encode the model’s derived understanding of the context and are reused throughout response generation.

The C2C architecture designates one model as the Sharer and another as the Receiver. Both models process the same shared context independently, building their own KV caches. A trained cache fuser then maps the Sharer’s cache into the Receiver’s representation space and combines it with the Receiver’s existing cache. A learned gating mechanism controls which layers of the Receiver receive the additional information, allowing the system to selectively inject insights from the Sharer without overwhelming the Receiver’s own processing.

In routing systems, this means the Receiver can incorporate knowledge from the Sharer’s cache without any intermediate text message. In collaborative systems, C2C creates an additional communication channel that runs alongside traditional text exchange, enriching the information flow.

Benchmark Results and Performance Gains

The research team evaluated C2C on established reasoning and knowledge benchmarks including MMLU-Redux, OpenBookQA, ARC-Challenge, and C-Eval. Using Qwen3-0.6B as the Receiver paired with three different Sharer models, C2C delivered measurable improvements:

• Accuracy improvement — C2C increased the Receiver’s average accuracy by 9.6 to 11.9 percentage points over the baseline of the Receiver working alone, outperforming text-to-text communication by 3.1 to 5.4 percentage points.
• Latency reduction — Speed improvements varied significantly by model pair, ranging from 1.51x to 14.41x faster than text-based communication. The most dramatic result came with Qwen3-4B Base as the Sharer, where the text baseline was particularly slow due to the model generating longer-than-expected messages.
• Cross-architecture compatibility — C2C worked across models with different architectures and specializations. Tests included Gemma-to-Qwen, Qwen Math-to-Qwen, and Qwen Coder-to-Qwen combinations, with C2C outperforming text communication across all heterogeneous configurations tested.

In one illustrative experiment, text-to-text communication required the Sharer to generate an average of 80 communication tokens, taking 1,312 milliseconds. The resulting message also increased the token count the Receiver had to process. C2C accomplished the same information transfer in approximately 90 milliseconds of cache fusion, with no additional tokens generated.

Deployment Requirements and Limitations

C2C does not require fine-tuning either participating model. The researchers freeze both the Sharer and Receiver and train only the cache-fusion module using a standard next-token prediction objective. This significantly reduces the engineering burden compared to approaches that require model retraining.

However, cross-model communication introduces its own challenges. Different models may use different tokenizers, have different numbers of transformer layers, and produce representations of different sizes. C2C addresses this by aligning tokens generated by different tokenizers and mapping corresponding transformer layers before fusing caches. There is also an upfront cost to training the bridge module for each model pair.

The team has released C2C code and configurations on GitHub under an Apache 2.0 license, along with pretrained cache fuser checkpoints on Hugging Face, making the technique accessible to researchers and developers.

One important caveat is that C2C requires access to model internals. Because it must read, transform, and replace KV-cache states, it only works with inference stacks that expose those internals. This makes it relevant today primarily to teams that control their own inference infrastructure, not applications built solely on closed API access.

Broader Implications for Machine Learning Architecture

C2C joins a growing body of research challenging the assumption that text should remain the default interface within multi-model AI systems. Nvidia has explored cross-model KV-cache transfer for model switching during long sessions, reporting 2.7x to 25x speed improvements over re-prefilling on compatible model pairs. Another framework called RecursiveMAS replaces textual messages between agents with continuous latent representations, achieving up to 2.4x faster inference and a 75.6% reduction in token usage.

These approaches differ in mechanism but share a common thesis: models may communicate more efficiently when they do not have to translate everything into text first. For enterprise teams building multi-agent pipelines or model routing systems, the communication layer itself is emerging as a viable optimization target, one that could yield both cost savings and performance improvements.

As multi-model architectures become standard in production AI systems, techniques like C2C could reshape how we think about inter-model communication. The research community’s shift from text-only handoffs to direct representation sharing signals a maturing understanding of where inefficiencies truly live in complex AI pipelines. While C2C remains a research result measured on benchmarks rather than production agent workloads, its open-source release and demonstrated cross-architecture compatibility make it a development worth watching for any organization investing in multi-model AI infrastructure.

---

Edited by Palawan @QUE.COM
Website: https://QUE.COM Intelligence
Sponsored by: https://MAJ.COM AI Autonomous