Cache-to-Cache: Direct Semantic Communication Between Large Language Models — part 1/4
Cleaned-text code points [0, 15977). Bibliography omitted here; source.md includes it.
# Cache-to-Cache: Direct Semantic Communication Between Large Language Models
###### Abstract
Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model’s KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5× speedup in latency. Our code is available at https://github.com/thu-nics/C2C .
## 1 Introduction
With the rapid progress of Large Language Models (LLMs) ( Guo et al., 2025 ; Yang et al., 2025a ; OpenAI, 2025 ) , they are now applied across increasingly diverse domains and tasks. To meet versatile demands, LLMs are trained with distinct focuses, such as coding ( Hui et al., 2024 ) , mathematics ( Yang et al., 2024a ) , visual understanding ( Bai et al., 2025 ) , edge computing ( Zhang et al., 2024b ) , and so on. Meanwhile, general-purpose LLMs can also simulate specialized capabilities through prompt engineering, enabling flexible role adaptation across downstream applications.
Leveraging the diversity of LLMs, many multi-LLM systems are proposed to further enhance overall performance and efficiency ( Guo et al., 2024 ; Tran et al., 2025 ) . In collaborative multi-LLM systems ( Li et al., 2023 ; Wu et al., 2023 ) , LLMs are assigned distinct roles and proactively exchange text messages. Mirroring human collaboration, these systems accumulate partial understandings or sub-solutions from different agents via verbal communication. They harness the collective capabilities of multiple LLMs to solve complex problems that a single model cannot. By contrast, routing-based multi-LLM inference systems rely on passive context inheritance rather than active message exchange. These systems coordinate models of varying parameter sizes or reasoning depths for more dynamic and efficient responses ( Li et al., 2024 ; Fu et al., 2025a ; Ong et al., 2024 ; OpenAI, 2025 ) . Downstream models inherit the context from preceding models in multi-round conversations, then generate follow-up responses to the new questions based on their own understanding of the conversation history.
However, current text-to-text (T2T) interfaces restrict information exchange among LLMs, particularly when conveying rich or diverse semantic interpretations of a shared context. As illustrated in Figure 2 , these limitations arise from several inherent constraints of T2T communication. First, as a low-bandwidth medium, text introduces an information bottleneck. The high-dimensional internal representations must be repeatedly compressed into linear strings and then decompressed by the receiver LLM. When models differ in knowledge or assigned roles, some signals may be irrecoverable (e.g., interpreting <p> as a section marker). Second, natural language is inherently ambiguous, with idioms, underspecified references, and vague expressions. Although recent agent protocols aim to standardize text messages ( Anthropic, 2024 ; Surapaneni et al., 2025 ) , rigid templates remain insufficient for flexible, open-domain collaboration. Third, T2T communication incurs noticeable latency. Every exchange requires exhaustive, token-by-token decoding of contextual explanations in sequence. These limitations motivate a key question: Can LLMs communicate beyond text?
In this work, we explore using KV-Cache as the medium for LLM communication. KV-Cache is a naturally richer representation than text. It also enables fully parallel communication through direct projection, avoiding the slow sequential decoding in text exchanges. Our oracle experiments show that (1) enriching KV-Cache under the same context length increases accuracy, (2) KV-Cache is convertible between LLMs, (3) different LLMs encode distinct semantic understandings and contextual knowledge of the same input, reflecting their complementary strengths.
Encouraged by these findings, we propose Cache-to-Cache (C2C), a new paradigm for richer and faster multi-LLM communication. As shown in Figure 1 (b), C2C projects the KV-Cache from a source model into the space of a target model and merges them through a neural cache fuser. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the T2T paradigm by approximately 3.1-5.4%, while delivering an average 2.5 × \times speedup in latency.
## 2 Related Work
### 2.1 KV-Cache Sharing and Reuse
Based on the similarity of KV-Cache between layers, intra-model cache sharing methods ( Yang et al., 2024b ; Wu and Tu, 2024 ; Sun et al., 2024 ; Brandon et al., 2024 ; Wu et al., 2025 ) have been proposed to reuse shallow layers’ KV-Cache for deeper layers to accelerate single LLM inference. Another research focus is to reuse a portion of KV-Cache (e.g., common prefix, reference documents) for the same model in multiple user queries ( Bang, 2023 ; Ye et al., 2024 ; Yao et al., 2024 ; Qin et al., 2024 ; Yang et al., 2025b ; Ye et al., 2025 ; Eyuboglu et al., 2025 ) . DroidSpeak ( Liu et al., 2024a ) extends cache reuse to models fine-tuned from the same base model. Unlike existing works that focus on computational efficiency through cache reuse, our approach leverages the KV-Cache as a medium for semantic transfer between LLMs. Furthermore, unlike existing cache sharing methods that are restricted to only a single model or models with identical structure and size, our method supports sharing across different model families and varying model sizes.
### 2.2 Multi-LLM Systems
Collaborative multi-LLM systems . Collaborative systems treat multiple LLMs as peers that exchange information to improve collective performance. Chain-of-Agents ( Zhang et al., 2024c ) and MetaGPT ( Hong et al., 2023 ) create sequential message flows where agents directly communicate using natural language interfaces. Mixture-of-Agents ( Wang et al., 2024 ) , DyLAN ( Liu et al., 2024b ) , and Owl ( Hu et al., 2025 ) introduce layered communication architectures. Target LLMs aggregate messages from multiple models using voting or summarization mechanisms. Multi-agent debate methods ( Estornell and Liu, 2024 ; Liang et al., 2024 ; Du et al., 2023 ) involve iterative communication rounds, letting LLM agents discuss and refine responses. Recent works such as MCP Anthropic (2024) and A2A Surapaneni et al. (2025) establish formal text protocols beyond natural language, standardizing agent interaction and tool usage in collaborative multi-LLM systems. These approaches rely on text-level interfaces, where communication requires one model to generate text token-by-token and another to ingest it as input. Our work explores a deeper and more efficient collaboration ( Pidkuiko and Starkov, 2025 ; Zheng et al., 2025b ) by directly sharing internal KV-Cache representations.
Routing-based multi-LLM inference systems . To accelerate LLM inference, several systems leverage multiple models with different capabilities and costs. Dynamic model selection methods ( OpenAI, 2025 ; Ong et al., 2024 ; Feng et al., 2024 ; Ning et al., 2024 ) route queries to different models with varying sizes and configurations to balance efficiency and performance. Token-level routing methods ( Zhang et al., 2024a ; Shen et al., 2024 ; Zheng et al., 2025a ; Fu et al., 2025a ) enable finer-grained selection, utilizing smaller models for simple token generation within the reasoning process of complex tasks. While these systems achieve efficiency through strategic model switching, they either completely drop context from other models, or simply rely on their own understandings of the context. Without understanding sharing, smaller models cannot benefit from the richer representations already computed by larger models.
## 3 Method
### 3.1 Preliminaries
LLM inference . Autoregressive LLM inference involves two stages: prefill and decode . Prefill encodes the full input to produce the first output token; decode then generates subsequent tokens iteratively using the last token and the cached key–value (KV) states. Formally, let X [ 0 : n ] = [ x 0 , … , x n − 1 ] X_{[0:n]}=[x_{0},\dots,x_{n-1}] be the input token sequence. After prefill, LLM produces a per-token KV-Cache 𝒞 ( X [ 0 : n ] ) = [ c 0 , … , c n − 1 ] ∈ ℝ n × d \mathcal{C}(X_{[0:n]})=[c_{0},\dots,c_{n-1}]\in\mathbb{R}^{n\times d} . For notation brevity, d d denotes the KV dimensionality that is flattened from all layers into a single vector per token. The range subscripts are omitted when clear. During decoding, with the current token y i y_{i} and caches from the input and the generated prefix, the next token is predicted as y i + 1 = 𝒫 ( y i ∣ 𝒞 ( X ) ⊕ 𝒞 ( Y [ 0 : i ] ) ) , y_{i+1}=\mathcal{P}\!\left(y_{i}\mid\mathcal{C}(X)\,\oplus\,\mathcal{C}(Y_{[0:i]})\right), (1) where ⊕ \oplus denotes sequence-wise concatenation. The cache updates as 𝒞 ( Y [ 0 : i + 1 ] ) = 𝒞 ( Y [ 0 : i ] ) ⊕ 𝒞 ( y i ) \mathcal{C}(Y_{[0:i+1]})=\mathcal{C}(Y_{[0:i]})\oplus\mathcal{C}(y_{i}) .
LLM communication . In LLM communication scenarios, we define the LLM that provides contextual understanding or knowledge as Sharer , and the one that utilizes it as Receiver .
### 3.2 Oracles for Cache-to-Cache Communication
We aim to explore whether LLMs can have direct semantic communication through KV-Cache. Specifically, we design two oracle experiments to answer the following questions: (1) Benefit : can a model’s capabilities be improved through KV-Cache semantic enrichment without extending sequence length? (2) Convertibility : can the KV-Cache of one model be effectively utilized by another model?
#### 3.2.1 Cache Enrichment Oracle
To validate the benefit of cache enrichment, we first explore whether the semantic quality of KV-Cache can be improved without increasing its size. Few-shot prompting suggests this might work: providing exemplars E E before the question X X often improves accuracy. But does this arise from attending to more context tokens, or from E E enriching how X X is embedded in KV-Cache?
We evaluate this via three setups: (1) Direct : prefill on X X only and decode with 𝒞 ( X ) \mathcal{C}(X) ; (2) Few-shot : prefill on E ⊕ X E\oplus X and decode with 𝒞 ( E ⊕ X ) \mathcal{C}(E\oplus X) (longer cache); (3) Oracle : prefill on E ⊕ X E\oplus X but discard the exemplar segment and keep only the question-aligned slice 𝒞 ∗ ( X ) = 𝒞 [ | E | : | E | + | X | ] ( E ⊕ X ) , \mathcal{C}^{*}(X)\;=\;\mathcal{C}_{[\,|E|:|E|+|X|\,]}(E\oplus X), (2) so that decoding uses a question-length cache with no extra tokens. Here, | ⋅ | |\cdot| denotes sequence length. In Equation 1 , this corresponds to substituting 𝒞 ( X ) \mathcal{C}(X) with 𝒞 ∗ ( X ) \mathcal{C}^{*}(X) before decoding.
Comparing Direct and Oracle isolates the effect of cache enrichment: any gain arises from the richer question embeddings induced by E E , not from attending to additional token caches as in Few-shot . As shown in Table 1 , the Oracle setup improves response quality at the same cache length.
Additionally, we analyze how cache enrichment affects different transformer layers. Our findings show substantial variation across layers: while some layers benefit from cache enrichment, others experience performance degradation (details in Appendix A.2.1 ). Furthermore, these layer-wise effects accumulate as more layers are augmented. As shown in Figure 4 , selectively applying cache enrichment to the top-performing layers (e.g., top-5) yields slightly higher accuracy than enriching all layers, while targeting the worst-performing layers leads to accuracy decline. This finding guides the gating mechanism of our cache fuser (Section 3.3.2 ).
#### 3.2.2 Cache Transformation Oracle
To verify that one model’s KV-Cache can be utilized by another, we conducted cross-model transformation experiments. We train a 3-layer MLP to map the KV-Cache from a source LLM (Qwen3-4B) to a target LLM (Qwen3-0.6B), with more setups detailed in Appendix A.3.2 .
T-SNE visualizations in Figure 3 reveal that the raw KV-Caches of the two LLMs are far apart in representation space. After transformation, the mapped KV-Cache lies inside the target model’s representation space. These results demonstrate that KV-Caches from different models are generally convertible, as the transformed cache is covered by the target model’s representation space.
One thing to note is that the transformed cache occupies only a smaller subset of the target’s space. It indicates that the source model’s semantic information cannot fully cover the target’s, despite the source being larger. This reflects inherent differences in how each model encodes context. Another observation also supports this interpretation: the correct-answer sets of different models exhibit limited overlap (Figure 7 ), despite the comparable aggregated accuracy of respective models. These findings suggest that if specialized contextual understanding from different models can be successfully projected and fused, it may harness the complementary strengths of the respective models.
### 3.3 C2C Design
#### 3.3.1 Overview
Building on the oracle experiments, we propose the C2C scheme. Its core objective is to extract useful contextual understanding or knowledge from one model (the Sharer) and fuse it into another model (the Receiver).
In general, the C2C paradigm contains a set of key/value cache fusers ℱ \mathcal{F} and a layer mapping strategy 𝒢 \mathcal{G} . During the prefill stage, fuser ℱ n \mathcal{F}_{n} takes the n n th layer cache of the Receiver Model 𝒞 n ( X ) \mathcal{C}_{n}(X) and the corresponding 𝒢 ( n ) \mathcal{G}(n) th layer cache of the Sharer Model 𝒞 𝒢 ( n ) 𝒮 ( X ) \mathcal{C}^{\mathcal{S}}_{\mathcal{G}(n)}(X) and generates the corresponding fused cache with residual connection: 𝒞 ℱ = { 𝒞 n ( X ) + ℱ n ( 𝒞 n ( X ) , 𝒞 𝒢 ( n ) 𝒮 ( X ) ) } n = 1 N \mathcal{C}^{\mathcal{F}}=\left\{\mathcal{C}_{n}(X)+\mathcal{F}_{n}\!\left(\mathcal{C}_{n}(X),\;\mathcal{C}^{\mathcal{S}}_{\mathcal{G}(n)}(X)\right)\right\}_{n=1}^{N} (3)
During decoding, with the current token y i y_{i} and caches from the input and the generated prefix, the next token is predicted as:
y i + 1 = 𝒫 ( y i | 𝒞 ℱ ( X ) ⊕ 𝒞 ( Y [ 0 : i ] ) ) y_{i+1}=\mathcal{P}\left(y_{i}\middle|\mathcal{C}^{\mathcal{F}}(X)\oplus\mathcal{C}(Y_{[0:i]})\right) (4)
#### 3.3.2 Fuser Structure
To enhance the Receiver’s KV-Cache without destructive overwriting of its information, the fuser is designed under a residual integration principle. As shown in Figure 5 , it contains three key modules:
(1) Projection module concatenates the Receiver’s KV-Cache with the Sharer’s KV-Cache, then processes the concatenated features through a projection layer followed by a feature fusion layer.
(2) Dynamic weighting module applies an input-aware head modulation layer to dynamically reweight the projected information.