Cache-to-Cache: Direct Semantic Communication Between Large Language Models — part 4/4

Cleaned-text code points [47725, 58309). Bibliography omitted here; source.md includes it.

In Figure 12 , we present the effective rank of key and value caches across all the layers. The plot shows a continuous increase in the effective rank of value caches after applying C2C, especially in the shallow layers. Key caches after applying C2C also have a comparable effective rank and increase at deep layers.

#### A.4.2 Gating Behavior

We analyze the behavior of the learnable gates by contrasting models trained on general-purpose versus task-specific data. This comparison reveals markedly different gating dynamics across the two regimes.

General-purpose training . When C2C is trained on the OpenHermes-2.5 dataset, the learned key and value gates remain almost fully open. Across the three model combinations reported in Table 4 , the average gate activation ratio exceeds 98.21%. Despite this near-complete activation, we observe that in certain layers the dynamic weights are concentrated at very small values—for example, the average key weight in some layers falls below 0.1. This suggests that under general-purpose training, C2C leverages the dynamic weighting mechanism to modulate how much information is incorporated from the sharer on a per-query basis, effectively treating dynamic weights as the primary control signal while leaving most gates open.

Task-specific training . In contrast, when C2C is trained on the MMLU auxiliary_train split, the gates exhibit a much sparser activation pattern. Across model combinations shown in Table 7 , the average gate activation ratio drops to 52.67%. For the layers where gates do open, however, the dynamic weights are substantially larger, with most layers exhibiting average weights above 0.4. This indicates that under task-specific training, the gating mechanism selects a smaller subset of layers that are consistently useful, while the dynamic weights primarily regulate the contribution strength of these selected layers.

Overall, these findings highlight the adaptive interplay between gates and weights: general-purpose training favors broad gate activation with fine-grained modulation via weights, whereas task-specific training favors sparse gate activation with stronger reliance on the selected layers.

#### A.4.3 Outlier cases in inference time

Llama3.2 . We observe that the Llama3.2 model achieves significantly lower inference time compared to other baselines in Table 4 . This improvement can be attributed to two factors. First, the Llama3.2 model itself has faster inference speed due to its implementation. Second, under the Non-CoT evaluation prompts described in Section A.3.6 , the model tends to output only a single option letter (e.g., “A” or “B”), rather than a longer formatted string such as “The correct answer is A.” The shorter outputs further reduce the average decoding time, leading to the observed advantage.

Qwen2.5-Math . In contrast, the Qwen2.5-Math model exhibits considerably longer inference time, as shown in Table 7 . The primary cause is its tendency to ignore the Non-CoT evaluation and T2T prompts described in Section A.3.6 , producing verbose, step-by-step solutions rather than concise answers. To accommodate these long outputs and avoid truncation, we set both the maximum response length and the maximum communication length to 1024 tokens during evaluation. Under this configuration, the model decodes substantially more tokens on average, resulting in significantly longer inference time.

#### A.4.4 Example Model Output

To help better understand the response pattern of different models, we show the example response from different methods using the CoT prompt. We use Qwen2.5-0.5B as the Sharer model and Qwen3-0.6B as the Receiver model.

As shown in the following examples, Qwen3-0.6B misunderstood the question. It treated a probe for deeper meaning as merely a rephrasing task. Qwen2.5-0.5B-Instruct misinterpreted the question type altogether, taking it as sentence completion and answering the replaced question instead, with overly verbose reasoning. T2T altered the context of the question itself. In contrast, C2C maintained a solid understanding, recognizing that the rewritten sentence aimed to explore deeper moral reasoning, and accordingly selected the correct answer.

#### A.4.5 Training Cost Analysis

Here we present the training cost in Table 12 . As shown in the table, accuracy quickly increases in the first few hundred steps. With 300 training steps (less than 9 GPU hours), C2C achieves a comparable result to the final checkpoint.

Additionally, in Figure 13 we have plots of training loss and validation loss to compare the settings using Qwen2.5-0.5B or Llama3.2-1B as Sharer. The plot shows that for both settings, the training loss converges at 250 steps and the eval loss becomes stable at step 1000, suggesting that C2C across different model families incurs no extra computational cost.

#### A.4.6 Failure Mode Analysis

We have shown that in various settings, C2C, as a new communication paradigm, can increase the receiver model performance by using the sharer’s contextual understanding. However, in some specific cases, C2C can fail to do so as the contextual understanding from the Sharer model is not always accurate and can mislead the Receiver into generating wrong answers. As shown in Figure 7 , though in general the number of correctly answered questions increases, there are some questions that the receiver can answer correctly but are answered incorrectly after applying C2C. Below we show an example of an accounting question where Qwen3-0.6B generates the correct answer. However, when using T2T and C2C, Qwen2.5-0.5B conveys incorrect information, leading Qwen3-0.6B to the wrong answer. This type of failure is more likely when using a very weak Sharer with strong Receiver and can lead to a performance degradation as mentioned in the limitation part.

### A.5 C2C Extensions and Future Work

As an initial exploration, this work focuses on pairwise communication to establish a foundation for future research. Communication via KV-Cache among multiple models is a direction that requires future exploration, with different designs that can be employed. Here, we explore two settings that involve multiple Sharers and/or Receivers.

#### A.5.1 One Receiver, Multiple Sharers

We explore the scenario in which three models are involved in the communication. Here we set Qwen3-0.6B as the Receiver and use both Qwen2.5-1.5B-Math and Qwen3-4B-Base as Sharers. Here, we separately trained the C2C fusers from Qwen2.5-1.5B-Math to Qwen3-0.6B and Qwen3-4B-Base to Qwen3-0.6B. The result in Table 13 shows that without additional training, the receiver can use C2C to fuse two Sharers’ semantic information and improve the performance.

#### A.5.2 Multiple Receivers Multiple Sharers

We provide a possible path towards more efficient scaling, with O ⁡ ( N ) O(N) fusers instead of O ⁡ ( N 2 ) O(N^{2}) fusers to communicate among N N LLMs.

The design philosophy is to decompose M × N M\times N communication routes into M × 1 M\times 1 , then 1 × N 1\times N steps, with 1 1 being the unified latent space. This is achieved by using Projectors P s i P_{s_{i}} to transform the KV-Cache from different Sharer LLMs s 0 , … , s M − 1 s_{0},\dots,s_{M-1} into a unified latent representation KV-Latent L L , then using the C2C fuser F r j F_{r_{j}} to fuse them with each Receiver model r 0 , … , r N − 1 r_{0},\dots,r_{N-1} . C s i C_{s_{i}} represents the KV-Cache from model s i s_{i} . For M Sharers, N Receivers, we have M projectors and N fusers. Each sharer-receiver pair has its own gate. During training, all Sharer KV-Caches are projected into a unified hidden space with respective Projectors. C s i → P s i U s i , i = 𝕄 = { 0 , … , M − 1 } C_{s_{i}}\xrightarrow{P_{s_{i}}}U_{s_{i}},\quad i=\mathbb{M}=\{0,\dots,M-1\} Then, each of the N N Receivers fuses all the M M available information from this hidden space. ( U s i , C r j ) → F r j C r j , j = ℕ = { 0 , … , N − 1 } (U_{s_{i}},C_{r_{j}})\xrightarrow{F_{r_{j}}}C_{r_{j}},\quad j=\mathbb{N}=\{0,\dots,N-1\} This yields N N updated caches C r j ′ C_{r_{j}}^{\prime} , which can be optimized using the same SFT loss discussed in Section 3.3.4. Losses from all Receivers are summed to update all Projectors and fusers. During inference, the model supports both one-to-one and many-to-one (multi-sharer to single receiver) settings. As an initial exploration, we directly adopted the training data, fuser architecture, and hyperparameter settings from the pairwise fuser training, with vast possibilities left for future work.

In Table 14 we show the result of using Qwen2.5-Coder-0.5B-Instruct and Qwen2.5-Math-1.5B-Instruct as Sharers, Qwen3-0.6B and Qwen2.5-0.5B-Instruct as Receivers.

#### A.5.3 Incorporating with Agentic Flow

Here we show an exploration of C2C in agentic flow. We adopt the agentic flow for math problem-solving, following the workflow and prompts from Cognify ( He et al., 2025 ) . The workflow consists of a problem interpreter that analyzes the math problem and a problem solver that calculates the answer. For C2C, we use the prompt of the math solver, so the problem interpretation is done by KVCache fusion. Here we present the result on GSM8K. As shown in the table, the multi-agent flow increases the accuracy of math problem solving by 20.01. C2C has an accuracy of 62.55, increasing the accuracy by 1.37 when compared with the multi-agent flow using T2T. The performance can be further elevated to 78.01 by using T-C2C, which lets the math interpreter generate the interpretation and use C2C on both the question and the interpretation.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .