Cache-to-Cache: Direct Semantic Communication Between Large Language Models — part 2/4

Cleaned-text code points [15977, 31799). Bibliography omitted here; source.md includes it.

(3) Learnable gate introduces a trainable per-layer gate value that decides whether to inject the Sharer’s context. The gate applies a Gumbel-sigmoid with temperature annealing to smoothly transition from differentiable during training to binary at inference.

We also explore a more complex yet potentially more powerful fuser variant in Appendix A.1.3 .

#### 3.3.3 Model Alignment

Fusing KV-Caches across model families and sizes requires alignment at two levels: tokens and layers. For token alignment , different tokenizers may produce slightly varied token sequences for the same input. We align them by decoding each Receiver token into its string form and re-encoding it using the Sharer’s tokenizer. When one-to-many mappings occasionally occur, we select the Sharer token with maximal string coverage to preserve information. For layer alignment, we adopt a terminal alignment strategy: the final layers of both models are aligned first, then the penultimate layers, and so on in reverse order until reaching the shallower model’s first layer. Details in Appendices A.1.1 and A.1.2 .

#### 3.3.4 Training Scheme

During training, we freeze both the Sharer and Receiver models, training only the C2C module for KV-Cache fusion. We employ standard next-token prediction loss on the Receiver’s response predictions, similar to supervised fine-tuning (SFT). The key difference is that the Receiver predicts responses conditioned on fused KV-Cache rather than its own.

The training procedure consists of three stages: (1) Forward: both models encode the input context to produce their respective KV-Caches. (2) Fusion: the C2C module fuses both KV-Caches and replaces the Receiver’s cache. (3) Supervision: the Receiver prefills the response using the fused cache, and gradients backpropagate through C2C to minimize prediction loss.

## 4 Experiment

### 4.1 Setup

We brief the experiment setups here, with more details in Appendix A.3 .

Models . We evaluate C2C across various model families, including Qwen2.5 ( Yang et al., 2024a ; Hui et al., 2024 ) , Qwen3 ( Yang et al., 2025a ) , Llama3.2 ( Dubey et al., 2024 ) , and Gemma3 ( Team et al., 2025 ) . To test generalizability, we select different configurations for the Sharer-Receiver model combinations, including models of different generations (Qwen3 and Qwen2.5), different families (Qwen, Llama, and Gemma), different sizes (0.6B to 14B), different specializations (general, code, and math model), and different training stages (pretrained and instruction fine-tuned models). For ablative and diagnostic analyses (scaling behavior, ablation study, behavior analysis), we fix the Receiver and Sharer to Qwen3 models unless otherwise specified. This consistency eliminates confounders from model alignment and isolates the core impact of C2C.

Baselines . We compare C2C with two LLM collaboration methods to contextualize performance: (1) Text-to-Text (T2T) communication: models collaborate through an analyze-then-respond hand-off for each query. The Sharer generates an analytical text of key information to solve the input question. This text is concatenated with the original question and fed to the Receiver to mirror standard collaborative pipelines. Corresponding prompts are in Appendix A.3.6 . (2) Query-level routing ( Ong et al., 2024 ) : models collaborate by selecting the appropriate LLM for different queries. We also include individual model performance (Sharer or Receiver alone) to establish a lower bound for collaborative gains.

Benchmarks . We evaluate on four widely used benchmarks spanning reasoning, knowledge, and language domains to ensure comprehensive coverage. OpenBookQA ( Mihaylov et al., 2018 ) for fact-based reasoning, MMLU-Redux ( Gema et al., 2025 ) for knowledge in the general domain, ARC-Challenge (ARC-C) ( Clark et al., 2018 ) for scientific and logistic reasoning, and C-Eval ( Huang et al., 2023 ) for comprehensive knowledge in the Chinese domain.

Training dataset . To ensure the generalizability of C2C, we utilize the first 500k samples of the OpenHermes2.5 Dataset ( Teknium, 2023 ) , a general finetuning dataset, to train C2C fusers. To reduce training cost, we use MMLU as the training set for scaling behavior and behavior analysis experiments, unless otherwise specified.

Evaluation settings . We use average accuracy as the performance metric. We use text generation and answer extraction as the evaluation mode for C2C and baselines, with the max generation length set to 64 for multi-choice benchmarks. All experiments are conducted in the zero-shot setting with zero generation temperature to ensure reproducibility. We use average inference time as the efficiency metric, measured using a single NVIDIA A100 GPU with batch size one.

### 4.2 Main Results

Performance . As shown in Table 4 , C2C consistently improves the Receiver model performance across different settings and benchmarks. Representative example output is provided in Appendix A.4.4 . After applying C2C, accuracy increases by an average of 11.00%, 9.64%, and 11.88% across the three Sharers. Compared with text-to-text communication, C2C achieves an average accuracy increase of 5.36%, 4.15%, and 3.06%. Query-level routing prioritizes efficiency but limits accuracy to the better of the two original models. Notably, Qwen3-4B Base as the Sharer often ignores instructions, resulting in poor standalone performance and excessively long T2T communication times. In contrast, C2C bypasses this issue, highlighting an interesting use case in which a weaker instruction-tuned Receiver can leverage a stronger base model’s knowledge via C2C, even when base model cannot follow instructions. We also explore strong-to-weak communication (details in Appendix A.2.2 ) using Qwen3-4B as the Sharer, showing that C2C effectively enables the Receiver to benefit from a stronger Sharer.

Efficiency . As shown in Table 4 , C2C achieves significant speedups of 3.46 × \times , 1.51 × \times , and 14.41 × \times over T2T by eliminating intermediate text generation. As detailed in Table 3 , T2T requires the Sharer to decode 80 output tokens, incurring 1312ms of decoding overhead, whereas C2C replaces this sequential decoding with parallel cache fusion in 90ms. Further analysis of Llama3.2-1B’s exceptionally fast inference is provided in Appendix A.4.3 .

### 4.3 Scaling Behavior

Scaling sequence lengths . We evaluate how C2C scales with respect to sequence length on long-context tasks from the LongBenchV1 benchmark. All C2C fusers are trained and tested on different sets of LongBenchV1. As shown in Table 6 , C2C consistently outperforms text-to-text communication across all sequence-length intervals. This demonstrates C2C’s advantages across input length ranges. More detailed setups and results are in Appendices A.2.2 and A.3.4 .

Scaling model sizes . We investigate how C2C scales with respect to the Sharer and Receiver model sizes. All C2C fusers are trained on MMLU’s auxiliary train split and evaluated on MMLU-Redux. As shown in Figure 6 , the x x -axis denotes Sharer size (Qwen2.5-Instruct series), the y y -axis shows accuracy gains of C2C over Receiver-only baselines ( Δ \Delta Accuracy), and each curve represents a Receiver from the Qwen3 series. We find that the accuracy improvements of C2C generally increase faster than T2T. This trend shows that when the Sharer possesses richer knowledge, C2C is able to more effectively transmit useful information to the Receiver. Note that the relative gains for larger Receivers are less pronounced due to their stronger baselines and higher overlap with the Sharer’s knowledge.

Different model combinations . We test different Sharer-Receiver combinations, including different model families and different task-specific models. The result in Table 7 shows that C2C outperforms text-to-text communication on all five combinations by an average of 8.59% on MMLU-Redux. This supports that by employing C2C, the Receiver model can effectively utilize contextual understanding from different models to enhance performance. Notably, when using Qwen2.5-Math as the Sharer, the communication text becomes substantially longer, as analyzed in Appendix A.4.3 . To further test the generalizability of C2C, we swap the Sharer and Receiver models. The results show that C2C robustly brings a 5.05% increase in accuracy while applying T2T results in a 6.30% decrease in performance.

Together, these experiments support the scalability of C2C as an effective and efficient new LLM communication paradigm.

### 4.4 Ablation Study

Sources of improvement . In Table 6 , we ablate the source of C2C performance gain by fixing the Receiver (Qwen3-0.6B) and varying the Sharer. Single denotes standard full fine-tuning of the Receiver without Sharer. Identical denotes C2C where both Sharer and Receiver are Qwen3-0.6B. Our default C2C uses Qwen2.5-0.5B as the Sharer. Under the same training configuration, C2C consistently attains higher accuracy than both Single and Identical . This confirms that C2C improvements do not purely come from added trainable capacity or overfitting to the training set. Instead, it points to complementary contextual understanding contributed by the heterogeneous Sharer. Identical still outperforms Single , indicating that cache-level self-communication can provide useful auxiliary understanding, echoing effects observed in latent reasoning and looped transformers ( Saunshi et al., 2025 ; Fu et al., 2025b ) .

Fuser architecture . In Table 8 we show the effect of different components in the C2C design. Compared with pure projection that discards the Receiver’s cache, fusing both models’ KV-Caches and retaining the Receiver’s via residual connection increases accuracy by 24.18%. Adding a gate for fused layer selection further increases the average accuracy by 3.07%.

### 4.5 Behavior Analysis

Effective rank analysis . We analyze the effective rank of KV-Cache before and after cache-to-cache communication. Effective rank ( Roy and Vetterli, 2007 ) is a common measure of the intrinsic dimensionality of model weights or activation values; a higher intrinsic dimension means richer semantic information, as formalized in Appendix A.4.1 . As Table 2 shows, after cache-to-cache fusing, the effective ranks of K and V increased from 388 to 395 and from 532 to 560, respectively. This indicates that C2C enriches the semantic space by successfully transforming the Sharer’s representations and injecting knowledge into the Receiver model.

Accuracy breakdown . Our analysis reveals that the source of C2C’s accuracy gains depends on the relative capacity of the communicating models and varies across task subcategories. Details can be found in Appendix A.2.3 . We also find that in some specific cases, C2C may fail as the contextual understanding from the Sharer model is not always accurate and can mislead the Receiver into generating the wrong answer. Representative example is provided in Appendix A.4.6 .

Progressive behavior . We analyze the progressive behavior of C2C by gradually increasing the percentage of context KV-Cache being updated by C2C (details in Appendix A.2.4 ). When the percentage is above 50%, increasing the percentage continuously yields better performance.

Gate behavior . We analyze the behavior of C2C ’s learnable gates under different training regimes in Appendix A.4.2 . We can draw the conclusion that general-purpose training favors broad gate activation with fine-grained modulation via weights, whereas task-specific training favors sparse gate activation with stronger reliance on the selected layers.

## 5 Discussion

Future work . C2C opens several directions for future research. (1) Cache communication in multi-agent systems: C2C may serve as a better communication primitive for real-world agentic tasks with complex multi-round reasoning, coding, and tool use. A preliminary case study on mathematical problem solving is provided in Appendix A.5.3 . (2) Cross-modal collaboration: beyond text-only models, fusing caches among vision–language models (VLMs) and vision–language–action (VLA) models may enable richer multi-modal collaboration. (3) Inference acceleration: C2C can enhance speculative decoding and enable token-level routing across heterogeneous models for lower latency and cost. (4) Privacy-aware collaboration: LLMs can transmit KV-Cache segments without explicit text, limiting content exposure and improving privacy.

Limitations . (1) Multi-LLM systems experience performance degradation when a much weaker Sharer provides noisy information to a stronger Receiver. This limitation is shared by both T2T and C2C, as Sharer semantic quality directly impacts Receiver performance. (2) While pairwise KV-Cache communication is feasible and advantageous, scaling up the number of communicating LLMs with O ⁡ ( N ) O(N) training cost remains an open problem. Preliminary efforts towards this goal are in Appendix A.5.1 and A.5.2 .

## 6 Conclusion

We demonstrate that LLMs can communicate beyond text. We introduce Cache-to-Cache (C2C), a general paradigm that transforms and fuses key–value (KV) caches across models to enable direct semantic communication. Across diverse tasks and model configurations, C2C consistently achieves higher task performance and better efficiency than text-to-text communication. These results establish cache-to-cache as a practical alternative to token-based communication and highlight its promise for scalable, low-latency multi-LLM systems.

## Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 62506197, 62325405, 62104128, U19B2019, U21B2031, 61832007, 62204164, 92364201), Tsinghua EE Xilinx AI Research Fund, and Beijing National Research Center for Information Science and Technology (BNRist). We thank Xuefei Ning for her valuable discussions and suggestions.

## Ethics Statement

This study raises no ethical issues. No human subjects or sensitive personal data were involved in the experiments.

## Reproducibility Statement

We provide sufficient information to allow the results reported in this paper to be reproduced. All experiments were conducted using publicly available datasets along with open-source models and code. Implementation details, including data selection, model architectures, hyperparameters, and training procedures, are provided in Appendix A. The code, configuration files, and model checkpoints are released at https://github.com/thu-nics/C2C .

## Appendix A Appendix

### A.1 Design Choice Exploration

We detail the design of C2C and discuss alternative design choices.

#### A.1.1 Layer Alignment

Terminal alignment . In this strategy, the layers of the two models are aligned starting from the output side. Specifically, the final layer of the smaller model is paired with the final layer of the larger model, the penultimate layer with the penultimate layer, and so on. This scheme prioritizes alignment between the deeper layers across models, which typically capture higher-level semantic representations.

Depth-normalized alignment . In this strategy, both models’ layer indices are normalized to [ 0 , 1 ] [0,1] by dividing by ( L − 1 ) (L-1) , where L L is the total number of layers in the model. Let the model with fewer layers ( L min L_{\min} ) serve as the anchor. For each anchor layer i i (with normalized index i / ( L min − 1 ) i/(L_{\min}-1) ), we select the layer j j in the other model ( L max L_{\max} ) whose normalized index j / ( L max − 1 ) j/(L_{\max}-1) is closest: j ⋆ = arg ⁡ min j ​ | i L min − 1 − j L max − 1 | . j^{\star}\;=\;\arg\min_{j}\Bigl|\tfrac{i}{L_{\min}-1}-\tfrac{j}{L_{\max}-1}\Bigr|. (5) This method produces an alignment that distributes correspondences approximately uniformly across the model depth.