Latent Cache Flow: Model-to-Model Communication Without Text — part 1/3

Cleaned-text code points [0, 15844). Bibliography omitted here; source.md includes it.

# Latent Cache Flow: Model-to-Model Communication Without Text

###### Abstract

LLM agents today communicate via text, which incurs considerable latency and information loss due to the need to autoregressively decode the sharer model’s state and encode at the receiver model. Recent work such as Cache-to-Cache ( Fu et al., 2026 , C2C;) seeks to exchange KV caches by learning adapters that translate sharer KV matrices to the receiver model. However, the adapters are large and expensive to train, and translate individual tokens, which requires the target context to be identical. This is unsuitable for agent communication, where the LLMs have differing context.

We introduce Latent Cache Flow (LCF). To address efficiency, we observe that keys and values can be jointly translated and compressed, reducing the adapter to about 4% of C2C’s size. To address differing context, we design the adapter to transmit a summary of new information that the target model does not have. Our early experiments show that a pruned 13 MB LCF adapter can be more accurate than C2C at 956 MB in shared-context settings; for different contexts, LCF improves F1 by 7.5% and Exact Match by 23% while 8.5 × 8.5\times faster than text-based communication.

###### Keywords:

## 1 Introduction

Language-model workflows involve multiple models with different roles, tools, and contexts ( Yao et al., 2023 ; Du et al., 2024 ; Wu et al., 2024 ; Chen et al., 2024 ) . These models must communicate to combine their work, and they usually do so through generated text. For example, one model may summarize a document chunk, and another may use that summary to answer a question over a different chunk. This text-mediated channel is slow because the receiver must wait for the sender to decode, and lossy because much of the sender’s internal state is compressed into discrete tokens.

Cache-to-Cache (C2C) ( Fu et al., 2026 ; Dery et al., 2026 ) addresses the text bottleneck by exchanging KV caches directly. In C2C, the sharer and receiver process the same input, and a learned adapter fuses their caches at matching token positions. This bypasses intermediate text generation, but it makes C2C position-wise by design. The assumption is restrictive for deployed multi-model systems, where models are used because they handle different sub-tasks, tools, or context. When their contexts differ, their token positions no longer align, and position-wise cache exchange does not apply. C2C also introduces substantial adapter overhead: for a 1.1B-parameter sharer–receiver pair, its adapter is 956 MB .

We introduce Latent Cache Flow (LCF) to address both limitations. LCF treats cache transfer as compressed communication rather than token-by-token translation. It replaces separate key and value fusers with a shared low-dimensional cache channel, and uses layer pruning to remove receiver layers that do not benefit from cache updates. We extend LCF to cross-context communication with LCF-X. Instead of translating each sharer token into a receiver-aligned cache update, LCF-X summarizes the sharer’s full KV cache into a fixed-size tensor that the receiver can condition on.

Text-to-text (T2T) and C2C represent opposite extremes. T2T is flexible because the sharer can send free-form text, but it pays decoding latency and token-projection loss. C2C avoids decoding by exchanging caches at matching positions, but requires aligned inputs. Figure 1 illustrates the middle ground targeted by LCF and LCF-X: compressed cache-level communication that improves efficiency while relaxing the shared-context constraint.

## 2 Background: Cache-to-Cache

The natural target for cross-model transfer is the KV cache. This cache is the residue of the sharer’s forward pass and holds the per-layer key and value tensors that summarize its reading of the prompt. However, direct cache reuse is not feasible across different architectures. The sharer and receiver may differ in the number of layers, attention heads, head dimensions, or the geometry of the learned latent space. As a result, the receiver cannot consume the sharer’s KV tensors verbatim. Cache-to-Cache (C2C) addresses this challenge by treating the sharer’s cache as a learned semantic conditioning signal. C2C trains a fuser module that takes both models’ caches as input and generates a residual update, which is added to the receiver cache before decoding.

Figure 2 illustrates the C2C fuser, which contains two independent duplicate modules for the key and value tensors. We describe the key side; values are handled similarly.

After the sharer and receiver process the prompt, they emit layer-aligned multi-head keys S K S_{K} and R K R_{K} . The fuser concatenates these keys and flattens them along the head dimension to form a joint per-token feature vector. A linear layer then projects this vector down to the receiver’s hidden dimension. An MLP performs feature fusion, mixing the sharer and receiver projections into a joint encoding.

The fused features split into two parallel paths. The projection path produces a per-head cache update Δ K \Delta_{K} in the receiver’s ( H t , D h ) (H_{t},D_{h}) shape. The dynamic weighting path produces a per-head scalar α K \alpha_{K} that re-weights the update. Their element-wise product is masked by a learned Gumbel-sigmoid gate, a differentiable relaxation of an on/off decision that is annealed toward a binary layer-selection at inference. Finally, the result is added to R K R_{K} as a residual, yielding the enriched cache R K ′ R^{\prime}_{K} .

## 3 Latent Cache Flow

In this section, we first describe how the design of Latent Cache Flow (LCF) improves the efficiency of KV transfer when the contexts match. We then describe an extension called LCF-X that pre-processes the sharer model’s KV cache to support communication between differing contexts.

### 3.1 Latent Cache Flow for Exact Context Sharing

We address two main sources of inefficiencies in C2C. First, C2C processes keys and values independently by duplicating the fuser pipeline for each. However, keys and values are projections of the same hidden state. Recent architectures, such as Gemma 4 ( Google, 2026 ) , showcase this redundancy through unified key-value tensors for global attention layers. LCF replaces C2C’s dual pipelines with a single shared key-value pipeline.

Second, C2C keeps communication in the receiver’s high-dimensional cache space. It concatenates the sharer and receiver caches, fuses them at full width, and outputs receiver-shaped residuals. This makes transfer expensive: each cache type needs its own full-width fuser.

LCF instead transfers information through a low-dimensional latent bottleneck. This bottleneck is motivated by two observations: KV states can be compressed within a model ( DeepSeek-AI, 2024 ) , and cross-model representations can be approximately linearly aligned ( Chen et al., 2025 ; Dery et al., 2026 ) . LCF combines these observations by projecting the joint key–value representation into a compact latent space, mixing it, and up-projecting the result back into receiver-cache residuals.

Figure 3 shows the full architecture in two parts: the LCF-X pooling module at the top and the core LCF projector at the bottom. This section focuses on the core LCF projector, which assumes token-aligned sharer and receiver caches.

1. Input. Each receiver transformer layer receives a pair of sharer cache ( 𝐒 K , 𝐒 V ) (\mathbf{S}_{K},\mathbf{S}_{V}) and receiver cache ( 𝐑 K , 𝐑 V ) (\mathbf{R}_{K},\mathbf{R}_{V}) .

2. Concat & Flatten The KV cache tensors from the Sharer and Receiver are concatenated and flattened along the head dimension into a single tensor.

3. Shared Latent Pipeline. The concatenated tensor 𝐱 \mathbf{x} is linearly down-projected to d d dimensions: 𝐳 = W down ​ 𝐱 \mathbf{z}=W_{\mathrm{down}}\mathbf{x} , followed by a two-layer MLP. We select d ∈ { 64,128,256,512 } d\in\{64,128,256,512\} as a hyperparameter based on downstream accuracy (Figure 4 ).

4. Latent split. The latent tensor is then split into separate key and value tensors: [ 𝐳 K , 𝐳 V ] = 𝐳 [\mathbf{z}_{K},\mathbf{z}_{V}]=\mathbf{z} where 𝐳 i ∈ ℝ B × N × d / 2 \mathbf{z}_{i}\in\mathbb{R}^{B\times N\times d/2} . At this point, the keys and values will be reconstructed independently.

5. Gates. Per-head gates α i ∈ ℝ h r \alpha_{i}\in\mathbb{R}^{h_{r}} are produced from the full latent 𝐳 \mathbf{z} , allowing K and V updates to be independently scaled using shared latent features.

6. Upward project. The Key and Value tensors are projected back to receiver-cache size: Δ i = W up i ​ 𝐳 i , i ∈ { K , V } . \Delta_{i}=W_{\mathrm{up}}^{i}\mathbf{z}_{i},\quad i\in\{K,V\}. We adopt C2C’s per-layer Gumbel-sigmoid gate, producing g i ∈ { 0 , 1 } g_{i}\in\{0,1\} at inference. The receiver cache is updated as: 𝐑 i ′ = 𝐑 i + g i ⋅ ( α i ⊙ Δ i ) , i ∈ { K , V } . \mathbf{R}^{\prime}_{i}=\mathbf{R}_{i}+g_{i}\cdot(\alpha_{i}\odot\Delta_{i}),\quad i\in\{K,V\}.

7. Enriched receiver KV. The gated residuals update the receiver’s K and V caches.

Adapter Parameters. For a pair of 1.1B parameter models, C2C requires a 477.8M-parameter adapter model. At d = 128 d=128 , LCF only requires 19.4M parameters across 28 layers (693K per layer)— 24.6 × 24.6\times fewer than C2C.

### 3.2 LCF-X: Cross-Context Communication

LCF reduces C2C’s adapter overhead, but it still assumes shared context: the sharer and receiver must process the same prompt so that each receiver token has a matching sharer token. This position-wise requirement prevents cross-context communication. LCF-X removes it by replacing matched sharer tokens with a position-free summary of what the sharer learned from its own context. As shown in Figure 3 , LCF-X adds a sharer-side pooling module before the standard LCF projector; the receiver therefore conditions on the sharer’s learned context, not on a token-aligned variation of its own prompt.

We divide the sharer’s context into P P spans S 1 , … , S P S_{1},\ldots,S_{P} . A span is a contiguous block of source tokens, such as a paragraph, passage, or fixed-size chunk. The first part of Figure 3 shows this pooling step. LCF-X uses attention as a pooling operation: within each span, a learned query attends only to the tokens in that span and produces one key–value summary. Across all spans, this gives 𝐊 ~ , 𝐕 ~ ∈ ℝ B × H × P × D . \tilde{\mathbf{K}},\tilde{\mathbf{V}}\in\mathbb{R}^{B\times H\times P\times D}. The P P summaries keep span identity before global pooling. Larger P P gives cross-span attention more local summaries to weight; smaller P P gives it fewer, coarser summaries. Because the same pooling query is reused for every span, P P can vary at inference; one checkpoint can summarize one passage, five passages, or a sliding set of chunks. Appendix E.4 gives the span-invariance analysis.

LCF-X then pools across spans. A second attention step aggregates the P P span summaries into one per-head sharer summary, 𝐊 ∗ , 𝐕 ∗ ∈ ℝ B × H × 1 × D . \mathbf{K}^{*},\mathbf{V}^{*}\in\mathbb{R}^{B\times H\times 1\times D}. This summary is broadcast across the receiver sequence and replaces the sharer’s matched-position cache in LCF. The lower part of Figure 3 shows that, after this replacement, the standard LCF projector and receiver cache are unchanged.

LCF and LCF-X address different limits of C2C. LCF replaces C2C’s high-dimensional, cache-specific fusers with a compressed shared key–value pipeline. LCF-X removes the shared-context assumption by replacing matched-position sharer caches with pooled, position-free summaries. The next section tests whether these changes preserve cache-transfer accuracy while reducing adapter overhead.

## 4 Experiments

We first evaluate LCF as compared to C2C in a shared-context setting, and then evaluate LCF-X in a cross-context setting as compared to T2T communication, where models observe different contexts and must share novel information.

### 4.1 Shared-Context

#### 4.1.1 Experimental Setup

The shared-context setting assumes aligned token positions between models. It tests LCF’s compression claim: whether a smaller latent fuser can match or improve C2C’s full-width cache fusion.

The sharer is Qwen2.5-0.5B-Instruct ( Qwen Team, 2024 ) , and the receiver is Qwen3-0.6B ( Qwen Team, 2025 ) . We use OpenHermes 2.5 ( Teknium, 2023 ) , matching C2C’s training data, and keep both base models frozen. For C2C, we use the released full-schedule nics-efc/C2C_Fuser checkpoint. We train LCF under the same public C2C recipe, changing only the fuser architecture. Full hyperparameter, filtering, and compute details are provided in Appendix B .

We evaluate LCF’s sensitivity to bottleneck dimension with d = { 64,128,256,512 } d=\{64,128,256,512\} . We also evaluate layer pruning, which removes receiver layers whose learned gates are near zero. This tests whether LCF can further reduce adapter size without losing downstream accuracy.

We evaluate on four logit-based zero-shot multiple-choice benchmarks. We use MMLU-Redux (MMLU-R) ( Gema et al., 2024 ) , ARC-Challenge (ARC-C) ( Clark et al., 2018 ) , and OpenBookQA (OBQA) ( Mihaylov et al., 2018 ) from C2C’s benchmark suite, and replace C-Eval with MMLU-Pro ( Wang et al., 2024 ) for stronger English-language coverage. Logit-based scoring avoids free-form parsing noise by reading the predicted answer directly from the model’s logits over the answer choices. We also compare with an optimal-routing frontier (ORF), a per-question oracle that selects the correct answer when either base model is correct.

#### 4.1.2 Training Cost

Under the matched pipeline, training time is dominated by frozen model prefills, not adapter computation. Each optimizer step runs a sharer prefill, a receiver prefill, cache fusion, and a backward pass through the adapter. This approach holds the communication workload fixed for both adapters, rather than optimizing around LCF.

With this setup, LCF-128 trains for 300 optimizer steps in about 4.5 hours on a single Colab A100. Each optimizer step takes 52 seconds, with an effective batch size of 260. For comparison, our C2C run on a single A100 takes 66 seconds per optimizer step after warmup, measured with an effective batch size of 256. We use 300 LCF steps because both the published C2C curve and our C2C reproduction achieve their best MCQ accuracy at step 250. Later checkpoints do not improve downstream accuracy.

The modest per-step wall-clock difference reflects where training time is spent. Most computation comes from the frozen sharer and receiver forward passes, which both methods share. The smaller LCF adapter reduces trainable parameters, adapter memory, and adapter-side backward cost. However, these savings are partly hidden by the frozen-model prefill cost. When KV caches are precomputed or reused, adapter-side costs become more visible. In those settings, smaller adapters may provide larger practical savings.

#### 4.1.3 Shared-Context Results

Figure 4 summarizes the shared-context results, including both bottleneck scaling and layer-pruned variants. LCF establishes an accuracy–efficiency frontier for inter-model communication, outperforming C2C at substantially lower adapter overhead. The frontier is traced by the progression from LCF-128-9L to LCF-128, and finally to LCF-256. Increasing latent capacity improves accuracy while keeping adapter size much smaller than C2C.

LCF-256 achieves the strongest overall performance, reaching a weighted average of 32.88 across the evaluated benchmarks while using only 107 107 MB. LCF-128 provides a particularly efficient operating point near the Pareto knee, achieving 31.94 weighted accuracy with only 39 MB of overhead. Aggressively pruned variants remain competitive, the 9-layer configuration reaches 31.26 weighted accuracy using only 13 MB .