Latent Cache Flow: Model-to-Model Communication Without Text — part 2/3

Cleaned-text code points [15844, 31745). Bibliography omitted here; source.md includes it.

The frontier also reveals both under- and over-capacity regimes. LCF-64 underperforms, suggesting insufficient communication bandwidth. In contrast, LCF-512 increases adapter size substantially but provides weaker accuracy than LCF-256. This non-monotonic behavior suggests that the bottleneck acts as a compression mechanism and as a regularizer for communication.

Notably, LCF-256 exceeds the optimal routing frontier (ORF) on ARC-C and MMLU-Pro, suggesting gains that retrospective routing cannot capture, whether through transfer or shifted decision boundaries.

#### 4.1.4 Layer Pruning

Layer pruning reveals that LCF communication concentrates in a small subset of receiver layers. We prune LCF-128 by removing layers whose learned Gumbel-sigmoid gates converge toward zero during training.

As shown in Table 2 , performance degrades gradually as layers are removed, indicating that LCF communication is highly concentrated rather than uniformly distributed across the network. The 19-layer pruned model reduces projector size from 39 MB to 26 MB with almost no loss in weighted accuracy. The 9-layer critical subset continues to outperform C2C while requiring only 13 MB of overhead.

The aggressive 6-layer variant approximately matches C2C performance (29.41 vs. 29.13) using only 8 MB of projector parameters, corresponding to a 115 × 115\times reduction in overhead. This suggests that a small subset of receiver layers accounts for most inter-model communication gains. Layer indices and pruning details are provided in Appendix D .

### 4.2 Cross-Context Information Transfer

#### 4.2.1 Experimental Setup

The cross-context setting tests whether LCF-X can perform cache-level transfer without being restricted to token-aligned sharer and receiver inputs.

We create the cross-context setting by partitioning HotpotQA’s distractor context ( Yang et al., 2018 ) . In the standard distractor setting, each question is paired with ten paragraphs: two supporting paragraphs and eight distractors. We split these ten paragraphs into two disjoint sets of five, with each set containing one supporting paragraph and four distractors. One set is assigned to the sharer and the other to the receiver. This partition gives the models different contexts while preserving the original two-hop evidence structure. Since the receiver sees only one supporting paragraph, exact-match answering generally requires new context from the sharer’s partition.

Both models are Qwen3-0.6B instances, isolating communication effects from model capacity. We first establish three receiver-only baselines to measure how parametric knowledge and evidence availability affect performance: (i) question-only, where the receiver answers using internal knowledge without supporting paragraphs, (ii) half-context, where the receiver sees only its partition, and (iii) full-context, where the receiver sees all ten paragraphs.

We then compare T2T with LCF-X. In the T2T baseline, the sharer reads its partition and generates a text message, which is appended to the receiver’s input. We evaluate T2T with communication budgets of 50, 100, 150, and 200 generated tokens. In LCF-X, the sharer’s KV cache is pooled into a fixed-size cache summary and passed through the LCF projector, enabling cross-context cache transfer without token alignment.

We train LCF-X directly on the partitioned HotpotQA training set. Both the sharer and receiver are frozen; only the LCF projector and hierarchical pooling queries are trainable. Training runs for one epoch, 276 optimizer steps, with an effective batch size of 256. The process takes roughly 75 minutes on a single NVIDIA RTX PRO 6000 Blackwell GPU. Full training details are provided in Appendix E .

We evaluate answer quality on the HotpotQA-bridge ( n = 5,899 n=5{,}899 ) evaluation split. We report Exact Match (EM), which measures whether the predicted answer matches the ground truth string exactly. We also report token-level F1, which measures partial overlap between predicted and reference answers. We additionally report latency in terms of time-to-first-token (TTFT) and time-to-end-of-answer (TTEoA).

#### 4.2.2 Cross-Context Results

LCF-X improves both quality and latency in the tested cross-context setting. On HotpotQA-bridge, LCF-X reaches 35.13 F1 and 25.28 EM, outperforming the strongest T2T baseline, T2T Max 200, by +2.47 F1 and +4.75 EM. These gains are statistically robust under both paired F1 and exact-match tests (Wilcoxon p = 5.42 × 10 − 5 p=5.42\times 10^{-5} ; McNemar p = 4.23 × 10 − 12 p=4.23\times 10^{-12} ).

The receiver-only baselines show that evidence availability matters. Moving from question-only to half-context raises F1 from 7.48 to 24.62. Providing full context raises it further to 29.98. Both T2T and LCF-X exceed the full-context receiver baseline. This suggests that communication recovers missing paragraphs and changes how evidence reaches the receiver. Information now arrives through the sharer’s forward pass, either as generated text or as a pooled cache summary.

The key comparison is therefore between communication methods. Increasing the T2T budget from 50 to 200 tokens improves F1 from 30.81 to 32.66, but increases TTFT from 250 ms to 410 ms. T2T buys modest quality gains with additional decoding. LCF-X avoids this trade-off: it reaches higher quality than every T2T budget while reducing TTFT to 48 ms and TTEoA to 94 ms. Because TTEoA also depends on the generated answer length, we treat TTFT as a cleaner measure of communication overhead and report TTEoA as the end-to-end response time.

Figure 5 summarizes the quality–latency frontier. LCF-X lies above and to the left of the T2T sweep, indicating higher answer quality at lower communication latency.

## 5 Limitations and Future Work

While these results are promising, they are an initial validation of LCF. Our shared-context experiments use one model pair, Qwen2.5-0.5B-Instruct to Qwen3-0.6B, and our cross-context experiments use two Qwen3-0.6B instances. Although C2C works across model families and larger scales ( Fu et al., 2026 ) , LCF testing is future work.

Our different context setting is also deliberately controlled. Partitioned HotpotQA assesses if information can move through the channel, though we cannot fully separate transfer from tuned adaptation of the receiver. Richer agentic workflows with longer contexts, tool outputs, heterogeneous roles, and multi-turn communication remain future work.

## 6 Conclusion

Multi-model language workflows need communication channels that are fast, lightweight, and compatible with distinct model contexts. Text-to-text communication is flexible but sequential and lossy. Prior cache-based methods avoid text generation, but they require shared, token-aligned inputs and large adapters for each model pair.

Latent Cache Flow (LCF) shows that cache communication does not need full-width cache transfer. In the shared-context setting, LCF matches or outperforms C2C across all four benchmarks with a 13 ​ MB 13~\mathrm{MB} adapter, compared with C2C’s 956 ​ MB 956~\mathrm{MB} adapter. This result shows that cache information can flow from one model to another through a latent channel.

LCF-X then moves cache communication toward greater context flexibility. On partitioned HotpotQA, LCF-X transfers information across distinct contexts, improves exact-match accuracy by 23 % 23\% over text communication, and reduces time-to-first-token by 8.5 × 8.5\times . Together, these results move inter-model communication toward a different interface: cache-level rather than textual, compressed rather than full-width, and flexible beyond token-aligned contexts.

## Appendix A Full Results Overview

Table 4 lists all methods evaluated in this paper. T2T is a cited baseline from C2C. We do not re-evaluate it. ORF is the optimal-routing frontier. This is a per-question oracle that picks whichever base model gives the correct answer. ORF can therefore be seen as a retrospective optimal routing system.

† T2T numbers are from Fu et al. (2026) . We do not re-evaluate them. C2C does not report T2T on MMLU-Pro. No weighted average is computed.

‡ LCF-128-9L is the 9-layer critical pruned variant. It keeps 9 of 28 adapter layers. Total trainable parameters: 6.24M. This is a 76.7 × \times reduction versus C2C’s 477.8M fuser.

All rows except T2T use our logits-based evaluation pipeline. The same prompt template and alignment strategy are used. Significance testing is by per-question join. Receiver and sharer are bare frozen models. C2C uses the published nics-efc/C2C_Fuser checkpoint. LCF rows use our best checkpoints from bottleneck and pruning sweeps.

## Appendix B Training Details and C2C Comparison Protocol

To isolate the effect of the fuser architecture, we match C2C on the frozen model pair, training objective, OpenHermes data, and MCQ evaluation. C2C uses the published full-schedule nics-efc/C2C_Fuser checkpoint. LCF is trained under the matched protocol below. The main text reports the corresponding wall-clock and per-step training costs.

Across LCF variants, the only architectural variable is latent width; this width also sets the 4 × \times intermediate MLP dimension shown in Table 5 . All variants use the same data, optimizer, schedule, hardware, and seed. We adjust per-device batch size and gradient accumulation to keep VRAM under 80GB on a single A100, while keeping effective batch size at 260 ± 4 260\pm 4 .

### B.1 Matched Training Protocol

##### Training cost.

Under this matched pipeline, LCF-128 takes roughly 52 seconds per optimizer step on a single Colab A100, with an effective batch size of 260. C2C takes roughly 66 seconds per step at batch size 256. The modest wall-clock difference reflects that both methods spend most training time in frozen sharer and receiver forward passes. LCF’s size reduction appears primarily as lower trainable-parameter and adapter-memory cost, not as a proportional wall-clock speedup.

##### Why 300 steps.

C2C trains for one full OpenHermes epoch (1929 optimizer steps). In both the published C2C curve and our re-run, best MCQ accuracy is reached by step 300. Later checkpoints do not help and can degrade results. LCF shows the same pattern: all bottleneck widths peak between steps 200 and 300. We therefore train LCF ablations for 300 steps. The C2C baseline in our tables is not a 300-step re-run, but the published full-schedule checkpoint. This makes the comparison conservative: we compare 300-step LCF against full-schedule C2C.

### B.2 Eval-Loss Convergence

Held-out OpenHermes loss converges quickly for all bottleneck widths. Table 7 reports next-token prediction loss on the held-out 1% split every 50 steps.

LCF-64 through LCF-256 converge to within 0.006 nats of each other. This holds despite a 7 × \times difference in trainable parameters. LCF-512 reaches the lowest NTP loss, but this does not give the best downstream accuracy. It underperforms LCF-256 on three of four benchmarks. Validation NTP loss is not a reliable proxy for fuser quality here. Bottleneck selection requires direct downstream evaluation.

### B.3 Training-Curve Summary

For all bottleneck widths, train and eval loss track closely. There is no persistent train/eval gap. Larger bottlenecks converge faster at first, but downstream accuracy does not monotonically follow NTP loss. This supports the main ablation result: for this model pair and protocol, useful latent communication peaks at d = 256 d=256 . More projector capacity does not always help.

## Appendix C Statistical Significance

This appendix reports every significance test in the paper. We use a one-sided exact paired McNemar test on per-question correctness, with alternative H 1 : LCF > C2C H_{1}:\mathrm{LCF}>\mathrm{C2C} . Outputs are paired by joining method CSVs on (subject, question_id, true_answer) .

For each LCF variant and benchmark, let b b count questions where LCF is correct and C2C is wrong, and c c count the reverse. Concordant pairs are uninformative. Conditional on n = b + c n=b+c under H 0 H_{0} , b ∼ Binomial ⁡ ( n , 0.5 ) b\sim\mathrm{Binomial}(n,0.5) , giving the one-sided exact p-value p = Pr [ Binomial ( n , 0.5 ) ≥ b ] . p=\Pr[\mathrm{Binomial}(n,0.5)\geq b]. We use the exact binomial test rather than the χ 2 \chi^{2} approximation because some discordant counts are small. Significance markers are uncorrected: ∗ p < 0.05 {}^{*}p<0.05 , p ∗ ⁣ ∗ < 0.01 {}^{**}p<0.01 , ∗ ∗ ∗ p < 0.001 {}^{***}p<0.001 .

### C.1 Main LCF Variants

The strongest pattern appears on the two largest benchmarks. All LCF variants with d ≥ 128 d\geq 128 beat C2C on MMLU-Redux and MMLU-Pro at p < 0.001 p<0.001 . On ARC-C and OBQA, test sets are smaller. Gains of 1 or 2 points remain positive but are often not significant. LCF-256’s ARC-C gain clears p < 0.01 p<0.01 . LCF-512’s OBQA gain clears p < 0.05 p<0.05 .

### C.2 Pruned LCF-128 Variants

The 19-layer and 9-layer pruned variants remain significant on MMLU-Redux and MMLU-Pro, while ARC-C and OBQA do not clear p < 0.05 p<0.05 . The 6-layer variant does not significantly outperform C2C.

All p-values are computed from paired per-question CSV outputs joined on (subject, question_id, true_answer) using scipy.stats.binomtest with alternative="greater" .

## Appendix D Layer Pruning

LCF uses one independent projector per receiver layer. For Qwen3-0.6B, this means 28 projectors. Each contributes a residual KV edit at a specific receiver layer. This design makes LCF amenable to post-hoc layer pruning. A trained subset of projectors can be retained at evaluation time without retraining. Dropped projectors simply produce no edit at their layer.

Unlike conventional weight pruning, LCF pruning is structural. Entire layer-projectors are removed, not individual weights. Because projectors are independent across layers, dropping one is equivalent to setting its gate-times-residual contribution to zero. The remaining projectors operate exactly as they did during training.

##### Parameter accounting.

LCF-128 trains 28 projectors of about 693K parameters each, for a total of 19.4M trainable parameters. C2C trains 28 fuser layers totaling 477.8M parameters for the same model pair. Keeping K K LCF-128 projectors gives roughly K ⋅ 693 ​ K K\cdot 693\mathrm{K} trainable parameters. At K = 9 K=9 , this is 6.24M, or 76.7 × 76.7\times fewer than C2C.

##### Pruning method.

Layers are selected in three passes. First, a gate audit removes layers with non-positive trained Gumbel-sigmoid gate logits for both K and V, as these contribute zero signal under the hard evaluation gate. Second, layers that are net harmful by single-layer ablation are removed. Third, the remaining layers are ranked by ablation importance, and the top- K K layers are kept.

On the trained LCF-128 checkpoint, six layers are dead by the gate audit: { 0 , 1 , 2 , 3 , 5 , 12 } . \{0,1,2,3,5,12\}. Three more are harmful by ablation: { 20 , 24 , 27 } . \{20,24,27\}. The top-9 retained layers are: { 6 , 8 , 9 , 10 , 15 , 16 , 17 , 19 , 21 } . \{6,8,9,10,15,16,17,19,21\}. No additional fine-tuning is done after pruning.

The top-9 configuration is the main pruned result. With only 6.24M trainable parameters, it remains above C2C on all four benchmarks: + 3.11 +3.11 on MMLU-Redux, + 0.70 +0.70 on ARC-C, + 0.60 +0.60 on OBQA, and + 1.88 +1.88 on MMLU-Pro. The gains on MMLU-Redux and MMLU-Pro are statistically significant under the one-sided exact McNemar test in Appendix C . The top-9 configuration is the main pruned result. With only 6.24M trainable parameters, it remains above C2C on all four benchmarks: + 3.11 +3.11 on MMLU-Redux, + 0.70 +0.70 on ARC-C, + 0.60 +0.60 on OBQA, and + 1.88 +1.88 on MMLU-Pro. The gains on MMLU-Redux and MMLU-Pro are statistically significant under the one-sided exact McNemar test in Appendix C .

##### Why pruning works.