Latent Cache Flow: Model-to-Model Communication Without Text — part 3/3

Cleaned-text code points [31745, 37344). Bibliography omitted here; source.md includes it.

The trained gates already disable several layers, and ablations show some active layers are net harmful. Edits at one receiver layer propagate through later layers via the residual stream, so adjacent edits can have correlated effects. The pruning sweep suggests most useful LCF-128 signal is concentrated in a mid-to-deep subset of receiver layers.

##### Limitations.

The retained layer indices are specific to the trained Qwen2.5-0.5B-Instruct → \to Qwen3-0.6B checkpoint. The pruning method generalizes, but the selected layers may differ for other model pairs, training runs, or tasks.

## Appendix E LCF-X Training Details

LCF-X is trained on HotpotQA-bridge with the same joint-KV bottleneck projector as LCF-128, but includes two changes: a hierarchical attention pool (see Section 3.2 ) and a gate-logit initialization of + 1.0 +1.0 . Both Qwen3-0.6B models remain frozen. Only the LCF projector and the two pool queries per layer, q base ( ℓ ) q_{\text{base}}^{(\ell)} and q layer ( ℓ ) q_{\text{layer}}^{(\ell)} , are updated.

The agent setting differs from the base LCF in two ways. First, we use a symmetric Qwen3-0.6B/Qwen3-0.6B pair because both models must process partial context independently. Second, training is task-specific: LCF-X is trained directly on HotpotQA-bridge, so the latent channel learns to route partitioned multi-hop evidence.

### E.1 Hyperparameters

Training partitions the sharer’s input using HotpotQA’s five-paragraph structure, so P = 5 P=5 during training. The architecture is partition-agnostic: P P is not a parameter shape and need not remain fixed at inference; see Section E.4 .

We initialize the gate logit to + 1.0 +1.0 instead of 0.0 0.0 . In early LCF-X runs, zero-initialized gates stayed near sigmoid ( 0 ) = 0.5 (0)=0.5 because the residual branch was small, producing weak gate gradients. Setting it to + 1.0 +1.0 gives the residual branch a meaningful gain at step 0, sigmoid ( 1.0 ) = 0.73 (1.0)=0.73 , while preserving trainability.

### E.2 Training Trajectory

We log dev F1/EM every 50 steps during training. Validation metrics are reported on the held-out HotpotQA-bridge split ( n = 5,899 n{=}5{,}899 ) at the final checkpoint.

After warmup, performance improves steadily, with the best checkpoint at the end of the single-epoch run. The span-invariance check in Section E.4 rules out dependence on the specific five-paragraph partition used during training.

We use the final checkpoint for evaluation. On HotpotQA-bridge val ( n = 5,899 n{=}5{,}899 ), the final checkpoint achieves 35.13 F1 and 25.28 EM.

### E.3 Architectural Cost

The LCF-X pool adds two query vectors per receiver layer: a base query q base ( ℓ ) ∈ ℝ H × d q_{\text{base}}^{(\ell)}\in\mathbb{R}^{H\times d} for the first pooling stage and a layer query q layer ( ℓ ) ∈ ℝ H × d q_{\text{layer}}^{(\ell)}\in\mathbb{R}^{H\times d} for the second pooling stage: | q base ( ℓ ) | + | q layer ( ℓ ) | = 2 ​ H ​ d . |q_{\text{base}}^{(\ell)}|+|q_{\text{layer}}^{(\ell)}|=2Hd. For Qwen3-0.6B, with H = 8 H{=}8 KV heads and d = 128 d{=}128 , 2 ⋅ 8 ⋅ 128 = 2,048 2\cdot 8\cdot 128=2{,}048 parameters per layer, totaling 57,344 57{,}344 across the 28-layer receiver. This is 0.22 % 0.22\% of the LCF-X projector’s 25.87 25.87 M trainable parameters and about 0.005 % 0.005\% of the frozen two-model system.

Critically, the pool size is independent of P P : the same two queries are reused for every source span, regardless of how many spans the deployment chooses. The pool, therefore, supports variable span counts without retraining or changing parameter shapes.

### E.4 Span-Scheme Invariance

LCF-X is trained with the natural HotpotQA five-paragraph partition. Because the pool has no P P -dependent parameters, the same trained checkpoint admits different partition schemes at inference. We test this directly by holding the trained weights fixed and evaluating on HotpotQA-bridge validation under paragraph-, token-, and sentence-level span schemes.

F1 stays within a 0.32 0.32 -point band of the natural-paragraph reference, even across a 4.7 × 4.7\times span-count range and three granularities. The best result comes from 200-token windows with no overlap, improving F1 by + 0.32 +0.32 and EM by + 0.22 +0.22 . All schemes use the same 55.1 GB VRAM footprint, indicating that changing the span partition changes the source summary structure without changing the deployed checkpoint.

At inference time, P P can be chosen to match the input structure, using retrieved passages, sentence chunks, sliding windows, or a single span. No retraining or deployment-specific fine-tuning is needed.

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .