Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse — part 3/4

Cleaned-text code points [30351, 46117). Bibliography omitted here; source.md includes it.

The cost we want to skip is the receiver’s prefill, so we compare end-to-end re-prefill latency against KV cache transfer latency on Qwen3 14B ↔ \leftrightarrow 32B. The mapper replaces the target’s transformer body with a per-layer batched matrix multiply, letting the receiver decode directly from the mapped cache. Table 5 reports per-sequence-length latency. The mapper runs 4–25 × \times faster small-to-large and 3–7 × \times faster large-to-small across sequence lengths from 64 to 32,768 tokens. The large-to-small speedup widens with sequence length because the mapper’s wall time grows much slower than the receiver’s re-prefill. The full sequence-length sweep is in Appendix G .

## 5 Broader impact, future work, and limitations

#### Broader impact.

Cross-model KV cache transfer skips re-prefill on the receiver, reducing serving cost, energy, and tail latency for agentic workloads with frequent model swaps in cost-quality cascading, mid-conversation switching, and routing.

#### Future work.

(1) Trained mapper alternatives. Linear regression fits from a small calibration set without backpropagation, which made it the practical starting point. The MLP results in § 4.4 show that nonlinear variants can recover the matched-KV pairs where ridge falls short, so systematic comparison across MLP, MoE, and attention-aligned objectives that directly optimize cosine is a natural follow-up. (2) Predictive transferability. Attention-output cosine acts as a post-hoc diagnostic since it requires fitting the mapper. A signal that estimates transferability before fitting would accelerate deployment screening, with downstream retention on representative workloads such as HellaSwag as one candidate proxy. (3) Cross-family transfer. Whether the closed-form approach extends across families such as Qwen3 → \to Llama 3.1 is open. Different families may share enough representation structure to admit a similar mapping, or may require different machinery. (4) Hidden factors. Beyond architecture, training-data overlap, fine-tuning recipe, and other non-architectural factors may also influence transferability. We leave systematic study of these to future work. (5) Hybrid architectures. Extending the mapper to attention-restricted hybrids (sliding-window, local) and to attention-recurrent hybrids such as Nemotron 3 ( Blakeman et al., 2025 ) that carry SSM state alongside KV is a natural next step.

#### Limitations.

(1) Single-domain calibration. Calibration uses only FineWeb-Edu. Appendix C measures the cost of substituting Wikipedia or CodeAlpaca on one pair. Neither substitution separates subject matter from register, so the sweep does not bound calibration confined to a single field such as medicine or law. (2) k k selection. Per-pair k k is selected on the same log-likelihood benchmarks we report on. Appendix H measures the effect, at most 2.49 2.49 pp, and adds a held-out evaluation, though neither is a substitute for selecting k k out of sample. (3) Matched-KV is empirical. The closed-form mapper imposes no structural requirement on source or target dimensions, but our six pairs are all matched-KV by construction. We do not test mismatched-KV pairs. Whether ridge transfer can succeed under mismatched-KV conditions is left to future work. (4) Scope. Within-family transfer over dense full-attention models. Hybrid attention and attention-recurrent architectures are out of scope.

## 6 Conclusion

Cross-model KV cache transfer within an LLM family admits a closed-form, training-free fit. The cross-model KV relationship is largely linear, and a closed-form per-head ridge mapping retains 73–98% of standalone accuracy on the four highest-retention matched-KV pairs in our evaluation, runs 2.7–25 × \times faster than re-prefill, and remains stable across multi-turn handoff. Per-pair retention is determined by where the residual error lands relative to the target’s attention-sensitive subspaces, not by its magnitude. Attention-output cosine captures this and predicts cross-pair retention better than R 2 R^{2} ( r = + 0.57 r{=}{+}0.57 vs. r = − 0.20 r{=}{-}0.20 over 12 matched-KV pair evaluations from three families), and a nonlinear MLP adds up to + 37 +37 pp HellaSwag retention on harder pairs by redistributing error toward attention-irrelevant directions. These results shift evaluation criteria for cross-model mappers from raw reconstruction metrics to subspace-aware diagnostics, and motivate attention-aligned objectives for future cross-model mappers.

## Appendix A Cross-model KV cache transfer methods

This appendix tabulates the cross-model KV cache transfer methods discussed in the prose comparison of § 2.2 . Each row records whether the method is gradient-free, whether it works cross-scale (different parameter counts), whether it transfers KV values rather than attention patterns, and whether the mapping is closed-form. Our method is the only one that satisfies all four criteria.

† DroidSpeak transfers KV only between architecturally identical models (same hidden size, layer count, head configuration). The cross-scale and closed-form columns are therefore marked “—” as inapplicable rather than absent.

## Appendix B Linear structure analysis

This appendix collects the empirical evidence for the linear-structure claim in § 2.3 : the greedy forward-selection curves that quantify how many source layers the mapper actually needs. The pairwise R 2 R^{2} heatmaps that motivate the four qualitative patterns are inlined as Figure 2 in § 2.3 , and the production-ridge R 2 R^{2} at k = 1 , 8 , all k{=}1,8,\text{all} is in Appendix C .

#### Source layer selection by greedy forward selection.

The single-source R 2 R^{2} probe in § 2.3 measures how predictable each target layer is from a single source layer. Here we ask how many source layers the mapper actually needs by greedy forward selection. Starting from the best single source layer, we iteratively add the source layer that maximally increases joint R 2 R^{2} at each step, producing curves from k = 1 k{=}1 to k = all k{=}\text{all} .

The curves climb steeply from k = 1 k{=}1 to k = 4 k{=}4 and are close to their k = all k{=}\text{all} value by k = 6 k{=}6 . Quantitatively, even on the sharpest pair (Qwen3 14B → \to 32B), picking the best single source layer per target and averaging across target layers reaches only R 2 = 0.56 R^{2}{=}0.56 for K stripped K_{\text{stripped}} and 0.32 0.32 for V V , while k = 8 k{=}8 aggregation reaches 0.79 0.79 and 0.65 0.65 respectively. The same 0.56 0.56 appears as the production-ridge k = 1 k{=}1 row of Table 7 ; this is a weaker quantity than the per-cell heatmap peak of 0.81 0.81 in § 2.3 , which singles out the most predictable target layer. Complementary information is distributed across layers for both K and V, motivating top- k k cross-layer selection in our mapper design (§ 3.2 ). Figure 5 uses greedy forward selection (joint R 2 R^{2} maximization per target layer and head). Our production mapper (§ 3.2 ) uses fixed top- k k by single-source head-averaged R 2 R^{2} for tractability. Both support the same qualitative conclusion that complementary information is distributed across multiple source layers.

## Appendix C Extended ablation

This appendix collects the ablation tables and figures supporting § 4.3 : the source layer count sweep, the full sequential-removal table, and calibration sensitivity to ridge λ \lambda , sample count N N , and domain.

#### Source layer count.

The fit R 2 R^{2} increases with k k but with diminishing returns:

Going from k = 1 k{=}1 to k = 8 k{=}8 provides the largest K R 2 R^{2} gain. k = 8 k{=}8 to k = all k{=}\text{all} yields diminishing returns.

Downstream, the same saturation pattern holds across pairs and benchmarks (Figure 6 ). k = 1 k{=}1 is uniformly insufficient. Every pair drops sharply on at least one benchmark. Closer pairs saturate earlier. Qwen3 14B → \to 32B reaches within 0.3 0.3 pp of peak HellaSwag by k = 8 k{=}8 , while the more distant Llama 3.1 8B → \to 70B keeps improving until k = 24 k{=}24 .

#### Sequential component removal.

Removing each mapper component in turn isolates its contribution (Figure 7 and Table 2 ). § 4.3 already highlights cross-layer source selection ( k = 8 → k = 1 k{=}8\to k{=}1 ) as the largest contributor. Fitting and inferring on RoPE-coupled keys (the “ − - all RoPE” row) lands within noise of the full decoupled pipeline on every benchmark at the 1,024-token fit context. The decoupled formulation is preferred because it generalizes across RoPE configurations and to longer contexts by construction (§ 3.3 ), not because the coupled variant fails on the calibration distribution.

The numerical sequential-removal table is in the body (Table 2 , § 4.3 ).

#### Calibration protocol sensitivity.

Both ridge λ \lambda (four orders of magnitude around the production 0.01 0.01 ) and sample count N N (50–1000 sequences) have wide flat regions around the production value, because the per-head system is over-determined. Collapse appears only at extremes. λ = 1 \lambda{=}1 ( − 15.79 -15.79 pp on HellaSwag) drives the ridge penalty past the least-squares objective. N N flattens after 200 sequences, with N = 50 N{=}50 still within ∼ \sim 1.6 pp of production. Domain is the one axis with a real cost (Table 8 ).

#### Calibration domain across benchmarks.

We repeat the domain sweep across seven log-likelihood benchmarks, evaluating each of the three domain-calibrated mappers on all of them (Table 9 ). Scoring every cell by log-likelihood separates the calibration-domain effect from the additional variance of generation decoding.

Retention stays above 90 % 90\% in every cell. The two out-of-domain corpora differ in how their cost is spread. Wikipedia holds 99.4 % 99.4\% on average and varies little from benchmark to benchmark. CodeAlpaca averages 95.9 % 95.9\% but ranges over 11.1 11.1 pp, matching FineWeb-Edu on MMLU and BoolQ while costing the most on coreference and hard science QA.

## Appendix D Experimental setup details

This appendix collects the experimental-setup details referenced from § 4.1 and § 3 : the architectural axes of the transfer pairs we study (Table 10 ), the benchmarks evaluated (Table 11 ), benchmark coverage caveats, the perplexity protocol used in our family tables, and per-pair mapper sizes (Table 12 ).

#### Benchmarks.

We evaluate transfer quality on five accuracy benchmarks plus WikiText-2 perplexity, with CoQA for multi-turn handoff. Log-likelihood scoring follows the lm-evaluation-harness defaults. MMLU uses 5-shot prompts and GSM8K uses 8-shot chain-of-thought prompts [ Wei et al., 2022 ] . The rest are zero-shot.

#### Coverage.

All five accuracy benchmarks are evaluated on all six pairs at each pair’s selected k k . Transfer perplexity is reported for the Qwen3 pairs and Llama 3.1. Ministral 3 is reasoning-tuned, and its standalone WikiText-2 perplexity is orders of magnitude above the other two families’, so prefix-KV perplexity is not a meaningful comparison for it and is not reported. Missing entries appear as “—” in the per-family tables.

#### Perplexity protocol.

Standard sliding-window perplexity does not isolate KV-cache quality from the target model’s own forward pass. We split WikiText-2 into non-overlapping 2,048-token chunks and score each chunk’s second 1,024 tokens conditioned on a 1,024-token prefix KV cache, taken from either the target’s own forward pass (standalone) or from the source via the mapper (transfer). Both modes score identical continuation tokens, isolating prefix-KV quality.

#### Mapper configuration.

ARC-Challenge, HellaSwag and MMLU are evaluated at every k k . WinoGrande, the generation benchmarks and PPL are evaluated only at the selected k k and at k = all k{=}\text{all} , to bound compute cost. For Ministral 3B → \to 8B, the criterion-optimal k = 20 k{=}20 and k = all k{=}\text{all} differ by < 0.6 <0.6 pp on log-likelihood benchmarks. We use k = all k{=}\text{all} to provide a single configuration across all benchmarks.

#### Serving cost.

Mapper size is 2 ⋅ L t ⋅ n kv t ⋅ ( k ⋅ n kv s ⋅ d h s ) ⋅ d h t 2\cdot L_{t}\cdot n_{\text{kv}}^{t}\cdot(k\cdot n_{\text{kv}}^{s}\cdot d_{h}^{s})\cdot d_{h}^{t} for K and V together. It is set by target depth and head count and by k k , and does not grow with sequence length or cache size.

Mappers need not be GPU-resident: inference is one batched matmul per target layer, so a serving stack can hold them on CPU memory or disk and page in the active pair. At an assumed 25–50 GB/s host-to-device link (PCIe Gen4/Gen5), that is ∼ \sim 80–480 ms across the 4–12 GB range, paid once when a pair becomes active and amortized over its requests. These figures are computed from size and bandwidth, not measured.

The ridge fit is directional, so each direction needs its own mapper and a router over P P models covers up to P ⁡ ( P − 1 ) P(P{-}1) ordered pairs. At the average size above ( ∼ \sim 6.5 GB), 3-, 4- and 5-model fleets come to roughly 39, 79 and 131 GB. The growth is quadratic in P P , but the budget is disk or host memory, not VRAM. The six mappers in the table span 3 × 3\times around that average.

## Appendix E Run information

This appendix records the calibration data, precision regime, MLP optimization settings, and compute used. All experiments use a single 8 × 8\times H100 node.

## Appendix F Extended main results

This appendix collects the per-family standalone vs. transfer accuracies that back the headline numbers in § 4.2 (Table 13 ), the per-benchmark floor-normalized retention (Table 15 ), and a per-pair retention visualization on ARC-C, HellaSwag and MMLU (Figure 8 ).

#### Floor-normalized retention.

Table 14 reads the metric of § 4.1 at two anchors on the Qwen3 32B target. The target’s own ground-truth cache should score 100%. The 8B → \to 32B mapper stripped of RoPE separation and cross-layer selection ( k = 1 k{=}1 ), the counterpart of the fourth row of Table 2 , scores at or below chance on ARC-C, WinoGrande and MMLU, so a retention-like metric should read ≈ \approx 0% there.

Normalization changes the per-benchmark ordering. WinoGrande has the highest floor and moves the most, falling from 87.1% to 58.5% on Llama 3.1 8B → \to 70B. GSM8K is unchanged, its floor already being ≈ \approx 0, and on the strongest pair it now outranks WinoGrande.

## Appendix G Prefill latency details

This appendix extends the single-pair measurement of § 4.7 to seven pairs across the three families in both transfer directions.

#### Setup.

Measured on a single 8 × 8\times H100 node with NVLink, bf16, 50 warmup and 30 timed trials per cell. KV cache transfer latency includes the cross-GPU transfers needed to move the source cache to the target. Re-prefill runs the target’s transformer body with flash_attention_2 , excluding the LM head. The mapper runs in eager mode, with no torch.compile or CUDA graphs. At short sequences a fixed floor from Python dispatch and cross-GPU transfers dominates the mapper’s wall time (14.0 ms for the Qwen3 14B → \to 32B mapper).

#### Caveats.

Both conditions operate on synthetic inputs, which isolates compute cost. End-to-end transfer would also include shipping the mapped cache to the target process, which we do not measure. Ministral 3 re-prefill is timed on the language-model decoder body, excluding the vision tower, which is the correct comparison for text cache transfer but not a full-model forward.

## Appendix H Source-layer count selection

The source-layer count k k is chosen per pair on benchmarks that also appear in the results. This appendix gives the selection protocol and bounds how much a benchmark’s own inclusion raises its reported accuracy.

#### Protocol.