Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse β€” part 2/4

Cleaned-text code points [14469, 30351). Bibliography omitted here; source.md includes it.

Stacking N N calibration tokens gives a design matrix 𝐗 ∈ ℝ N Γ— ( k ​ n kv s ​ d h s ) \mathbf{X}\in\mathbb{R}^{N\times(k\,n_{\text{kv}}^{s}\,d_{h}^{s})} and a response matrix 𝐘 ∈ ℝ N Γ— d h t \mathbf{Y}\in\mathbb{R}^{N\times d_{h}^{t}} . We use ridge with Ξ» = 0.01 \lambda=0.01 rather than pure OLS for numerical conditioning. The feature dimension can reach tens of thousands at large k k and the top- k k selected source layers are by construction correlated, so 𝐗 ⊀ ​ 𝐗 \mathbf{X}^{\top}\mathbf{X} is near-singular. The Tikhonov term stabilizes the inverse with negligible fit bias. The closed-form solution is 𝐖 βˆ— = ( 𝐗 ⊀ ​ 𝐗 + Ξ» ​ 𝐈 ) βˆ’ 1 ​ 𝐗 ⊀ ​ 𝐘 . \mathbf{W}^{*}=(\mathbf{X}^{\top}\mathbf{X}+\lambda\mathbf{I})^{-1}\mathbf{X}^{\top}\mathbf{Y}. (4) We center 𝐗 \mathbf{X} and 𝐘 \mathbf{Y} before solving, so 𝐖 βˆ— \mathbf{W}^{*} above estimates the slope and the bias in Eq. 3 is recovered as 𝐛 = 𝐘 Β― βˆ’ 𝐗 Β― ​ 𝐖 βˆ— \mathbf{b}=\bar{\mathbf{Y}}-\bar{\mathbf{X}}\mathbf{W}^{*} . Calibration data consists of 500 FineWeb-Edu sequences of length 1024, stride-4 subsampled to N β‰ˆ 128 N\approx 128 K tokens per target head. Total mapper size (K plus V) is 2 ​ L t ​ n kv t ​ ( k ​ n kv s ​ d h s ) ​ d h t 2\,L_{t}\,n_{\text{kv}}^{t}\,(k\,n_{\text{kv}}^{s}\,d_{h}^{s})\,d_{h}^{t} , with per-pair sizes in Appendix D . Fitting takes ∼ \sim 47–87 min per pair on a single 8 Γ— 8\times H100 node with no gradient-based training. Fitting-time scaling is sub-linear in the target head count because forming 𝐗 ⊀ ​ 𝐗 \mathbf{X}^{\top}\mathbf{X} ( π’ͺ ⁑ ( N ​ d s 2 ) \mathcal{O}(Nd_{s}^{2}) , with d s = k ​ n kv s ​ d h s d_{s}=k\,n_{\text{kv}}^{s}\,d_{h}^{s} the source feature dimension) dominates wall time and is computed once per target layer, shared across its heads. Per-pair times are in Appendix E .

### 3.2 Cross-layer source selection

The source and target have different numbers of layers, raising the question of which source layers should inform each target layer’s mapping. Β§ 2.3 showed that not all source layers are equally informative for a given target layer. For each target layer l l , we therefore select the top- k k source layers by head-averaged R 2 R^{2} averaged over RoPE-stripped keys and values, concatenate their KV features, and fit the multi-source ridge regression of Β§ 3.1 : 𝐗 K l = [ 𝐊 Β― s l 1 βˆ₯ 𝐊 Β― s l 2 βˆ₯ β‹― βˆ₯ 𝐊 Β― s l k ] , \mathbf{X}_{K}^{l}=[\bar{\mathbf{K}}_{s}^{l_{1}}\,\|\,\bar{\mathbf{K}}_{s}^{l_{2}}\,\|\,\cdots\,\|\,\bar{\mathbf{K}}_{s}^{l_{k}}], (5) where 𝐊 Β― s l i ∈ ℝ T Γ— ( n kv s β‹… d h s ) \bar{\mathbf{K}}_{s}^{l_{i}}\in\mathbb{R}^{T\times(n_{\text{kv}}^{s}\cdot d_{h}^{s})} is the concatenation of all Key heads from source layer l i l_{i} , and { l 1 , … , l k } \{l_{1},\ldots,l_{k}\} are the top- k k source layers for target layer l l . 𝐗 V l \mathbf{X}_{V}^{l} is constructed analogously from Value heads. All heads within a target layer share the same selected source layers, enabling cross-head information flow. The number of source layers k k is a hyperparameter selected per pair by a sweep (Β§ 4.1 ). Cross-layer source selection is the largest single contributor among the three mapper components (Β§ 4.3 ).

### 3.3 Content-space mapping (RoPE factoring)

Standard RoPE ( Su et al., 2024 ) applies a position-dependent rotation to queries and keys. The KV cache stores only the rotated keys 𝐀 RoPE ​ ( t ) = 𝐑 Θ ​ ( t ) ​ 𝐀 content \mathbf{k}_{\text{RoPE}}(t)=\mathbf{R}_{\Theta}(t)\,\mathbf{k}_{\text{content}} , while queries are recomputed at decode time. Decoupling RoPE from content makes the mapping pipeline modular. The per-head ridge weights are computed in a position-free space, so they remain valid across pairs with different RoPE configurations and at any position the target’s RoPE supports. Fitting the mapper directly on RoPE-coupled keys also works within noise on short-context benchmarks (Table 2 ), but ties the weights to the 1024-token position distribution seen at fit. The decoupled formulation extends to longer contexts by construction, which matters for serving prompts up to 32k tokens (Β§ 4.7 ).

Concretely, we strip source RoPE, apply 𝐖 K \mathbf{W}_{K} in the position-free space, and re-encode with target RoPE: 𝐊 ^ t = ( 𝐊 s ​ 𝐑 Θ s βˆ’ 1 ​ ( t ) ​ 𝐖 K + 𝐛 K ) ​ 𝐑 Θ t ​ ( t ) \hat{\mathbf{K}}_{t}=(\mathbf{K}_{s}\,\mathbf{R}_{\Theta_{s}}^{-1}(t)\,\mathbf{W}_{K}+\mathbf{b}_{K})\,\mathbf{R}_{\Theta_{t}}(t) . During calibration the regression targets 𝐘 \mathbf{Y} are constructed by stripping the target RoPE from the target’s ground-truth keys, so 𝐖 K \mathbf{W}_{K} is fit entirely in the position-free space. Since 𝐑 Θ \mathbf{R}_{\Theta} is orthogonal, the inversion is exact at negligible cost. Values carry no positional encoding and are mapped directly.

## 4 Experiments

We evaluate four questions: (1) How well does ridge match the target’s own KV across three families (Β§ 4.2 )? (2) Which components matter (Β§ 4.3 )? (3) Can a nonlinear mapper recover the pairs where ridge falls short (Β§ 4.4 ), and what determines per-pair outcomes (Β§ 4.5 )? (4) Does transfer hold up in multi-turn (Β§ 4.6 ), and what is the latency advantage (Β§ 4.7 )?

### 4.1 Setup

#### Model families.

We evaluate three matched-KV families (Qwen3, Llama 3.1, Ministral 3) where source and target share KV head count and per-head dimension across scales (full table in Appendix D ). All families use dense full-attention, so every target layer receives mapped KV. Models are post-trained (Qwen3, Ministral 3) or base (Llama 3.1), evaluated in completion mode.

#### Benchmarks.

We evaluate on five accuracy benchmarks plus WikiText-2 perplexity: ARC-Challenge ( Clark et al., 2018 ) , HellaSwag ( Zellers et al., 2019 ) , WinoGrande ( Sakaguchi et al., 2020 ) , MMLU 5-shot ( Hendrycks et al., 2021 ) , GSM8K 8-shot with chain-of-thought prompting ( Cobbe et al., 2021 ; Wei et al., 2022 ) , and prefix-conditioned WikiText-2 perplexity. CoQA ( Reddy et al., 2019 ) measures multi-turn handoff (Β§ 4.6 ). All five accuracy benchmarks are evaluated on all six pairs at each pair’s selected k k . Perplexity coverage and its protocol are in Appendix D . Retention is ( transfer accuracy ) / ( target standalone accuracy ) Γ— 100 % (\text{transfer accuracy})/(\text{target standalone accuracy})\times 100\% . Benchmarks differ in their chance floor, so we also report floor-normalized retention, ( acc βˆ’ chance ) / ( target βˆ’ chance ) (\text{acc}-\text{chance})/(\text{target}-\text{chance}) , which places chance at 0% and the target’s own accuracy at 100% on every benchmark.

#### Mapper configuration.

All pairs use ridge ( Ξ» = 0.01 \lambda=0.01 ) and content-space mapping. The number of source layers k k is swept over { 1 , 2 , 4 , 6 , 8 , 10 , 12 , 16 , 20 , 24 , all } \{1,2,4,6,8,10,12,16,20,24,\text{all}\} . We select a single k k per pair as the value that maximizes the arithmetic mean of the log-likelihood benchmark accuracies (ARC-C, HellaSwag, WinoGrande, MMLU), breaking near-ties toward larger k k for evaluation coverage. GSM8K, CoQA multi-turn, and prefill latency are held out from this selection. Including a benchmark in the criterion raises its own reported accuracy by at most 2.49 2.49 pp (Appendix H ). Per-pair selected k k , total mapper parameters (1.01–3.36 B), storage (4–12 GB), and the family table are in Appendix D .

### 4.2 Main results

Main results focus on small-to-large (S β†’ \to L) transfer across six matched-KV pairs from three families. Large-to-small (L β†’ \to S) evaluation is limited to HellaSwag and appears in the mechanism analysis (Β§ 4.5 ) and multi-turn evaluation (Β§ 4.6 ). All pairs are evaluated at their selected k k with the full pipeline. Outcomes vary widely (42–98% Avg retention). Table 1 reports the headline per-pair numbers. Full per-family tables are in Appendix F .

Two observations structure the rest of the paper. (1) Tier 1. Four matched-KV pairs retain 73 73 – 98 % 98\% of target accuracy averaged across the five benchmarks: Qwen3 14B β†’ \to 32B, Qwen3 8B β†’ \to 32B, Llama 3.1 8B β†’ \to 70B (the most extreme parameter ratio in our evaluation), and Ministral 3B β†’ \to 8B. (2) Tier 2. Two matched-KV pairs degrade sharply: Ministral 8B β†’ \to 14B and Ministral 3B β†’ \to 14B fall to 42–44% Avg, and to 11–15% once floor-normalized. Together, the two tiers show that matched KV correlates with success but does not guarantee it. Β§ 4.4 and Β§ 4.5 investigate what additional factors determine per-pair outcomes. Figure 8 (Appendix F ) visualizes retention across benchmarks.

### 4.3 Ablation

We characterize the mapper on Qwen3 14B β†’ \to 32B, the highest-retention pair, where any degradation is most visible. Table 2 reports component ablations, and calibration sensitivity to Ξ» \lambda , N N , and calibration domain is in Appendix C .

(1) Cross-layer source selection is the largest contributor. Reducing k k from 8 8 to 1 1 drops K R 2 R^{2} from 0.79 0.79 to 0.56 0.56 (Appendix C ). (2) Content-space mapping is benchmark-specific. Disabling inference-time RoPE handling collapses MMLU and GSM8K to near random while HellaSwag falls only ∼ \sim 5 pp. (3) Calibration is robust. A four-order-of-magnitude sweep of Ξ» \lambda and a 50–1000 sequence sweep of N N both show wide flat regions, with collapse only at Ξ» = 1 \lambda{=}1 . Sample count flattens after N = 200 N{=}200 , with N = 50 N{=}50 still within ∼ \sim 1.6 pp of production. Domain is the one axis with real cost: CodeAlpaca drops 5.24 pp on HellaSwag while Wikipedia stays within noise.

### 4.4 Substituting MLP for ridge

The natural follow-up to Β§ 4.2 is whether a nonlinear mapper recovers the pairs where ridge falls short. We train a per-(target layer, head, K | | V) MLP on the same MSE loss as ridge and use it as a drop-in replacement at inference. The MLP has two 1,024-unit ReLU-activated hidden layers trained with Adam (full settings in Appendix E ). We evaluate four Qwen3 and Ministral 3 pairs covering the success-to-failure range using the same downstream evaluation pipeline as ridge, so the only factor that changes is the mapper functional form.

Table 3 shows two observations. On the pairs where ridge already succeeds, MLP slightly underperforms ridge. On the pairs where ridge fails, MLP recovers HellaSwag retention by + 24.3 +24.3 to + 36.8 +36.8 pp, which puts all four pairs above 90 % 90\% under the MLP. Linear ridge is sufficient where the cross-model KV relationship is already linear, and the MLP helps only where ridge falls short rather than dominating ridge across the board.

What changes between ridge and MLP on the failure pairs? On the HellaSwag tokens used for evaluation, ridge’s R K 2 R^{2}_{K} is deeply negative (Table 4 ), meaning the calibration-fit linear mapper does not extrapolate. MLP closes most of that gap. Β§ 4.5 shows that calibration R 2 R^{2} does not predict retention across pairs, and that this gain coincides with lower error concentration and higher attention-output cosine.

### 4.5 What determines transfer quality?

Calibration R 2 R^{2} is the natural a priori metric for transfer quality. If it predicted retention, deployment screening could rely on the fit alone. Across our six matched-KV pairs evaluated in both directions, however, R 2 R^{2} alone does not predict per-pair outcomes. Llama 3.1 8B β†’ \to 70B fits ridge with R K 2 = 0.84 R^{2}_{K}{=}0.84 on calibration data and retains 94% HellaSwag small-to-large, but only 37% large-to-small. Ministral 3B β†’ \to 8B fits at the same R K 2 = 0.84 R^{2}_{K}{=}0.84 and retains 93% in both directions. The same calibration R 2 R^{2} produces very different downstream outcomes. Β§ 4.4 additionally showed that switching ridge for an MLP recovers up to + 37 +37 pp downstream on the pairs where ridge falls short. Two questions follow. What scalar predicts retention better than R 2 R^{2} across pairs, and what does the MLP do differently from ridge on the failure pairs?

#### Attention-output cosine.

R 2 R^{2} measures how closely the mapper reconstructs each K and V channel on average, weighting all dimensions equally. Attention does not. It scores K against the target’s queries 𝐐 \mathbf{Q} and weights V by the resulting attention pattern. The quantity that decides whether downstream behavior is preserved is the attention output the target would have computed. We measure it directly with cosine similarity between the attention output from mapped KV and from ground-truth KV, averaged over layers and heads. Across 12 matched-KV pair evaluations from three families (six S β†’ \to L and six L β†’ \to S directions), mean cosine correlates with HellaSwag retention at Pearson r = + 0.57 r{=}{+}0.57 , while calibration-domain R K 2 R^{2}_{K} shows essentially no correlation ( r = βˆ’ 0.20 r{=}{-}0.20 ). R 2 R^{2} remains useful within a single pair, for example for source-layer selection, but is not the right scalar across pairs.

Cosine summarizes attention fidelity at the pair level. To explain why different mappers with similar R 2 R^{2} produce different cosine, we look at where the residual error lands per head.

#### Where the mapper’s error lands.

We measure error concentration per head. For K, we project the mapper’s per-token K error onto the right singular vectors of the target’s per-head query matrix 𝐐 h \mathbf{Q}_{h} and weight each component by the matching squared singular value. Dividing by the mean error across all components gives the K-concentration. For V, we weight per-position V error by the squared GT attention weight at that position and divide by the mean per-position error. Both quantities are averaged across heads. Concentration above 1 means the error concentrates where attention reads, while below 1 means it lands where attention ignores.

#### MLP intervention.

This explains the Β§ 4.4 puzzle and clarifies the role of the nonlinear mapper. We compare ridge and MLP on the same four pairs, same calibration data, only the mapper changing. Table 4 shows the per-pair shifts. On the pairs where ridge falls short the MLP substantially lowers K-concentration ( Ξ” β‰ˆ βˆ’ 2.5 \Delta{\approx}{-}2.5 on average) and raises attention-output cosine ( Ξ” β‰ˆ + 0.45 \Delta{\approx}{+}0.45 ), with HellaSwag retention rising by + 24.3 +24.3 to + 36.8 +36.8 pp. Where ridge already succeeds the shifts are far smaller and retention does not follow: Ministral 3B β†’ \to 8B improves on both quantities and still loses ground on HellaSwag. Redistributing error is therefore not sufficient on its own. It changes downstream accuracy only where the misplaced error was large enough to bind, which is why linear ridge is enough wherever the cross-model KV relationship is already linear. The same shift lifts eval-domain R K 2 R^{2}_{K} from deeply negative to near zero on the failure pairs, though it stays below zero: ridge’s calibration-domain fit does not extrapolate to HellaSwag tokens, while the MLP closes most of that gap.

### 4.6 Multi-turn handoff

The mid-conversation switching scenario alternates between source and target across turns, so we measure drift on Qwen3 14B ↔ \leftrightarrow 32B with CoQA ( Reddy et al., 2019 ) on 100 conversations of ∼ \sim 15 turns each across five domains. We score each turn’s answer with F1 against the ground-truth and define drift at turn t t as the F1 gap between the target’s standalone accuracy and the mapper’s accuracy at the same turn. Drift stays small in both directions (Figure 4 ). The small-to-large gap widens by 1.7 1.7 pp from turn 1 to turn 10, with the mapper holding steady while the 32B ceiling rises. Large-to-small drift grows linearly at 0.33 0.33 pp/turn. Both slopes are too small for cascading failure within ten turns on this single-pair evaluation, though linear large-to-small drift would still accumulate over very long sessions. Tuning k k on the multi-turn task changes drift by at most 2.0 2.0 pp (dotted lines).

### 4.7 Prefill latency