Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse — part 1/4

Cleaned-text code points [0, 14469). Bibliography omitted here; source.md includes it.

# Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse

###### Abstract

Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer , where the receiver reuses the source’s KV cache, skipping prefill. We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension. On Qwen3 14B → \to 32B, one source layer explains 56% of variance in the target’s keys and 32% in values, rising to 79% and 65% with multiple source layers. Building on this, we design a closed-form ridge mapper that operates per head and proceeds in three steps. First, for each target layer we select the top- k k most predictive source layers and concatenate their KV as input. Second, we strip RoPE from the keys before mapping, so the fit is position-free and reusable across context lengths. Third, we fit ridge regression on a small calibration set of 500 FineWeb-Edu sequences of 1,024 tokens each. Surprisingly, across six pairs in three families, this linear mapper retains 73–98% of the receiver’s standalone-prefill accuracy on four pairs, while two degrade sharply. A nonlinear MLP recovers up to + 37 +37 pp HellaSwag retention on the failures. The mapper runs 2.7–25 × \times faster than re-prefill and remains stable across multi-turn handoff, making cross-model KV cache transfer practical.

## 1 Introduction

Production LLM serving increasingly involves long agentic sessions, where context accumulates across many turns. It also relies on multi-model orchestration for cost-quality cascading, mid-conversation switching, and routing ( OpenAI, 2025 ) . These practices swap between different-sized members of a model family ( Yang et al., 2025 ; Grattafiori et al., 2024 ) . Family members share core architectural choices and substantial pretraining data, but differ in architectural details and training recipe across scales. Both trends compound prefill cost. Long sessions stretch the prompt, and each model swap re-prefills the accumulated context on the receiver. Prefill is the forward pass that populates the KV cache before generation. Its cost scales with model size and prompt length. Prefix caching mitigates this within a single model only.

Since prefill’s output is the KV cache, reusing it across models reduces to a representation problem of transforming one model’s KV cache into another’s expected format. We call this cross-model KV cache transfer and restrict the present study to within-family transfer. The mapping is bidirectional. Small-to-large transfer upgrades quality, and large-to-small reduces cost. Figure 1 illustrates the pipeline. This pipeline generalizes intra-model KV reuse. Cross-layer sharing ( Brandon et al., 2024 ; Wu and Tu, 2024 ; Chang et al., 2025 ) exploits redundancy within a model, and prefix caching ( Zheng et al., 2024 ; Gim et al., 2024 ) reuses KV across requests of the same model. Cross-model KV cache transfer maps the cache values from one model to another .

Cross-model KV cache transfer is challenging because source and target can differ in layer count, hidden dimension, and KV head configuration. Prior work bridges these gaps with neural fusers ( Fu et al., 2026 ) , learned latent-space adapters ( Dery et al., 2026 ) , attention-pattern mapping ( Zhao et al., 2025 ) , or same-architecture KV sharing ( Liu et al., 2026 ) , each requiring gradient-based training or strong architectural assumptions (§ 2.2 ).

In this study, we show that the cross-model KV relationship exhibits substantial linear structure, admitting a small-sample, gradient-free fit. On Qwen3 14B → \to 32B, a single source layer explains 56% of variance in the target’s keys and 32% in values, rising to 79% and 65% with multiple source layers (§ 2.3 ). Building on this observation, we propose a closed-form per-head ridge mapper (§ 3 ) that combines three components: per-head ridge regression fit from a small calibration set, cross-layer source selection where each target layer draws from its top- k k most predictive source layers, and content-space (RoPE-stripped) mapping that decouples positional rotation from semantic content so the fit transfers across context lengths.

Across six matched-KV pairs from three families, where source and target share KV head count and per-head dimension (§ 4.2 ), the linear mapper retains 73–98% of standalone accuracy averaged across five benchmarks on its best pairs. We further find that error placement, not error magnitude, determines per-pair retention. A nonlinear MLP redistributes residual error away from attention-sensitive subspaces and adds up to + 37 +37 pp HellaSwag retention on harder pairs (§ 4.4 ). Across 12 matched-KV pair evaluations from three families, attention-output similarity correlates with HellaSwag retention at Pearson r = + 0.57 r{=}{+}0.57 and outperforms R 2 R^{2} (§ 4.5 ).

#### Contributions.

(1) Closed-form mapping framework. We propose a gradient-free framework for cross-model KV cache transfer based on per-head ridge regression. Two design choices are central: each target layer is fit from its top- k k most predictive source layers, and keys are mapped in RoPE-stripped content space so the fit transfers across context lengths (§ 3 ). (2) Multi-family validation. We validate the framework on six matched-KV pairs across three families. Four pairs retain 73–98% of standalone accuracy on five benchmarks at 2.7–25 × \times lower prefill latency than re-prefill in the small-to-large direction (§ 4.7 ). (3) Nonlinear extension. A nonlinear MLP adds up to + 37 +37 pp HellaSwag retention on harder Ministral pairs by redistributing residual error away from attention-sensitive subspaces (§ 4.4 , § 4.5 ). (4) Attention-output cosine predicts cross-pair retention. Across 12 matched-KV pair evaluations from three families, attention-output cosine correlates with HellaSwag retention at Pearson r = + 0.57 r{=}{+}0.57 , outperforming R 2 R^{2} ( r = − 0.20 r{=}{-}0.20 ). The same concentration analysis explains the + 37 +37 pp MLP gain as error redistribution toward attention-irrelevant directions (§ 4.5 ).

## 2 Background and motivation

### 2.1 Problem formulation

Consider a source model 𝒮 \mathcal{S} with L s L_{s} layers and a target model 𝒯 \mathcal{T} with L t L_{t} layers. Both use grouped-query attention with n kv s n_{\text{kv}}^{s} and n kv t n_{\text{kv}}^{t} KV heads of dimension d h s d_{h}^{s} and d h t d_{h}^{t} , respectively. For an input sequence 𝐱 = ( x 1 , … , x T ) \mathbf{x}=(x_{1},\ldots,x_{T}) of length T T , layer l ∈ { 1 , … , L s } l\in\{1,\ldots,L_{s}\} , head h ∈ { 1 , … , n kv s } h\in\{1,\ldots,n_{\text{kv}}^{s}\} of the source model produces keys and values 𝐊 s l , h , 𝐕 s l , h ∈ ℝ T × d h s \mathbf{K}_{s}^{l,h},\mathbf{V}_{s}^{l,h}\in\mathbb{R}^{T\times d_{h}^{s}} . We write 𝒞 𝒮 \mathcal{C}_{\mathcal{S}} for the full source KV cache across all layers and heads, and 𝒞 𝒯 \mathcal{C}_{\mathcal{T}} analogously for the target. Source and target share a tokenizer within a family, so the input sequence has the same length T T for both. We say a transfer pair has matched KV when n kv s = n kv t n_{\text{kv}}^{s}=n_{\text{kv}}^{t} and d h s = d h t d_{h}^{s}=d_{h}^{t} , even if L s L_{s} , L t L_{t} , or parameter counts differ across scales.

We seek a mapping f : 𝒞 𝒮 → 𝒞 ^ 𝒯 f\colon\mathcal{C}_{\mathcal{S}}\to\hat{\mathcal{C}}_{\mathcal{T}} such that the target model, decoding from 𝒞 ^ 𝒯 \hat{\mathcal{C}}_{\mathcal{T}} in place of its own 𝒞 𝒯 \mathcal{C}_{\mathcal{T}} , produces equivalent outputs: m ⁡ ( 𝒯 ⁡ ( 𝐱 , 𝒞 ^ 𝒯 ) ) ≈ m ⁡ ( 𝒯 ⁡ ( 𝐱 , 𝒞 𝒯 ) ) , m\bigl(\mathcal{T}(\mathbf{x};\hat{\mathcal{C}}_{\mathcal{T}})\bigr)\;\approx\;m\bigl(\mathcal{T}(\mathbf{x};\mathcal{C}_{\mathcal{T}})\bigr), (1) for a downstream task τ \tau with metric m m . We decompose f f into per-head mappings f K l , h f_{K}^{l,h} and f V l , h f_{V}^{l,h} that produce the target’s keys and values for each layer and head. We measure transfer quality by the target model’s downstream accuracy, not by reconstruction metrics alone.

### 2.2 Prior approaches

Recent cross-model KV-reuse methods each require either gradient-based training or architectural constraints. C2C ( Fu et al., 2026 ) trains per-pair neural fusers. LatentAlign ( Dery et al., 2026 ) learns per-model adapters into a shared latent space. IAM ( Zhao et al., 2025 ) substitutes small-model attention patterns , not KV values. DroidSpeak ( Liu et al., 2026 ) requires identical architecture. Further training-time approaches ( Woo et al., 2026b ; Woo et al., 2026a ) share the same gradient-based constraint. To the best of our knowledge, none investigates whether the cross-model KV relationship is simple enough for a closed-form, training-free mapping. Three orthogonal lines compose with the present work: intra-model cross-layer KV reuse ( Brandon et al., 2024 ; Wu and Tu, 2024 ; Chang et al., 2025 ) , prefill acceleration ( Liu et al., 2025 ; Upasani et al., 2026 ; Qiao et al., 2025 ) , and linear cross-LLM representation alignment ( Huh et al., 2024 ; Chen et al., 2025 ; Bello et al., 2025 ; Huang et al., 2025 ) . Table 6 (Appendix A ) summarizes the cross-model comparison.

### 2.3 Linear structure in cross-model KV

Before committing to a mapper architecture, we ask what structure the cross-model KV relationship has. We analyze this structure on Qwen3. The linear-structure result generalizes to other matched-KV pairs. We probe three cache types. K rope K_{\text{rope}} denotes keys as the model uses them. K stripped K_{\text{stripped}} denotes keys with the position-dependent RoPE rotation removed via 𝐑 Θ − 1 \mathbf{R}_{\Theta}^{-1} (see § 3.3 ). V V denotes values, which carry no positional encoding. For each cache type C ∈ { K rope , K stripped , V } C\in\{K_{\text{rope}},K_{\text{stripped}},V\} and each triple of source layer l ′ ∈ { 1 , … , L s } l^{\prime}\in\{1,\ldots,L_{s}\} , target layer l ∈ { 1 , … , L t } l\in\{1,\ldots,L_{t}\} , and head h h , we fit a single-source ordinary least-squares regression at the token level. Each of the N N calibration tokens, subsampled from FineWeb-Edu sequences as detailed in § 3.1 , contributes one observation, mapping the source’s per-token feature vector C s l ′ , h ∈ ℝ d h s C_{s}^{l^{\prime},h}\in\mathbb{R}^{d_{h}^{s}} to the target’s C t l , h ∈ ℝ d h t C_{t}^{l,h}\in\mathbb{R}^{d_{h}^{t}} , C ^ t l , h = C s l ′ , h ​ 𝐖 + 𝐛 , 𝐖 ∈ ℝ d h s × d h t , 𝐛 ∈ ℝ d h t , \hat{C}_{t}^{l,h}=C_{s}^{l^{\prime},h}\,\mathbf{W}+\mathbf{b},\qquad\mathbf{W}\in\mathbb{R}^{d_{h}^{s}\times d_{h}^{t}},\ \mathbf{b}\in\mathbb{R}^{d_{h}^{t}}, (2) treating C s l ′ , h C_{s}^{l^{\prime},h} as a row vector to match the production-mapper convention in § 3 . This is a per-(triple, cache type) probe, not the production mapper of § 3 , which concatenates several source layers and adds ridge regularization. We measure fit quality by the coefficient of determination R 2 R^{2} , averaged across the n kv t n_{\text{kv}}^{t} target heads to give a head-averaged R 2 R^{2} for each ( l ′ , l ) (l^{\prime},l) pair. Visualizing this as a heatmap with rows indexing source layers and columns indexing target layers (Figure 2 ) reveals four qualitative patterns. Per-pair quantitative numbers are reported in § 4 .

(1) High linear fit: a single linear regression already captures a large fraction of cross-model KV variance on the most predictable target layers. Head-averaged R 2 R^{2} is well above zero along the diagonal, and the best single source–target cell reaches K stripped K_{\text{stripped}} R 2 = 0.81 R^{2}{=}0.81 on Qwen3 14B → \to 32B and 0.65 0.65 on the weaker 8B → \to 32B at the heatmap peaks. Layer-averaged production-ridge R 2 R^{2} is reported in Appendix C . (2) Closer models give sharper diagonals: larger architectural and depth gaps diffuse the pattern. (3) RoPE contaminates the fit: stripping RoPE generally sharpens the diagonal, motivating the position–content decomposition in § 3.3 . (4) K is more predictable than V: typically a ∼ \sim 0.2 gap in head-averaged R 2 R^{2} (Appendix B ).

#### How many source layers does the mapper need?

The probe uses a single source layer. We extend it with greedy forward selection, iteratively adding the source layer that maximally increases R 2 R^{2} . Complementary information is distributed across multiple source layers. On Qwen3 14B → \to 32B, k = 1 k{=}1 captures only 66 % 66\% of the k = all k{=}\text{all} R 2 R^{2} for K stripped K_{\text{stripped}} and 42 % 42\% for V V , with the largest gain from k = 1 k{=}1 to k = 4 k{=}4 and R 2 R^{2} close to its k = all k{=}\text{all} value by k = 6 k{=}6 (Appendices B and C ). This motivates the top- k k selection (§ 3.2 ).

## 3 Mapper design

Figure 3 illustrates the mapper for one target (layer l l , head h h ) pair. A top- k k subset of source layers is selected per target layer by R 2 R^{2} , the selected source keys and values are concatenated, and independent ridge regressions 𝐖 K \mathbf{W}_{K} , 𝐖 V \mathbf{W}_{V} map them to the target. Each mapper is a closed-form linear solve repeated independently for all L t × n kv t L_{t}\times n_{\text{kv}}^{t} target heads. The mapper has three components: per-head ridge (§ 3.1 ), cross-layer source selection (§ 3.2 ), and content-space mapping (§ 3.3 ).

### 3.1 Per-head ridge regression

Source and target differ in layer count ( L s ≠ L t L_{s}\neq L_{t} ), head dimension ( d h s ≠ d h t d_{h}^{s}\neq d_{h}^{t} ), and potentially KV head count ( n kv s ≠ n kv t n_{\text{kv}}^{s}\neq n_{\text{kv}}^{t} ). We address this by fitting an independent linear mapping per target (layer, head): 𝐊 ^ t l , h = 𝐗 K l ​ 𝐖 K l , h + 𝐛 K l , h , 𝐕 ^ t l , h = 𝐗 V l ​ 𝐖 V l , h + 𝐛 V l , h , \hat{\mathbf{K}}_{t}^{l,h}=\mathbf{X}_{K}^{l}\mathbf{W}_{K}^{l,h}+\mathbf{b}_{K}^{l,h},\qquad\hat{\mathbf{V}}_{t}^{l,h}=\mathbf{X}_{V}^{l}\mathbf{W}_{V}^{l,h}+\mathbf{b}_{V}^{l,h}, (3) where 𝐗 K l \mathbf{X}_{K}^{l} concatenates source key features from the k k selected source layers (§ 3.2 ) and 𝐖 K l , h ∈ ℝ ( k ⋅ n kv s ⋅ d h s ) × d h t \mathbf{W}_{K}^{l,h}\in\mathbb{R}^{(k\cdot n_{\text{kv}}^{s}\cdot d_{h}^{s})\times d_{h}^{t}} is the weight matrix.

#### Fitting.