XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communication — part 3/4

Cleaned-text code points [31223, 45920). Bibliography omitted here; source.md includes it.

XBridge achieves 11 × \times lower latency than NLComm (0.15 s vs. 1.70 s per sample on HotpotQA) by eliminating autoregressive decoding: the sender performs a single forward pass ( ∼ {\sim} 0.05 s) with negligible bridge overhead ( < < 0.01 s), as reported in Table 5 . The modular bridge design also enables zero-shot composability: two independently trained bridges (Llama and Mistral as senders) combine at inference without retraining, achieving 70.4% F1 on HotpotQA compared to 67.0% for single-sender full context and 56.8% for dual NLComm (Appendix C.3 ). Finally, LAM is sender-size invariant since it depends only on the vocabulary mapping, not the sender’s capacity: even a 1.5B sender achieves 59.5%, approaching FullComm, while the 7B and 14B senders surpass it (Table 5 ).

## 5 Conclusion

In this work, we identified the entity grounding problem in heterogeneous LLM communication, where continuous bridges lose entity identity in the latent bottleneck. We proposed XBridge , a decode-free protocol that addresses this by combining LAM with LEB, decomposing communication into discrete entity anchors and continuous contextual enrichment. Extensive experiments across three model families, seven benchmarks, and both communication directions demonstrated that XBridge outperforms NLComm on every task with 11 × \times lower latency. We highlight the complementarity of the two mechanisms: LAM determines which entities appear in the output, while LEB determines how the receiver reasons about them, as confirmed by the entity perturbation experiment. Our work shows that heterogeneous LLMs can communicate at the latent level while preserving entity fidelity, and we see this as an early step toward cross-architecture multi-agent collaboration.

## Limitations

XBridge is trained for a single sender to receiver direction, so one bridge carries information one way over one exchange. A two-way setup would add a second bridge for the reverse direction, and since each bridge is lightweight and decode-free, the cost per exchange would stay low, but multi-turn latent dialogue, where the two models alternate and each turn conditions on what the other sent, is beyond what we evaluate here. The same holds for the number of agents: we have tested composition of two independently trained bridges, leaving open how bridge signals interact when more agents participate. Extending the protocol along both directions, bidirectional exchange and many-agent composition, is what we see as the path toward a cross-architecture communication protocol suited to full multi-agent systems.

## Appendix A LAM Details

### A.1 Cross-Vocabulary Mapping

Table 6 summarizes the vocabulary overlap and token transfer statistics for the two sender–receiver pairs used in our experiments.

Table 7 shows representative examples of tokenization mismatches and how the string-based fallback resolves them.

The mismatch patterns reflect fundamental differences in tokenizer design. Llama-3.1 uses a 128K BPE vocabulary that overlaps substantially with Qwen2.5’s 152K vocabulary, resulting in mismatches confined almost entirely to multi-digit numbers. Mistral-7B uses a smaller 32K SentencePiece vocabulary, causing common English words and proper nouns to fall back to string-based re-tokenization. In both cases, the fallback is lossless: the surface string content is fully preserved, with only positional expansion as a side effect.

### A.2 Grounding Format Independence

A critical question is whether LEB’s effectiveness depends on the specific grounding format (embedding prepend vs. text prompt) or merely on the presence of entity anchors.

LEB consistently improves all grounding formats ( + 12.5 +12.5 to + 22.3 +22.3 pp), demonstrating that entity presence, not format, is what enables LEB. Whether entity anchors arrive as prepended embeddings, as text within a chat template, or as argmax-decoded tokens, LEB extracts contextual enrichment from the sender’s H S H_{S} via cross-attention regardless of how those anchors enter the receiver.

The smaller Δ \Delta for NLComm ( + 12.5 +12.5 pp) reflects its higher base: NLComm text already contains sender reasoning, leaving less room for LEB improvement. Notably, NLComm with LEB (81.0%) surpasses LAM with LEB (78.8%), because the sender’s autoregressive decoding performs reasoning compression that benefits the receiver. However, LAM remains the preferred grounding format because it eliminates decoding latency entirely ( 14 × 14\times faster than NLComm) while achieving comparable performance with LEB.

## Appendix B LEB Design

### B.1 Bridge Architecture Ablations

We ablate bridge design decisions on HotpotQA (Llama → \to Qwen). All variants use balanced training (587 samples).

#### Number of bridge modules.

We vary the number of cross-attention modules inserted into the receiver (Qwen2.5-7B, 28 layers). 4 modules (264M parameters) is optimal. 7 modules (462M) overfits due to the higher parameter-to-data ratio with balanced training. 2 modules (132M) provides insufficient capacity for cross-architecture translation. This follows Flamingo’s [ 9 ] finding that sparser insertion is preferred for larger models.

#### Sender layer selection.

Bridge performance increases monotonically from early to final sender layers, with layers before the midpoint falling below the no-bridge baseline. The last layer is optimal because it integrates information across all context positions through the sender’s full attention stack, producing the most globally-aware representations.

#### Gate initialization.

We initialize the tanh \tanh gate at α = 1.0 \alpha=1.0 ( tanh ⁡ ( 1.0 ) ≈ 0.76 \tanh(1.0)\approx 0.76 ), differing from Flamingo’s zero initialization. With α = 0 \alpha=0 , the bridge is initially invisible to the receiver, requiring training to “discover” the bridge signal. With α = 1.0 \alpha=1.0 , the bridge immediately contributes cross-architecture context. In our setting with limited training data (587 samples), warm initialization converges faster and achieves higher final performance.

#### Auxiliary losses.

Adding a contrastive loss (encouraging the bridge to distinguish correct sender context from random contexts) does not improve performance beyond NTP loss alone. The NTP objective already implicitly requires the bridge to extract useful information from the sender’s hidden states.

### B.2 Bridge Training Data Composition

The composition of bridge training data significantly affects per-task performance.

HotpotQA-only bridge (20K samples) is inferior even on its own training domain (76.1% vs. 78.8% for the balanced bridge), and catastrophically degrades on unseen task types: Countries drops to 27.7% and QASPER to 36.2%. Overfitting to a single task’s context-length distribution and reasoning pattern prevents the bridge from generalizing.

Unbalanced universal bridge (42K samples, 94% HotpotQA+MuSiQue) improves over the single-task bridge across all tasks but remains below the balanced bridge on average (59.3% vs. 63.2%). The heavy skew toward multi-hop QA patterns limits generalization to dissimilar tasks such as Countries and QASPER.

Balanced universal bridge (587 samples, ∼ \sim 100 per task) achieves the best performance on every metric despite using 30 × 30\times less training data than the unbalanced variant. The balanced bridge learns task-appropriate gating: it remains near-closed for near-saturated factual tasks (Tipsheets 99.8%) while actively contributing on complex reasoning tasks (HotpotQA 78.8%, Countries 72.5%). The average across all 7 tasks (63.2%) is the highest among all bridge variants.

#### Implication for sender size scaling.

The balanced training set (587 samples) is sufficient for 7B senders but becomes a bottleneck for larger models. The 14B sender (hidden dim 5120) produces richer representations that require more training data to learn effective projections. Within this data budget, the 14B bridge plateaus at 62.1%, matching the 7B sender (61.9%) despite its larger capacity. We expect that scaling the balanced training set would unlock the 14B sender’s advantage, but leave this to future work.

## Appendix C Additional Results

### C.1 Homogeneous Comparison

To verify that XBridge generalizes beyond heterogeneous settings, we train a same-architecture bridge using Qwen2.5-7B as both sender and receiver. LAM reduces to an identity mapping in this setting, so the comparison isolates LEB’s contribution on top of full context reading.

Table 11 compares XBridge homo {}_{\text{homo}} against NLComm homo and KVComm [ 5 ] . XBridge homo {}_{\text{homo}} outperforms NLComm homo on all 7 tasks and KVComm on 6 of 7 tasks, with an average improvement of +12.0 pp over the stronger baseline. Notably, XBridge homo {}_{\text{homo}} surpasses even FullComm on 6 of 7 tasks (+7.9 pp on average), demonstrating that LEB adds value beyond what the receiver extracts from reading the context alone. The sole exception is MultiFieldQA, consistent with the pattern observed in heterogeneous settings (Section 4.3 ).

### C.2 Bridge Contribution per Task

Table 12 breaks down LEB’s contribution across all seven tasks for the Llama → \to Qwen pair. LEB contribution scales with the reasoning complexity of the task.

On Countries, the context states a person’s location at a landmark (e.g., “Eve is at the Uppland Runic Inscription 896”) and the question asks which country the person is in, but the answer (“Sweden”) appears nowhere in the context. Only LEB can transmit the sender’s implicit landmark-to-country inference, yielding the largest contribution (+35.8 pp). On multi-hop tasks (HotpotQA, 2WikiMQA, MuSiQue), LEB transmits reasoning chains integrated across documents (+19 to +22 pp).

On MultiFieldQA, the question asks for a specific fact (e.g., “What is the advantage of decorrelating the data before running the PLS algorithm?”) and the answer is a single extractable sentence located deep within a ∼ {\sim} 7K-token document. No cross-entity reasoning is needed, and LEB adds only +7.0 pp. This is also the sole task where KVComm [ 5 ] outperforms XBridge (48.3 vs. 44.2): same-architecture KV sharing preserves position-level attention patterns that directly encode answer location, an advantage specific to retrieval tasks that disappears when reasoning is required (e.g., HotpotQA +13.5 pp, MuSiQue +15.1 pp over KVComm).

On Tipsheets, the answer is directly extractable from structured text, leaving minimal room for LEB improvement (+2.0 pp).

### C.3 Multi-Agent Communication

We test composability on HotpotQA, where each question requires reasoning over two supporting documents. We assign each document to a separate sender, Llama for the first and Mistral for the second, and let the receiver (Qwen) integrate both via their respective bridges. Each bridge was trained independently for its own model pair; no joint training or adaptation is performed.

Dual XBridge achieves 70.4% F1 (Table 13 ), outperforming both the single-sender full-context baseline (67.0%, +3.4 pp) and dual NLComm (56.8%, +13.6 pp). The single-sender baseline gives one Llama sender both documents, so the dual setup’s improvement demonstrates that distributing documents across specialized senders is more effective than compressing everything through a single sender. This zero-shot composability, combining independently trained bridges without additional training, suggests that the framework scales naturally to multi-party agent settings.

## Appendix D Additional Analysis

### D.1 Per-Sample Consistency

The bridge mechanism analysis in Section 4.4 reports aggregate metrics across 196 HotpotQA samples (Llama → \to Qwen). Here we verify that these improvements are consistent at the individual sample level rather than driven by a few outliers.

Figure 4 (c) plots the mean-centered cosine similarity between the generation representation and the answer entity representation for each sample, comparing NoComm and XBridge . Points above the y = x y=x line indicate samples where LEB improves alignment. Of 196 samples, 184 (94%) fall above y = x y=x ( p ≈ 0 p\approx 0 , paired t t -test). The population mean shifts from μ = − 0.053 \mu=-0.053 (NoComm) to μ = + 0.091 \mu=+0.091 ( XBridge ), a sign change indicating that without LEB the receiver’s generation state moves away from the answer entity, while with LEB it moves toward it.

The 12 samples (6%) where LEB does not improve alignment tend to involve short, unambiguous contexts where LAM alone already provides sufficient grounding. Even in these cases, LEB does not substantially degrade performance, indicating that the gating mechanism (Equation 11 ) successfully attenuates the bridge signal when it is not needed.

### D.2 Entity Perturbation Examples

Table 14 shows representative examples from the entity perturbation experiment (Section 4.4 ). In each case, swapping LEB input alone does not change the output entity, while swapping LAM changes it entirely.

In all three examples, conditions A and D produce identical outputs, confirming that LEB does not influence entity selection. Condition C shifts the output to the swapped entity every time. LEB adapts its contextual enrichment accordingly: contextual signals such as “port city 25 km north of the museum” or “studied under in the Peripatetic school” bind to whichever entity LAM provides.

## Appendix E Task Details

Table 15 summarizes the seven benchmarks used in our evaluation. Training and evaluation splits are strictly separated with zero overlap.

∗ MuSiQue statistics shown are for supporting paragraphs only; full context with distractors is approximately 4–5 × \times longer. All evaluations use the full context.

## Appendix F Related Work

#### Text-based multi-agent communication.

LLMs are increasingly deployed in multi-agent systems where agents collaborate to solve tasks beyond individual capability [ 16 , 17 ] . Multi-agent debate improves factuality by having models critique each other’s reasoning [ 4 , 18 ] , while frameworks such as CAMEL [ 19 ] , AutoGen [ 20 ] , and ChatDev [ 21 ] coordinate agents through structured natural language exchanges. Natural language communication (NLComm) [ 4 ] generalizes this pattern: one model generates a text summary and relays it to another. These text-based protocols are architecture-agnostic but require autoregressive decoding at every communication step, incurring high latency. They also discard the sender’s internal representations, compressing context into a short summary that loses both the sender’s contextual understanding and relational structure among entities.

#### Latent multi-agent communication.