Communicating Activations Between Language Model Agents — part 1/3

Cleaned-text code points [0, 14463). Bibliography omitted here; source.md includes it.

# Communicating Activations Between Language Model Agents

###### Abstract

Communication between multiple language model (LM) agents has been shown to scale up the reasoning ability of LMs. While natural language has been the dominant medium for inter-LM communication, it is not obvious this should be the standard: not only does natural language communication incur high inference costs that scale quickly with the number of both agents and messages, but also the decoding process abstracts away too much rich information that could be otherwise accessed from the internal activations. In this work, we propose a simple technique whereby LMs communicate via activations ; concretely, we pause an LM B B ’s computation at an intermediate layer, combine its current activation with another LM A A ’s intermediate activation via some function f f , then pass f f ’s output into the next layer of B B and continue the forward pass till decoding is complete. This approach scales up LMs on new tasks with zero additional parameters and data, and saves a substantial amount of compute over natural language communication. We test our method with various functional forms f f on two experimental setups—multi-player coordination games and reasoning benchmarks—and find that it achieves up to 27.0 % 27.0\% improvement over natural language communication across datasets with < < 1 / 4 1/4 the compute, illustrating the superiority and robustness of activations as an alternative “language” for communication between LMs.

###### Keywords:

## 1 Introduction

Language is for the purpose of communication. As large language models (LLMs) have been increasingly used to power autonomous, goal-driven agents capable of reasoning, tool usage, and adaptive decision-making ( Yao et al., 2023 ; Xi et al., 2023 ; Wang et al., 2024 ; Ahn et al., 2022 ; Schick et al., 2023 ; Shen et al., 2023 ; Park et al., 2023 ; Nakano et al., 2022 ) , communication between multiple cooperating agents has emerged as an intuitive approach to amplify the reasoning capabilities of LLMs ( Wu et al., 2023 ) . Explicit communication in natural language between multiple LLMs has been shown to encourage divergent thinking ( Liang et al., 2023 ) , improve factuality and reasoning ( Du et al., 2023 ) , enable integration of cross-domain knowledge ( Sukhbaatar et al., 2024 ) , and allow for modular composition of abilities in a complementary manner ( Wu et al., 2023 ; Prasad et al., 2023 ) .

A critical problem with natural language communication, however, is that it incurs extremely high inference costs that scale quickly with the number of agents as well as length and number of messages ( Du et al., 2023 ; Yang et al., 2023 ; Wu et al., 2023 ) . Restricting LLM communication to natural language also raises the question: as LLMs are increasingly capable of handling larger, more complex tasks (sometimes with “super-human” ability) ( Wei et al., 2022 ; Burns et al., 2023 ) , might they communicate more effectively in representations of higher dimension than natural language? While using natural language as a communicative medium is appealing due to its interpretability, we claim that it may not be optimal for inter-LLM communication. Natural language generation uses only one token to represent the model’s belief over the entire vocabulary, which risks losing information embedded within the model output logits ( Pham et al., 2024 ) ; furthermore, a model’s belief over the entire vocabulary is itself not always better (for communicative purposes) than the model’s (often richer) representation of the input in earlier layers. Indeed, Hernandez et al. (2024) find that by around the halfway point of an LM’s computation, it has developed “enriched entity representations” of the input, where entities in the prompt are populated with additional facts about that entity encoded in the model’s weights; but by the later layers these embeddings are transformed into a representation of the next word which leverages only parts of the previous, richer representations, when that full embedding would be quite useful for communication.

Motivated by these concerns, this work outlines a simple technique whereby LLM agents communicate via activations , thus enabling more efficient (i.e., higher-entropy) communication at a fraction of the number of forward passes required at inference time. Concretely, we (1) pause a Transformer LM B B ’s computation at intermediate layer j j in the residual stream; (2) combine its post-layer j j activation with another LM A A ’s post-layer k k activation via some function f f ; and then (3) pass f f ’s output into the next layer j + 1 j+1 of B B and continue its forward pass till decoding is complete. This approach scales up LLMs on new tasks by leveraging existing, frozen LLMs along with zero task-specific parameters and data, applying to diverse domains and settings. Furthermore, in requiring only a partial forward pass through A A and one forward pass through B B , this method saves a substantial amount of compute over traditional natural language communication, which we quantify in Section 3.2 .

We validate our method by testing this approach with various functional forms f f on two experimental setups: two multi-player coordination games, where B B is asked to complete a task requiring information provided in a prompt to A A ; and seven reasoning benchmarks spanning multiple domains: Biographies ( Du et al., 2023 ) , GSM8k ( Cobbe et al., 2021 ) , MMLU High School Psychology, MMLU Formal Logic, MMLU College Biology, MMLU Professional Law, and MMLU Public Relations ( Hendrycks et al., 2021 ) . Our activation communication protocol exhibits up to 27.0 % 27.0\% improvement over natural language communication across these datasets, using < < 1 / 4 1/4 the compute. Critically, unlike prior work which test inter-LLM communication only on large-scale ( > > 70 70 B) models ( Du et al., 2023 ; Liang et al., 2023 ) , we find that our approach generalizes across a wide array of LLM suites and sizes, enabling even smaller LLMs to unlock the benefits of communication.

In summary, our contributions are two-fold: • We propose a novel inter-model communication protocol for LLM agents that is purely activation-based.

• We perform comprehensive experiments to validate the improved performance of activation communication over traditional natural language communication. We also formally quantify our approach’s compute savings over natural language communication, illustrating the superiority and robustness of activations as an alternative “language” for communication between LMs.

## 2 Related Work

#### Multi-agent communication

The field of multi-agent communication has a long-standing history. Notably, prior works on emergent communication have showed that agents can autonomously evolve communication protocols when deployed in multi-agent environments that enable cooperative and competitive game-play ( Sukhbaatar et al., 2016 ; Foerster et al., 2016 ; Lazaridou et al., 2017 ) . However, recent experiments have demonstrated that learning meaningful languages from scratch, even with centralized training, remains difficult ( Lowe et al., 2020 ; Chaabouni et al., 2019 ; Jaques et al., 2019 ) .

With the emergence of large pre-trained language models, allowing communication between LLMs in natural language has hence become a promising approach to enable coordination among multiple LLM agents ( Li et al., 2023 ) . Recent works have demonstrated that such conversations enable integration of cross-domain knowledge ( Sukhbaatar et al., 2024 ) , modular composition of abilities in a complementary manner ( Wu et al., 2023 ) , and improved task performance via splitting into subtasks ( Prasad et al., 2023 ) . Most notable is multiagent debate introduced by Du et al. (2023) , where LLMs provide initial responses and then make refinements by iteratively considering inputs from peers. While such methods have been shown to improve performance on various tasks over vanilla and majority-vote ( Wang et al., 2023 ) style prompting, these experiments have only focused on large models ( 𝙶𝙿𝚃 \mathtt{GPT} - 3.5 / 𝟺 \mathtt{3.5/4} , 𝙻𝙻𝚊𝙼𝙰𝟸 \mathtt{LLaMA2} - 𝟽𝟶 ​ 𝙱 \mathtt{70B} and up), leaving the efficacy of debate on smaller, open-source models underexplored; our study addresses this gap by reimplementing Du et al. (2023) in experiments with smaller-scale ( 1 − 70 1-70 B) models. More crucially, debate and similar natural language communication methods are extremely computationally expensive , which this work addresses ( Yang et al., 2023 ; Wu et al., 2023 ) .

Notably, Pham et al. (2024) propose CIPHER, which uses input (tokenizer) embeddings (as opposed to activations) to enable multi-agent communication; specifically, CIPHER passes the average tokenizer embedding (weighted by the LLM’s next-token probabilities) between models. While ( Pham et al., 2024 ) show this approach outperforms natural language debate, it (i) still faces substantial information loss relative to the model activations and (ii) does not save compute, as the number of these “average embeddings” passed between models is the same as the number of tokens passed between models in natural language communication.

A related class of methods involves spending extra test-time compute reasoning in latent space ( Geiping et al., 2025 ; Hao et al., 2024 ) . Such latent reasoning approaches involving doing ”chain-of-thought in activation space,” e.g. by grafting LM activations into other layers/later forward passes through the same model (e.g., a form of “recurrent AC” within a single model); our approach can be viewed as doing exactly the same thing, but instead ”outsourcing” the CoT to another model (and thus reaping benefits from greater diversity of thoughts/reasoning paths from distinct models).

#### Activation engineering

Activation engineering involves editing an LLM’s intermediate layer representations during a forward pass to create desired changes to output text ( Li et al., 2024 ; Turner et al., 2023 ) . Past work has explored extracting latent steering vectors from a frozen LLM to control quality and content of completions ( Subramani et al., 2022 ) , as well as using “direction” vectors (computed as the difference in activations between two prompts) that enable inference-time control over high-level properties of generations ( Li et al., 2024 ; Turner et al., 2023 ) . This work involves activation editing that is similar to such prior works at a high level, though for the purpose of communication between LLM agents.

#### Model composition and grafting

Composing expert models has been a recurring strategy to improve large models, with different methods imposing different restrictions on the types of base LLMs that can be combined. Mixture of Experts ( Shazeer et al., 2017 ) requires that all experts are trained simultaneously using the same data; Branch-Train-Mix ( Sukhbaatar et al., 2024 ) trains a single base LM multiple times on different datasets, then learns a router on outputs. Crucially, these methods do not work when neither model can do the task at hand well (i.e., they solve the problem of choosing which of several outputs is best, not that of generating a high-quality output by recombining the disparate abilities of the various base LMs).

Model grafting, in contrast, seeks to merge different models immediately prior to or at inference-time. Past works have explored this at the parameter level (e.g., task vector averaging as in Ilharco et al. (2023) , which requires that the base models be well aligned), probability distribution / token level as in Shen et al. (2024) (which imposes few restrictions on the relationship between the base models, but by virtue of being token-based can result in cascading errors during decoding), and activation level (e.g., CALM ( Bansal et al., 2024 ) which learns an attention layer on top of two models’ intermediate layer activations and thus enables broader integration of model abilities than token-level methods, but requires re-tuning of the attention mechanism for every model pair). In this work, we seek to unify CALM and other activation-level grafting techniques under a single framework, parameterized by the function f f used to combine activations; crucially, we explore simple forms of f f (e.g., sum, mean) that—unlike Bansal et al. (2024) —require zero additional task-specific parameters and data , and are far more compute-efficient.

## 3 Communicating Activations Between Language Models

We propose a simple yet effective technique whereby language models communicate via activations . We detail our approach in Section 3.1 ; provide analytical models of the compute saved over natural language communication in Section 3.2 ; and discuss the intuition behind this approach in Section 3.3 .

### 3.1 Method

Consider two language models, A A and B B , and some setting in which B B must perform a task where it would benefit from knowledge given to A A as a prompt/encoded in A A ’s weights (example settings in Section 4.1 / Section 4.2 respectively). We propose incorporating information from A A ’s post-layer k k activation 𝒉 A , k \bm{h}_{A,k} into B B ’s post-layer j j activation 𝒉 B , j \bm{h}_{B,j} (and vice versa, though for simplicity we henceforth only discuss the first direction) ( Figure 1 , left).

More formally, suppose A A and B B (which have model dimensions d A d_{A} and d B d_{B} respectively) are given prompts x A x_{A} and x B x_{B} respectively, where x A x_{A} is of length t A t_{A} tokens and x B x_{B} is of length t B t_{B} tokens. We first run a partial forward pass of B B until layer j j (henceforth denoted B ≤ j ​ ( x B ) B_{\leq j}(x_{B}) ) to get 𝒉 B , j ∈ ℝ t B × d B \bm{h}_{B,j}\in\mathbb{R}^{t_{B}\times d_{B}} . Then we (1) run a partial forward pass of A A until layer k k to get A ≤ k ​ ( x 1 ) := 𝒉 A , k ∈ ℝ t A × d A A_{\leq k}(x_{1}):=\bm{h}_{A,k}\in\mathbb{R}^{t_{A}\times d_{A}} ; (2) replace the activation of the last token ( 𝒉 B , j ) t B ∈ ℝ d B (\bm{h}_{B,j})_{t_{B}}\in\mathbb{R}^{d_{B}} ⟵ f ⁡ ( ( 𝒉 A , k ) t A , ( 𝒉 B , j ) t B ) \longleftarrow f((\bm{h}_{A,k})_{t_{A}},(\bm{h}_{B,j})_{t_{B}}) for some function f : ℝ d A + d B → ℝ d B f:\mathbb{R}^{d_{A}+d_{B}}\to\mathbb{R}^{d_{B}} ; then (3) continue B B ’s forward pass till decoding is complete, resulting in an output y = B > k ​ ( 𝒉 B , j ) y=B_{>k}(\bm{h}_{B,j}) .