Competition and Attraction Improve Model Fusion — part 2/3

Cleaned-text code points [15841, 31219). Bibliography omitted here; source.md includes it.

(3) θ ∗ = arg ⁡ max ⁡ ∑ j = 1 N θ ⁡ s ⁡ ( x j ∣ θ ) z j + ϵ ​ c j , where , z j = ∑ k = 1 P s ⁡ ( x j ∣ θ k ) \theta^{*}=\arg\max_{\theta}\sum_{j=1}^{N}{\frac{s(x_{j}\mid\theta)}{z_{j}+\epsilon}c_{j}},\text{where},z_{j}=\sum_{k=1}^{P}s(x_{j}\mid\theta_{k})

where ϵ \epsilon in the denominator is a small number to prevent the zero-division error. In the term that defines z j z_{j} , P P is the archive size. The capacity c j c_{j} , is task dependent and can be defined in multiple ways. For example, in binary scoring tasks (i.e. s ⁡ ( ⋅ ) ∈ { 0 , 1 } s(\cdot)\in\{0,1\} ) we simply set c j = 1 c_{j}=1 . In some experiments, we have a continuous reward from 0 to 1. Here, we define c j = max i ⁡ s ⁡ ( x j | θ i ) c_{j}=\max_{i}s(x_{j}|\theta_{i}) to ensure that partially solved data points (where max i ⁡ s ⁡ ( x j | θ i ) < 1 \max_{i}s(x_{j}|\theta_{i})<1 ) do not distribute the same amount of fitness points as fully solved data points (where max i ⁡ s ⁡ ( x j | θ i ) = 1 \max_{i}s(x_{j}|\theta_{i})=1 ).

### 3.3. Sampling Parents via Attraction

Many evolutionary algorithms use the crossover operation to combine the strengths of both parents. In biology, this combination (i.e., reproduction) is very expensive, and therefore, animals invest many resources in the process of mate selection. We believe that as we make use of more expensive crossover operations, like model merging, algorithms for mate selection become increasingly important.

In contrast to conventional methods that put more sampling probability mass on top performing models in the archive, M2N2 adds an extra layer of consideration that takes into account the complementarity of the parent models. Specifically, we sample the first parent based on their weighted sum of scores defined in Equation 3 , and then sample the second parent based on a “attraction score” generated by function g g that is specifically tailored for the first parent. The equation below gives the definition of this attraction score, it straightforwardly expresses a desire to choose a model B that performs well in the data points where model A performs less well, while giving an extra preference to resources with high capacity c j c_{j} and low competition z j z_{j} . (4) g ⁡ ( θ A , θ B ) = ∑ j = 1 N c j z j + ϵ ​ max ⁡ ( s ⁡ ( x j ∣ θ B ) − s ⁡ ( x j ∣ θ A ) , 0 ) g(\theta_{A},\theta_{B})=\sum_{j=1}^{N}{\frac{c_{j}}{z_{j}+\epsilon}\max\big(s(x_{j}\mid\theta_{B})-s(x_{j}\mid\theta_{A}),0\big)}

## 4. Experiments

We verify the effectiveness of our proposed method on three challenging tasks: First, we evolve image classifiers from scratch and from pre-trained models on the MNIST dataset, then scale up the experiment to merging LLMs and diffusion-based image generation models to demonstrate its general applicability.

### 4.1. Experiment 1: Evolving MNIST classifiers

Setup

Model : The model being optimized is a two-layer feedforward neural network with 19,210 parameters in total. When starting from scratch, we randomly initialize the models. For pre-trained models, we develop two specialized models: one is trained on digits 0 through 4, and the other is trained on digits 5 through 9.

Baselines : For the MAP-Elites algorithm, we use two diversity dimensions to create a 10 by 10 grid: the accuracy of the model in odd and even numbers. When starting from scratch, we use CMA-ES ( Hansen and Ostermeier, 2001 ) as a baseline, even though it does not perform model merging here. Since the models are randomly initialized, optimizing mixing coefficients alone would be insufficient. Instead, CMA-ES directly optimizes model weights, which incurs a cubic computational cost O ⁡ ( n 3 ) O(n^{3}) with respect to the number of parameters. While this method doesn’t scale to larger models, it serves as a benchmark for how a popular evolutionary algorithm performs in this experiment. When working with pre-trained models, we use a brute-force search baseline that merges the two seed models by adjusting a mixing coefficient that ranges from 0 to 1 in increments of 10 − 5 10^{-5} . This baseline is first evaluated on the training data; the best coefficient is subsequently evaluated on the test data.

Evolutionary Operators and Variables : All model merging methods (which excludes CMA-ES) sample a new candidate at a time and decide sequentially whether to insert the candidate into the archive. M2N2 and GA use an archive of 20, sampling each candidate sequentially and deciding whether to insert it, similar to MAP-Elites. MAP-Elites, uses a 10x10 grid, resulting in an archive size of 100. CMA-ES uses a population of 20, sampling and updating its parameters in batches. When starting from scratch all model merging methods use the same mutation operation (Gaussian noise) and the same crossover operation (SLERP with split-point, as described in section 3.1 ). However, when dealing with pre-trained models, mutation is omitted because we want to assess how our method would scale to larger models where random mutations are not effective.

Compute Resources : The 10 independent runs took about 15 hours for CMA-ES, and about 1h for each of the other methods. We ran this experiment using only CPUs.

Results

When starting from scratch, M2N2 achieves the highest test accuracy by a substantial margin when compared to the other model merging methods, as shown in Figure 2 (left). Interestingly, GA with crossover performs better early on (before step 12,000) than GA without crossover, however, it converges faster to an inferior solution. The early convergence happens because GA can’t maintain a diverse population which is crucial for effective crossover operations. Crossover reduces population diversity, and without a strong counteracting force, it diminishes exploration. In contrast, M2N2 leverages the crossover operation effectively, benefiting significantly from the diversity it manages to retain. GA is an extreme case where there is no competition, we observed that by progressively decreasing competition in M2N2, we progressively converge earlier to worse solutions (analyzed on the next section). MAP-Elites clusters individuals by their accuracy on odd and even numbers. This means it will always keep individuals who perform poorly on those tasks because there is a slot reserved just for them. Even though those individuals add to the diversity of the population, this is clearly not the type of diversity that leads to strong solutions and it highlights the difficulty of hand-engineering useful diversity metrics.

For models trained from scratch, the split-point and attraction score have a minimal impact (ablations omitted for clarity). However, as seen in Figure 2 (right), the split-point becomes crucial when starting from pre-trained models, while attraction significantly improves performance throughout the training process. GA has a low average test accuracy with large error bars as its performances is highly dependent on the quality of the first merges. Note that when starting from pre-trained models the mutation operator was not used (as explained in the Setup section), and therefore, the performance is worse.

Analysis

This section focuses exclusively on the experiment where models were evolved from scratch, as the later sections will provide ample discussion on evolving models starting from pre-trained LLMs and diffusion models.

Diversity : Figure 3 left, shows the percentage of training data points that can be correctly labeled by at least one model in the archive, we call this percentage the training coverage. We observe that the archive in M2N2 quickly spreads to cover the majority of the training data points and maintains this high coverage throughout the training process. The right-hand plot shows how the diversity in the performance of the population evolves with training. If either all models correctly or incorrectly classify a data point, the entropy is 0 (no diversity). In contrast, when the models are evenly split on a prediction, entropy reaches its maximum value of 1. The plot displays the average entropy across all data points. For M2N2 we see a sharp initial rise in entropy followed by a gradual decline as low-performing models go extinct. In contrast, MAP-Elites continually increases diversity by retaining lower-performing models, but it fails to achieve a high coverage. The Genetic Algorithms, lacking a diversity preservation mechanism, reduce coverage early on and show a sharp drop in entropy as they converge prematurely on the best solutions. This drop in diversity is accentuated with the crossover operation.

Overall, the graphs show that M2N2 maintains an archive of models with complementary strengths that facilitate effective merging, while systematically discarding weaker models as training progresses.

Competition : Figure 4 left, shows that smaller archives perform better in the beginning but converge faster to inferior solutions. This suggests that we should scale the archive size along the number of forward passes we want to make. Note that in our plot the computational cost does not increase with the archive size since the number of forward passes remains the same, however, the memory footprint does increase with larger populations. For very large models we can always store the archive on disk instead of keeping them all in the RAM.

For a fixed population size P P , we can adjust the intensity of competition by introducing a hyper-parameter α ≥ 0 \alpha\geq 0 , as described in the fitness function in eq. 5 .

(5) f ⁡ ( θ i ) = ∑ j = 1 N s ⁡ ( x j | θ i ) z j α + ϵ ​ c j f(\theta_{i})=\sum_{j=1}^{N}\frac{s(x_{j}|\theta_{i})}{{z_{j}}^{\alpha}+\epsilon}c_{j}

When α = 0 \alpha=0 , there is no competition because the total fitness available per data point becomes unlimited. When α = 1 \alpha=1 , the total fitness distributed among different individuals is limited to the capacity c j c_{j} . For α > 1 \alpha>1 , the total fitness distributed decreases with increasing competition ( z j z_{j} ), this scenario can be thought of as individuals needing to "fight" for resources, spending some fitness points in the process. Figure 4 right, shows that smaller values of α \alpha (i.e. lower competition) have a similar effect to decreasing the population size: it performs better in the beginning but it converges faster to inferior solutions.

### 4.2. Experiment 2: Combining LLMs with Math and Agentic Skills

Setup

Models : We combine a math specialist, WizardMath-7B-V1.0 ( Luo et al., 2023 ) , with a specialist on agentic enviroments, AgentEvol-7B ( Xi et al., 2024 ) , to achieve an agent that performs well on the math benchmark GSM8k ( Cobbe et al., 2021 ) and on the web shopping benchmark WebShop ( Yao et al., 2022 ) . These two models share the same architecture of Llamma-2-7b ( Touvron et al., 2023 ) , a decoder-only transformer with 32 layers.

Datasets : For the math task, we use the test split of GSM8k as our test split (1319 samples). For the training split, we use the first 1319 samples of GSM8k train dataset. In the web shopping task, we use the WebShop environment implemented in ( Xi et al., 2024 ) . The test split consistent of the first 100 tasks, while the training split were the next 100 tasks. We allow the agents to take up to 7 steps.

Baselines : The CMA-ES optimizes 32 mixing coefficients (one for each layer) for a SLERP merge between the two seed models. All methods ran for a 1000 evaluations on the training set. For the MAP-Elites we used two dimensions to create a 4 by 4 grid: the accuracy on the math and on the web shopping training splits.

Evaluation : In this experiment, all methods used 1000 evaluations on the training set. M2N2 and GA used an archive size of 15. CMA-ES used a population size of 25.

Evolutionary Operators and Variables : In our LLM Merging experiment we do not use a mutation operator since random mutations don’t work well on large models. Moreover, in these experiments we initialize the M2N2 and GA archives with seed models, followed by a short warm-up period (50 iterations or less) where the seed models merge randomly amongst themselves and populate the archive.

Compute Resources : For these experiments, we used 4 H100 GPUs to run each method for around 24h.

Results

Table 1 shows that M2N2 achieves the highest score. Both the attraction and the split-point techniques play a crucial role, however, the split-point seems to be slightly more important. Note that on Table 1 all algorithms run the same amount of evaluations and used the same merging method (SLERP). When combining the Math and Agentic skills, CMA-ES yielded a low score, likely due to suboptimal parameter partitioning, highlighting the need to include the merging boundaries in the optimization process.

Analysis

As shown in Figure 5 , the findings from the MNIST dataset generalize to LLM merging. The Natural Niches method maintains high training coverage, as seen on the left side of the figure. The entropy rises early on as the models explore diverse niches (right), followed by a gradual decrease as low-performing models are removed, and the strengths of the models are aggregated. In contrast, MAP-Elites focuses on maximizing entropy at the cost of training efficiency and coverage, as it retains low-performing models. GA quickly reduces both coverage and entropy as it greedily converges on its top solution, ultimately collapsing the entire archive onto a single solution, with entropy nearing zero.

### 4.3. Experiment 3: Merging Diffusion-Based Image Generation Models

Setup

Models : We evaluate our method in merging diverse text-to-image models. Our seed models include JSDXL ( Shing et al., [n. d.] ) , which was specifically trained on Japanese prompts, and three models primarily trained with English prompts: SDXL 1.0 ( Podell et al., 2023 ) , SDXL-DPO ( Wallace et al., 2024 ) , and Juggernaut-XL-v9 ( RunDiffusion, 2024 ) . All these models share the same architecture of the base model SDXL 1.0 . The primary objective is to create a model that combines the best image generation capabilities from each seed model while retaining JSDXL’s ability to understand Japanese prompts.

Evolutionary Operators and Variables : As in our LLM merging experiment, we omit the mutation operator. For model merging, we retain the VAE from SDXL – since most seed models left this component unchanged – and we preserve JSDXL ’s tokenizer and text encoder to leverage its superior Japanese language understanding. Therefore, we are combining only the U-Nets from the various models using Equation 2 . We merge the attention layers independently from the other components, as this was found effective on previous work ( Sakana AI, 2025 ) . This compartmentalization of parameters is analogous to the grouping of DNA into chromosomes, where each chromosome divides independently and has one or more split points. This design introduces useful inductive biases while maintaining the flexibility to explore more complex merging boundaries.

Dataset : We utilize the COCO dataset ( Lin et al., 2014 ) , specifically the validation split. We use the last 2,000 images for training and the first 10,000 images for testing, ensuring no overlap between the sets. The Japanese captions are sourced from the STAIR Captions dataset ( Yoshikawa et al., 2017 ) , which provides Japanese descriptions for COCO images.