Model Swarms enables collaboration between multiple LLMs

November 13, 2024

Model Swarms is a collaborative search algorithm inspired by the collective behavior of social animals like bees and ants. It enables multiple LLM experts to work together in a shared pool, where each expert contributes its specialized knowledge to solve complex problems that require diverse perspectives.

Developed by a research team from Google and the University of Washington, this algorithm is designed to be highly versatile and effective across various fields. It can be applied to tasks requiring multiple skills or areas of expertise, such as science, medicine, and law.

The swarm can handle different parts of a task, ensuring a more comprehensive response. For example, one model might focus on generating detailed technical explanations, while another could specialize in translating complex concepts into simpler terms.

Model Swarms compared to other model composition methods

Model composition methods bring a new perspective on AI development. Rather than simply creating one powerful model for all purposes, they combine multiple models, creating more adaptable and efficient systems. Traditional approaches on model composition include:

Mixture-of-experts (MoE) where each expert model focuses on specific parts of a task or input, and a router directs data to the most relevant expert. The final output is a combination of the expert outputs.
Learn-to-fuse trains a model to combine the outputs of different experts.
Static model arithmetic performs mathematical operations on the weights of different models to create a new, combined model.
Dynamic model arithmetic adjusts the weights of different models based on the specific task or input data.

Unlike these traditional models that typically rely on static configurations, Model Swarms creates a more interactive communication among the LLM experts. During the problem-solving process, the models share information and adjust their tasks based on both their own performance and the collective success of the group.

The architecture

The system has a decentralized architecture inspired by swarm intelligence and it aims to find the best way to perform various tasks. These tasks are divided among LLM experts, each specializing in areas such as natural language generation, data summarization, and sentiment analysis.

The picture below demonstrates how the LLM experts search for optimal configurations, based on their own expertise and the overall performance of the entire swarm.

Each LLM in the system starts with a specific location (x_i) and a velocity (v_i) in the weight space. The velocity (v_i) is influenced by:

Personal best location (p_i): The best location the LLM has found so far.
Inertia: The tendency to maintain its current velocity.
Global best location (g): The best location found by any LLM in the swarm.
Global worst location (g_w): The least optimal location found by any LLM in the swarm.

Guided by these factors, each LLM updates its velocity and location to explore the weight space and optimize a utility function (f). The best-performing expert (global best g) is retained as the final output. The utility function (f) can vary depending on the specific objectives of the task, such as dataset performance, reward model scores, human preferences or other performance metrics.

This adaptive search approach allows Model Swarms to optimize task performance without requiring supervised fine-tuning data or prior knowledge about the LLM experts.

The following image illustrates the Model Swarms algorithm. After the initialization of variables, it proceeds through (K) iterations as detailed below:

Initialize search. The pool of experts is expanded from the initial set of (n) models to (N) models. This expansion increases the chances of finding an optimal solution by providing more starting points. Each LLM expert (x_i) is assigned an initial velocity (v_i) and a personal best (p_i). Additionally, global best (g) and global worst (g_w) values are determined to guide the optimization process.
Search. During the search, which runs for a maximum of (K) iterations, each LLM updates its velocity by calculating a weighted average of four components: (v_i), (p_i), (g) and (g_w). This velocity controls each LLM’s movement, directing it toward more optimal regions in the search space. Subsequently, each LLM’s position (x_i) is updated based on its new velocity.
End of iteration. The algorithm continues iterating until the global best solution (g) hasn’t changed for a specified number of iterations or the maximum number of iterations (K) is reached. The best-performing LLM (g) is retained as the final output.

The variables (v_i), (p_i), (g) and (g_w) work together to guide the swarm towards optimal solutions. If a particular LLM is underperforming in a specific area, the swarm reallocates the task to other LLMs with better expertise. This way, each model is used to its maximum potential.

Evaluation

The evaluation aimed to determine whether Model Swarms could effectively adapt LLM experts through collaborative search. It was tested on 9 datasets and compared against 12 model composition baselines, focusing on 4 types of adaptation objectives:

Single task: Adapting the model to perform a specific, singular task efficiently.
Multi-task domain: Adapting the model to handle multiple tasks within a particular domain, ensuring versatility and robustness.
Reward model: Optimizing the model based on a reward function, which could be used in reinforcement learning scenarios.
Human interest: Adapting the model to align with human preferences or interests, potentially improving user satisfaction and engagement.

The results demonstrated that Model Swarms effectively adapted LLM experts to various task objectives, even with very little data.

The tables below illustrate the model’s performance on both single tasks and multi-task domain adaptation. It significantly outperforms three baseline methods: Trivial, Static, and Dynamic, achieving a 13.3% higher performance. Furthermore, the model shows a 21.0% performance improvement on 3 reasoning tasks.

The experiments have also shown that Model Swarms performs better when it works with a diverse set of LLM experts. The best model in the end was often not the most powerful from the start. Rather, its hidden potential was revealed through the collaborative interactions within the swarm environment.

Conclusion

Model Swarms is an innovative approach based on swarm intelligence, where multiple specialized AI models (LLMs) are working together to perform complex tasks.

This approach can be especially advantageous for small startups and individuals with innovative research ideas but limited resources. Rather than training large models, they can use a collection of open-source smaller models, each bringing its specialized expertise.