Meet Magentic-One: Microsoft’s New Multi-Agent Framework for Solving Complex Tasks

Author(s): Jesus Rodriguez

Originally published on Towards AI.

I recently started an AI-focused educational newsletter, that already has over 170,000 subscribers. TheSequence is a no-BS (meaning no hype, no news, etc) ML-oriented newsletter that takes 5 minutes to read. The goal is to keep you up to date with machine learning projects, research papers, and concepts. Please give it a try by subscribing below:

Another week another agentic framework. The market for AI agents seems as hot as it is fragmented but the level of innovation is remarkable. One of the areas that seems to be gaining a tremendous level of attention is the multi-agent systems. Microsoft is one of the companies really active in the agents space with frameworks such as AutoGen and TaskWeaver. Recently, Microsoft open sourced a new framework focus on multi-agent systems.

Magentic-One is a new generalist multi-agent system developed by Microsoft Research, designed to handle open-ended tasks based on web and file information across various domains. This essay will examine the architecture of Magentic-One, its capabilities, evaluation results, and potential risks.

Architecture

Magentic-One is built upon a multi-agent architecture where an **Orchestrator** agent leads four other agents in task execution. This approach allows for modularity and flexibility, exceeding the capabilities of traditional single-agent systems.

The Orchestrator agent is responsible for:

· Decomposing tasks and planning their execution.

· Directing other agents to carry out subtasks.

· Monitoring the overall progress of the task.

· Taking corrective actions when necessary.

Magentic-One utilizes a two-loop system for task management:

· Outer loop: Manages the Task Ledger which stores facts, guesses, and the overall plan for the task.

· Inner loop: Handles the Progress Ledger, tracking the current state of the task and subtask assignments.

The Orchestrator initiates the process by developing a plan and recording relevant information in the Task Ledger. Each step of the plan is then tracked in the Progress Ledger. The Orchestrator assigns subtasks to other agents, monitors their progress, and updates the Progress Ledger accordingly. If insufficient progress is made, the Orchestrator can adjust the Task Ledger and formulate a new plan.

Specialized Agents

In addition to the Orchestrator, Magentic-One includes four other specialized agents:

· WebSurfer: An agent trained to interact with web browsers, capable of navigation, web page actions, and information extraction. The WebSurfer utilizes the browser’s accessibility tree and prompting techniques to complete its actions.

· FileSurfer: An agent skilled in reading and navigating local files of various types using a markdown-based file preview application. This agent can list directory contents and move within a folder structure.

· Coder: An agent specifically designed for writing code, analyzing data from other agents, and creating new outputs.

· ComputerTerminal: This agent provides access to a console shell for executing code generated by the Coder and installing necessary programming libraries.

Models Flexibility

Magentic-One is designed to be model agnostic. While the default model for all agents is GPT-4o, different configurations can leverage various large language models (LLMs) and specialized language models (SLMs). This adaptability ensures that Magentic-One can be optimized for specific tasks and resource constraints.

The following code demonstrates how Magentic-One agents interact:

```python
# Initialize the Magentic-One agents
from autogen.magentic_one import MagenticOneAgent
orchestrator = MagenticOneAgent()
# Define the task
task = "Download a file from the internet, execute code to interact with the file"
# Run the task
orchestrator.run(task)
```

Evaluation

Magentic-One’s performance was assessed using AutoGenBench, a tool developed for evaluating agentic systems. The benchmarks chosen for evaluation were GAIA, AssistantBench, and WebArena, each involving multi-step tasks requiring planning and the use of web browsers.

The results indicated that Magentic-One exhibits competitive performance compared to other state-of-the-art methods. It achieves statistically similar results to existing methods on GAIA and AssistantBench, while demonstrating competitive performance on WebArena.

Some Risks

The increased capabilities of agentic systems like Magentic-One bring potential risks. Because Magentic-One operates in the real world, its actions can have unintended consequences. Examples of observed risks include:

· Repeated login attempts leading to account suspension.

· Attempts to involve humans in tasks through social media or emails.

To address these risks, Microsoft has implemented various mitigation strategies:

· Red-teaming exercises to identify potential vulnerabilities.

· Safety guidelines and best practices for using Magentic-One.

· Monitoring and oversight mechanisms.

· Emphasis on using aligned models with pre- and post-generation filtering.

What Comes Next ?

Future research in agentic systems and safety will likely focus on:

· Anticipating emerging risks, including phishing and misinformation attacks.

· Developing mechanisms for agents to understand the reversibility of their actions.

· Integrating human-in-the-loop systems for critical decision points.

Magentic-One marks a significant advancement in the field of agentic systems, capable of solving complex tasks in web and file-based environments. Its multi-agent architecture, modularity, and model-agnostic design offer flexibility and adaptability. While potential risks exist, ongoing research and mitigation efforts are crucial to ensure the responsible and safe development of such powerful AI systems.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI