AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
Qingyun WuGagan BansalJieyu ZhangYiran WuBeibin LiErkang ZhuLi JiangXiaoyun ZhangShaokun ZhangJiale Liu
Introduces AutoGen, an open-source framework that enables developers to build advanced applications by orchestrating customizable, conversable agents that combine large language models, human input, and computational tools.
The article introduces AutoGen, an open-source framework designed to build applications that use large language models through multi-agent conversations. Developers face growing challenges in scaling LLM capabilities for complex, multi-domain tasks, and prior single-agent systems often lack flexibility for collaboration, human input, or dynamic workflows. The work addresses this by creating a system that lets agents with varied roles converse to solve problems, combining LLM reasoning with tools and optional human oversight.
The authors developed two core concepts: conversable agents that can be customized with LLMs, humans, or tools, and a conversation programming approach that defines interactions through natural language or code. They implemented built-in agents and tested the framework on six applications across mathematics, code generation, question answering, decision-making in simulated environments, optimization, and interactive games. Evaluations used standard benchmarks such as the MATH dataset, Natural Questions, ALFWorld, and OptiGuide tasks, comparing results against baselines including GPT-4, ReAct, and commercial tools.
Key findings show that AutoGen systems achieved higher success rates than alternatives, such as 69.5 percent accuracy on the full MATH test set versus 55.2 percent for GPT-4 alone, and a 15 percent gain on ALFWorld tasks when adding a grounding agent. Multi-agent designs improved safety checks in coding tasks and reduced development code by roughly 75 percent in one case while cutting user interactions by three to five times. Dynamic group chats and human-in-the-loop modes enabled new interaction patterns without added complexity.
These results indicate that multi-agent conversation can improve task performance, lower development effort, and support modular, reusable components across diverse applications. The approach allows organizations to integrate LLMs more effectively into workflows that require collaboration, validation, or external tool use, potentially accelerating deployment while maintaining oversight.
Further work is needed to identify optimal agent configurations for specific tasks, integrate existing agent libraries, and develop safeguards against unintended behaviors as autonomy increases. Developers should begin with simple two-agent setups and built-in components, then add complexity only as required, while monitoring for bias, privacy, and accountability issues.
- Paper: ReAct: Synergizing Reasoning and Acting in Language Models, Shunyu Yao et al. (2023). Introduces the foundation of interleaving reasoning traces and external action execution in language models, which AutoGen builds upon and directly compares against in its multi-agent evaluations.
- Paper: Improving Factuality and Reasoning in Language Models through Multiagent Debate, Yilun Du et al. (2023). Demonstrates that multi-agent dialogue and debate improve reasoning and factual correctness, establishing the conceptual motivation for AutoGen's conversational agent architecture.
- Paper: Generative Agents: Interactive Simulacra of Human Behavior, Joon Sung Park et al. (2023). Pioneers memory- and reflection-based interactive language model agents, establishing key behaviors for autonomous multi-agent environments before AutoGen formalized conversation programming.
- Paper: Sparks of Artificial General Intelligence: Early experiments with GPT-4, Sébastien Bubeck et al. (2023). Provides the seminal empirical exploration of GPT-4's broad capabilities in tool use, code generation, and complex reasoning that AutoGen harnesses as core agent backbones.
- Paper: A Roadmap of Agent Research and Development, NICHOLAS R. JENNINGS et al. (2004). Presents foundational multi-agent systems theory covering situatedness, autonomy, and multi-party coordination protocols that underpin AutoGen's agent design.
- Paper: Program Synthesis with Large Language Models, Jacob Austin et al. (2021). Explores interactive program synthesis and human feedback loops with large language models, providing crucial background for AutoGen’s code generation and execution agents.
- Paper: Towards a Science of Scaling Agent Systems, Yubin Kim et al. (2025). Rigorously examines the conditions, scaling laws, and trade-offs under which multi-agent architectures outperform single-agent systems across diverse real-world tasks.
- Paper: OpenHands: An Open Platform for AI Software Developers as Generalist Agents, Xingyao Wang et al. (2025). Extends multi-agent and tool-using LLM architectures into an open-source platform specifically tailored for autonomous software development and real-world execution.
- Paper: Code as Agent Harness, Xuying Ning et al. (2026). Advances beyond natural language conversation programming by analyzing executable code as the primary substrate for agent reasoning, action, and multi-agent coordination.
- Paper: Learning to Orchestrate Agents in Natural Language with the Conductor, Stefan Nielsen et al. (2026). Builds on multi-agent collaboration frameworks by training a dedicated language model conductor to automate subtask division and inter-agent communication.
- Paper: Mixture-of-Agents Enhances Large Language Model Capabilities, Junlin Wang et al. (2024). Explores layered multi-agent architectures that aggregate and refine collective model outputs to surpass individual frontier models without retraining.
- Paper: AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security, Dongrui Liu et al. (2026). Addresses the essential downstream safety and diagnostic guardrail challenges arising from autonomous, multi-step agent actions and tool interactions.
- Paper: Agentic Reasoning for Large Language Models, Tianxin Wei et al. (2026). Provides a comprehensive taxonomy of agentic reasoning spanning single-agent tool use, dynamic memory, and collective multi-agent collaboration schemes.
- Paper: Toward Efficient Agents: Memory, Tool learning, and Planning, Xiaofang Yang et al. (2026). Investigates efficiency bottlenecks in agent memory, tool learning, and multi-agent planning to mitigate the computational and token costs incurred by conversational workflows.
