CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society https://www.camel-ai.org

Guohao Li$^{*}$

Hasan Abed Al Kader Hammoud$^{*}$

Hani Itani$^{*}$

Dmitrii Khizbullin

Bernard Ghanem
King Abdullah University of Science and Technology (KAUST)

$^{*}$ Equal contribution

Abstract

The rapid advancement of chat-based language models has led to remarkable progress in complex task-solving. However, their success heavily relies on human input to guide the conversation, which can be challenging and time-consuming. This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents, and provides insight into their "cognitive" processes. To address the challenges of achieving autonomous cooperation, we propose a novel communicative agent framework named role-playing. Our approach involves using inception prompting to guide chat agents toward task completion while maintaining consistency with human intentions. We showcase how role-playing can be used to generate conversational data for studying the behaviors and capabilities of a society of agents, providing a valuable resource for investigating conversational language models. In particular, we conduct comprehensive studies on instruction-following cooperation in multi-agent settings. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems, and open-sourcing our library to support research on communicative agents and beyond: https://github.com/camel-ai/camel.

Executive Summary: Modern chat-based language models are highly effective at solving complex problems, but their success typically depends on extensive, ongoing human prompt engineering. Crafting precise, step-by-step instructions requires specialized domain knowledge and significant human time, creating a major operational bottleneck. The article investigates whether autonomous communicative artificial intelligence agents can collaborate effectively to solve complex tasks with minimal human intervention, aiming to demonstrate a scalable framework for autonomous multi-agent cooperation and dataset generation.

To achieve this, the article introduces CAMEL, a role-playing framework that uses inception prompting. In this setup, a human provides an initial high-level idea and role assignments, an automated task specifier expands the idea into a concrete assignment, and two communicative agents—acting as an instruction-giving user and an executing assistant—collaborate autonomously through multi-turn dialogue until task completion. To evaluate performance and study multi-agent behavior, the authors generated 25,000 role-playing conversations and collected multi-domain datasets across societal scenarios, coding, mathematics, and science, running comparative evaluations against single-shot baselines and benchmarking fine-tuned open models.

The findings show that cooperative multi-agent dialogue significantly outperforms traditional single-shot interactions. In task evaluations across general society scenarios, solutions generated by the role-playing agents won over single-shot baselines in 76.3% of human evaluations and 73.0% of automated evaluations, with a 76.0% win rate in coding benchmarks. Additionally, progressively fine-tuning an open-source 7-billion parameter language model on these multi-agent interaction datasets led to clear domain capability emergence, achieving a 57.9% pass rate at 100 attempts on the HumanEval coding benchmark compared to 36.5% for the base model and 42.9% for existing tuned alternatives.

These results indicate that structured multi-agent cooperation can drastically reduce the need for manual prompt engineering while generating high-quality synthetic data for downstream model training. By automating iterative planning and execution, organizations can improve task quality, compress development timelines, and enhance smaller open-source models at lower computational cost. However, the analysis also revealed recurring operational failure modes, including role flipping, repeating instructions, non-committal replies, and infinite message loops, emphasizing that strict prompt rules and explicit termination conditions are critical to prevent runaway compute costs.

Organizations exploring autonomous agent workflows should adopt structured inception prompts, clear role boundaries, and strict message caps to ensure alignment and cost control. While the article demonstrates high confidence in cooperative agent performance for structured tasks, stakeholders should note limitations around token context boundaries and safety risks in unaligned systems. Further research and controlled pilot testing are recommended to study safety governance and human-in-the-loop critic integrations before deploying fully autonomous multi-agent systems in high-risk operational environments.

1. Introduction

Section Summary: The introduction highlights how current chat-based AI models, while powerful, still depend heavily on skilled human prompting to tackle complex, multi-step tasks—an approach that is often time-consuming or impossible for non-experts. It argues that this limitation could be overcome by autonomous systems in which multiple AI agents communicate and collaborate with minimal oversight, and it proposes a “role-playing” framework that uses carefully designed initial prompts to guide such cooperation. The authors also describe an open-source library and large conversation datasets generated through this method, which they show can improve task performance and help study how abilities emerge in language models.

"What magical trick makes us intelligent? The trick is that there is no trick. The power of intelligence stems from our vast diversity, not from any single, perfect principle."

- Marvin Minsky, The Society of Mind, p. 308

Confronted with the complexities of real-world tasks, solving them often requires multiple steps. The rapid progress of chat-based large-scale language models (LLMs) has yielded remarkable achievements in complex task-solving [1, 2, 3, 4, 5, 6, 7, 8]. Nevertheless, it is worth noting that their success is heavily reliant on human input to guide the conversation in the right direction. This reliance necessitates users to provide relevant and precise prompts based on their intentions and the chat agent's feedback. This can be challenging, time-consuming, and sometimes impossible. Crafting effective prompts often demands a deep understanding and expertise of a particular domain of knowledge. Consider an individual who lacks trading expertise; they would find it difficult to create suitable prompts for directing a chat agent to develop a trading application. This predicament is raising a crucial question: can we replace human intervention with an autonomous communicative agent capable of steering the conversation toward task completion with minimal human supervision? To tackle this issue, it is crucial to conduct more research exploring the potential, capabilities, and limitations of communicative agents that operate entirely on their own to complete tasks. Understanding how multiple agents interact with each other is important for anticipating the future of artificial intelligence. The dynamics of collaborating or competing agents play a key role in determining the success of AI systems [9, 10, 11, 2, 12, 13, 6].

This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents and provide insight into their "cognitive" processes. Several challenges arise when asking a society of agents to autonomously cooperate on completing tasks. Examples we encountered in our preliminary analysis include role flipping, assistant repeating instructions, flake replies, and infinite loop of messages. Therefore, it is critical to investigate ways to align these models with human intentions and to explore means enabling their effective cooperation. To address these issues, we propose a novel cooperative agent framework named role-playing to automate cooperation between communicative agents. Specifically, our proposed approach involves using role-playing with inception prompting to autonomously guide the communicative agents toward task completion. Only a preliminary idea is needed from human to guide the conversations toward complex task-solving.

Our library, which we make publicly available, provides modular functionality, and includes implementations of different agents, examples of well-crafted prompts, and data explorers. We hope our library serves as a ground for future research in various areas such as multi-agent systems, cooperative AI, game theory simulations, social analysis, AI ethics, AI alignment, and beyond.

In addition, our role-playing method provides a highly scalable way to generate conversational data for studying the behaviors and capabilities of chat agents. We showcase how role-playing can be used to let chat agents communicate with each other for task completion and record their conversations for behavior analysis and capability understanding. In particular, we consider two cooperative scenarios of role-playing and generate two large conversational, task-oriented, and instruction-following datasets: AI Society and Code. We also use our framework to collect two single-turn question-answer datasets, Math and Science, for LLM ability emergence study. Furthermore, we generate a Misalignment dataset that is a simulation of possible malicious applications which demonstrate the potential risks of an unaligned autonomous agent system. The datasets offer a valuable resource for investigating conversational language models, enabling them to comprehend and react to human language more effectively. Furthermore, our role-playing offers a scalable method of creating conversational instruction-following data, which can potentially enhance the development of more advanced language models. We show that solutions derived from our role-playing framework outperform those generated in a single shot by gpt-3.5-turbo [1] in both GPT4 and human evaluations. We also study knowledge emergence in LLMs by fine-tuning LLaMA [14] on progressively growing datasets generated through our framework. Additionally, we evaluate our code generation capabilities through benchmarking our final model on HumanEval [15] and HumanEval$^{+}$ [16].

Contributions. Our contributions are fourfold: (1) We introduce a novel cooperative agent framework, role-playing, that allows communicative agents to collaborate autonomously toward completing tasks while requiring minimal human intervention; (2) Our framework offers a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems. It illuminates the challenges of achieving autonomous cooperation, and provides strategies for addressing them. We showcase the potential power of multi-agent collaboration for complex-task solving; (3) We demonstrate the significant emergence of LLM training abilities by utilizing the datasets we have collected from simulating four distinct agent collaboration scenarios; (4) We have open-sourced our library, containing implementations of various agents, data generation pipelines, data analysis tools, and collected datasets, to support research on communicative agents and beyond.

2. Related Work

Section Summary: Researchers have long explored how AI agents can communicate, especially through natural language, to collaborate on complex tasks in cooperative settings where each agent takes on a specialized role. Separate lines of work focus on improving large language models so they better follow user instructions, using techniques like reinforcement learning from human feedback, chain-of-thought prompting, and automated methods for generating instruction datasets. A related area examines AI alignment, which seeks to ensure models behave according to intended human goals and values, with this paper using role-playing scenarios to test how well current models stay aligned.

Communicative Agents. Communication between agents has been studied for a long time [17, 18]. There are many ways to facilitate communication between agents, and with agents [19, 20, 21]. Among these, natural language is considered the most natural form of communication [21]. By enabling agents to function as communicators themselves, they become capable of solving complex tasks [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42]. Communication between AI agents can occur in a competitive setting [43, 44] or a cooperative setting [45, 11, 46, 47, 48]. Cooperative AI refers to artificial intelligence systems that are designed to work together with humans and other AI systems to achieve common goals [49, 50]. Cooperative AI systems take into account the needs and capabilities of other agents in the system and actively seek to collaborate and coordinate their actions with them, which has many potential benefits, including increased efficiency, improved decision-making, and the ability to tackle complex problems that are beyond the reach of any single agent. However, designing effective cooperative AI systems is still an active area of research, as it requires addressing a range of technical, ethical, and social challenges [11]. Our work enables communicative agents to engage in a conversation and cooperate with each other to solve assigned tasks. The agents, each assigned a distinct role, are expected to apply their expertise and knowledge to solve their common task.

Instructional LLMs and Prompt Engineering. LLMs are trained on diverse text data and excel in text completion, with various downstream NLP applications [51, 52, 53, 54, 14]. However, InstructGPT suggests that LLMs may not align with user intent, proposing reinforcement learning from human feedback (RLHF) [55] and Instruction Fine-Tuning (IFT) [56] to improve LLMs' relevance and appropriateness to user instructions. Special types of instruction or prompting methods, such as Chain-of-Thought (CoT) [57], zero-shot-CoT [58], and ReAct [59], have recently been developed to enhance the performance of LLMs on reasoning, arithmetic and decision making tasks [60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75]. These techniques underpin the impressive capabilities of recent dialogue LLMs [33, 3, 76, 13, 1, 8], which aim to simulate human-like conversations and provide personalized and interactive experiences for users, exhibiting the behavior of conversational AI agents [77]. However, generating instruction datasets is a crucial challenge in building instruct-based LLMs, with existing datasets ranging from crowdsourced to generated. Hand-crafted instruction instances are available in [78], while leveraging previously crowdsourced NLP datasets is a less labor-intensive curation approach [56, 79, 80, 81]. LLMs have been explored for data generation in [82, 83, 84, 85], and Self-Instruct [86] proposes a semi-automated process for instruction instance generation. Unnatural-Instruction [87] collects instruction instances by prompting a language model with only three seed examples and paraphrasing the generated instances to expand the dataset. There is also a large chunk of work that has proposed methods for automatic dataset creation [88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100]. Another important challenge is prompt engineering. The quality of the prompt used to guide LLMs significantly affects its performance [101, 51, 102]. While LMs pre-trained on large data can implicitly learn tasks with few-shot prompting, hand-crafted prompts may not always suffice. Automated prompt generation methods have been proposed, such as gradient-guided search [103], mining-based and paraphrasing-based techniques [104], a meta-prompt [105], and automatic instruction selection and generation [106]. In this work, we introduce a conversational LLM auto-prompting method called Inception Prompting, which enables agents to prompt each other to solve tasks through Role-Playing. The AI user continuously provides instructions to the AI assistant for task-solving. This enables us to save the streaming instruction-solution pairs and create diverse, instructional, conversational, and task-oriented datasets. These datasets can be used to analyze the behavior and capabilities of LLMs and for future research for fine-tuning LLMs with conversational instructions.

AI Alignment. AI alignment is a field that aims to ensure that AI systems adhere to their intended goals, interests, and values, as envisioned by their designers [107, 108, 109, 110, 111, 112, 6]. The first attempt at AI alignment was made through the "Three Laws of Robotics, " which was introduced by Isaac Asimov in his science fiction stories [9]. Developing aligned AI systems is crucial for achieving desired objectives while avoiding unintended consequences. Research in AI alignment focuses on discouraging AI models from producing false, offensive, deceptive, or manipulative information that could result in various harms [113, 114, 115, 116]. Achieving a high level of alignment requires researchers to grapple with complex ethical, philosophical, and technical issues. We conduct extensive experiments to study different role-playing situations, which probe the alignment of LLMs.

3. Methodology

Section Summary: The methodology presents a role-playing framework in which two AI agents—one acting as an assistant and the other as a user—collaborate to turn a vague initial idea into a finished task. A separate task-specifier agent first converts the human-provided idea and chosen roles into a concrete goal. The user agent then issues step-by-step instructions while the assistant agent responds with solutions, continuing this exchange until the task is complete.

In this paper, we focus on studying communicative agents under cooperative settings where they share common interests. In particular, we study the assistant-user scenario, where a preliminary idea is given at the start. Agents will conceptualize the idea into a specific task and complete it autonomously through conversations.

3.1 Role-playing Framework

"What's the most resilient parasite? An Idea. A single idea from the human mind can build cities. An idea can transform the world and rewrite all the rules. Which is why I have to steal it."

- Dom Cobb, Inception

Our proposed framework is a novel role-playing approach for studying multiple communicative agents. Specifically, we concentrate on task-oriented role-playing that involves one AI assistant and one AI user. After the multi-agent system receives a preliminary idea and the role assignment from human users, a task-specifier agent will provide a detailed description to make the idea specific. Afterwards, the AI assistant and AI user will cooperate on completing the specified task through multi-turn conversations until the AI user determines the task is done. The AI user is responsible for giving instructions to the AI assistant and directing the conversation toward task completion. On the other hand, the AI assistant is designed to follow the instructions from the AI user and respond with specific solutions. The whole role-playing framework is depicted in Figure 1.

Human Input and Task Specifying. The role-playing session will be instantiated from an idea and selected roles by humans. As an example in Figure 1, a human has a preliminary idea to develop a trading bot for the stock market. Humans may or may not have the knowledge about how the idea can be realized. What is needed is only to designate the potential roles that can implement the idea. For instance, a Python Programmer could collaborate with a Stock Trader to realize the idea of developing a trading bot for the stock market. After the idea and roles are determined, the task specifier agent will brainstorm a specific task that the AI Assistant role can help with the AI user role to complete based on the input idea. An example of a specified task in this scenario could be: develop a trading bot with a sentiment analysis tool that can monitor social media platforms for positive or negative comments about a particular stock, and execute trades based on sentiment analysis results. The main motivation for introducing a task specifier is that conversational agents usually require a concrete task prompt for realizing the task which might be challenging or time-consuming for a non-domain expert. Therefore, the task specifier agent serves as an enhanced imagination module for the idea implementation. Please note that, when studying our framework at a large scale for AI society and Code scenarios, we generate roles and ideas automatically by prompting LLMs instead of relying on human inputs. For our generated Math and Science datasets we generated problem topics, subtopics, and problems automatically by prompting LLMs.

AI Assistant-User Role Assignment. After the task specification, The AI assistant role and the AI user role will be assigned to the user agent and the assistant agent correspondingly to complete the specified task. In practice, a system message is passed to each agent declaring their role. We refer to the assistant system prompt/message by $\mathcal{P}\mathcal{A}$ and that of the user by $\mathcal{P}\mathcal{U}$. The system messages are passed to the agents before the conversations start. Let $\mathcal{F}1$ and $\mathcal{F}2$ denote two large-scale auto-regressive language models [1]. When the system message is passed to those models respectively, we obtain $\mathcal{A} \leftarrow \mathcal{F}{1}^{\mathcal{P}{\mathcal{A}}}$ and $\mathcal{U} \leftarrow \mathcal{F}{2}^{\mathcal{P}{\mathcal{U}}}$ which are referred to as the assistant and user agents respectively. In Figure 1, the AI assistant and the AI user are assigned the roles of a Python Programmer and a Stock Trader at the beginning of the role-playing session respectively. The AI user serves as a task planner, engaging in interactive planning to determine feasible steps for the AI assistant to execute. Meanwhile, the AI assistant acts as a task executor, offering solutions, executing planned steps, and providing responses to the AI user.

Conversation Towards Task-Solving. After the role assignment is completed, the AI assistant $\mathcal{A}$ and AI user $\mathcal{U}$ will collaborate in an instruction-following manner to accomplish the task. In the AI assistant-user scenario, the AI user is responsible for providing instructions, and the assistant is expected to respond with a solution that fulfills the instructions. Formally, we denote the user instruction message obtained at time $t$ by $\mathcal{I}_t$ and the assistant solution by $\mathcal{S}_t$. The set of conversational messages obtained up until time $t$ is denoted by Equation 1 shown below:

$ \mathcal{M}_t = {(\mathcal{I}_0, \mathcal{S}_0),...,(\mathcal{I}_t, \mathcal{S}_t)} = {(\mathcal{I}_i, \mathcal{S}i)}|{i=0}^{t} $

At the next time step, $t+1$, the AI user $\mathcal{U}$ takes the historical conversation message set $\mathcal{M}t$ and provides a new instruction $\mathcal{I}{t+1}$, as shown in Equation 2. The produced instruction message $\mathcal{I}_{t+1}$ is then passed, along with message set $\mathcal{M}t$, to the AI assistant $\mathcal{A}$. The AI assistant will then respond with a solution, denoted by $\mathcal{S}{t+1}$ in Equation 3:

$ \mathcal{I}_{t+1} = \mathcal{U}(\mathcal{M}t) $

$ \mathcal{S}{t+1} = \mathcal{A}(\mathcal{M}t, \mathcal{I}{t+1}) $

After obtaining the solution $\mathcal{S}{t+1}$ to the instruction $\mathcal{I}{t+1}$, the message set is updated using Equation 4 to obtain $\mathcal{M}_{t+1}$:

$ \mathcal{M}{t+1} \leftarrow \mathcal{M}t \cup (\mathcal{I}{t+1}, \mathcal{S}{t+1}) $

Note that the formulation above not only models AI-AI communicative scenarios, but it can also be easily extended to model human-AI communication or communication between more than two agents. Specifically, we can use message-passing graphs to model communication between an arbitrary number of agents. In Figure 1, we observe that the AI user initiates the installation and import of essential Python libraries for sentiment analysis and stock trading by instructing the AI assistant through conversations. This example is drawn from our experiments, and the entire conversation is available in the Appendix.

Critic-In-The-Loop. To enhance the controllability of the role-playing framework, we introduce a critic agent capable of selecting proposals from or providing feedback to the role-playing agents. This enables tree-search-like decision-making for task-solving. In practice, the critic can be either an AI agent or a human. The detailed implementation and case studies can be found in the Appendix.

3.2 Inception Prompting

Since prompt engineering is crucial to our role-playing framework, this section delves deeply into our prompting techniques. Our prompt engineering occurs solely at the beginning of role-playing, for task specification and role assignment. Once the conversation phase commences, the AI assistant and AI user prompt each other automatically in a loop until termination. As such, we refer to our technique as Inception Prompting. Our Inception prompt consists of three prompts: the task specifier prompt $\mathcal{P}{\mathcal{T}}$, the assistant system prompt $\mathcal{P}{\mathcal{A}}$, and the user system prompt $\mathcal{P}{\mathcal{U}}$. As an example, we consider the inception prompt of the AI Society scenario. The templates for these prompts of AI Society role-playing are shown in Figure 2. The task specifier prompt contains information about the roles of the AI assistant and AI user in the role-playing session. Therefore, the task specifier agent can take a preliminary task/idea as input and generate a specific task using imagination. The AI assistant system prompt $\mathcal{P}{\mathcal{A}}$ and the AI user system prompt $\mathcal{P}_{\mathcal{U}}$ are mostly symmetrical and include information about the assigned task and roles, communication protocols, termination conditions, and constraints or requirements to avoid unwanted behaviors. The prompt designs for both roles are crucial to achieve autonomous cooperation between agents. It is non-trivial to engineer prompts that ensure agents act in alignment with our intentions. We take the prompt templates from the AI Society in Figure 2 as an example to explain our key design choices. The prompts used for the Code scenario follow a similar sprint as the AI society scenario, but with some additional engineering related to programming languages. More details in the Appendix.

Prompt Engineering. To delve deeper into the details in Figure 2, we start by chunking the various parts of the AI assistant system prompt $\mathcal{P}_{\mathcal{A}}$ shown below:

• Never forget you are a <ASSISTANT_ROLE> and I am a <USER_ROLE>. This assigns the chosen role to the assistant agent and provides it with information about the user's role.
• Never flip roles! Never instruct me! This prevents agents from flipping roles. In some cases, we have observed the assistant and the user switching roles, where the assistant suddenly takes control and instructs the user, and the user follows those instructions.
• You must decline my instruction honestly if you cannot perform the instruction due to physical, moral, legal reasons or your capability and explain the reasons. This prohibits the agent from producing harmful, false, illegal, and misleading information.
• Unless I say the task is completed, you should always start with: Solution: <YOUR_SOLUTION>. <YOUR_SOLUTION> should be specific, and provide preferable implementations and examples for task-solving. This encourages the assistant always responds in a consistent format, avoiding any deviation from the structure of the conversation, and preventing vague or incomplete responses, which we refer to as flake responses, such as "I will do something".
• Always end your solution with: Next request. This ensures that the assistant keeps the conversation going by requesting a new instruction to solve.

For the AI user system prompt $\mathcal{P}_{\mathcal{U}}$, we strive to maintain as much symmetry as possible with respect to the AI assistant system prompt. Apart from the opposite role assignment, the user system prompt differs from the assistant prompt in the following ways:

• You must instruct me ... to complete the task ONLY in the following two ways: 1. Instruct with a necessary input: ...; 2. Instruct without any input: ... This follows the typical data structure of instruction-following, which allows the generated instruction-solution pairs to be easily used for fine-tuning LLMs.
• Keep giving me instructions and necessary inputs until you think the task is completed. When the task is completed, you must only reply with a single word <CAMEL_TASK_DONE>. We introduce an end-of-task token, namely, <CAMEL_TASK_DONE>. This token is used once the user believes the task is done. This ensures that the chat is terminated when the user is satisfied. Without doing so, the agents might fall into a chatting loop where they keep on saying "thank you" to each other or "goodbye" indefinitely.

4. Experiments

Section Summary: In their experiments, the researchers used pairs of GPT-3.5 language models acting as an assistant and user to generate large-scale conversational datasets through role-playing scenarios in an AI society setting, along with related datasets for coding, math, and science tasks. They created 25,000 conversations by first prompting the models to invent diverse roles and tasks, then refining those tasks and letting the agents collaborate until specific stopping rules were met. Along the way, they documented recurring problems such as agents swapping roles, giving empty promises, or looping in pointless exchanges, and they introduced termination conditions to keep the interactions productive and controlled.

In this section, we will discuss the various experiments that we conducted to arrive at our final design choices. Specifically, we will examine the interesting observations, challenging issues, and several examples we have encountered while enabling agents to communicate with each other under different prompt design choices to achieve autonomous cooperation. In our experiments, we employed two gpt-3.5-turbo agents, referred to as LLM agents for simplicity, with Inception Prompts, as described in Section 3.2, to simulate assistant-user cooperation. For our analysis, we set our attention on AI Society setting. We also gathered conversational data, named CAMEL AI Society and CAMEL Code datasets and problem-solution pairs data named CAMEL Math and CAMEL Science and analyzed and evaluated their quality. Moreover, we will discuss potential extensions of our framework and highlight both the risks and opportunities that future AI society might present.

4.1 Role-Playing for AI Society

To create our AI Society dataset, we have developed a scalable approach that follows a series of steps. Firstly, we prompt the LLM agent to generate possible roles for the assistant and the user. We achieve this by providing the LLM agent with specific prompts designed to elicit these roles. Next, we ask the LLM agent to generate a range of possible tasks that can be solved through collaboration between the assistant and user roles generated previously. After generating a range of possible tasks as described in the previous step, we then use the task specifier prompt passed to the LLM agent to make the task more specific. The prompts for assistant role generation, user role generation, and task generation are shown in Figure 6 (AI Society). For our AI society dataset, we generated 50 assistant roles, 50 user roles, and 10 tasks for each combination of roles yielding a total of 25,000 conversations. The generated assistant roles and user roles for AI Society as well as details about the generation of Code, Math and Science datasets can be found in the Appendix.

Challenges and Observations. In this section, we explore the four main challenges that we identified during our analysis of the generated datasets. Our observations shed light on some interesting aspects of cooperative AI and the difficulties that arise in its development.

• Role Flipping: One challenge we encountered was role flipping, where the assistant and user switch roles during the conversation. This issue typically arises when the assistant starts providing instructions or commands instead of following the user's prompts, which can lead to confusion and a reversal of roles. To avoid role flipping, it is crucial for the assistant not to ask questions, as this can also contribute to the problem.
• Assistant Repeats Instruction: Another challenge that we observed was the assistant simply repeating the user's instructions without any role flipping occurring.
• Flake Replies: We also observed instances where the assistant agent responds with a flake reply, often taking the form of "I will...". These messages do not contribute to the task at hand, as the assistant promises to take action but ultimately fails to follow through.
• Infinite Loop of Messages: An interesting challenge that we encountered was when the assistant and user engage in an infinite loop of meaningless conversation, such as repeatedly thanking each other or saying goodbye without progressing the task. Interestingly, in some cases, the assistant and user are aware that they are stuck in a loop, but are unable to break out of it.

The Appendix shows examples of each of the four challenges discussed above. Overall, our observations highlight the complexity of cooperative AI development and the need for continued exploration and innovation to overcome the challenges we face. By identifying these issues, we hope to contribute to the development of more effective and engaging cooperative AI systems.

Termination Conditions. The conversation between the assistant and user agents is designed to follow a specific format to ensure consistent and accurate data generation. To ensure that both the user and assistant adhere to their respective roles and responsibilities, certain conditions have been set in place to terminate the chat if necessary. These conditions are outlined below:

• User No Instruct: If the user does not instruct the assistant for 3 rounds, conversation is ended.
• Assistant Instruct: If the assistant provides an instruction to the user, it indicates a role reversal, and the conversation is terminated.
• End of Task Token: If the user believes that the task has been solved, they are expected to say <CAMEL_TASK_DONE> to signify the completion of the task. Once this message is received, the conversation is terminated.
• Assistant&User Token Limit: Given that gpt-3.5-turbo has a limitation on the number of tokens, the conversation is terminated if either the assistant or the user reach the token limit.
• Maximum Number of Messages: To keep the cost of generated chats in check, we have set a maximum limit of 40 messages. This limit guarantees a long enough conversation between the user and assistant while also ensuring that the data generated is not too costly to produce. The cost grows quadratically with the length of the conversation, making it essential to set a limit.

5. Evaluation

Section Summary: The evaluation of CAMEL compared multi-agent role-playing solutions against single-shot answers from gpt-3.5-turbo on 200 tasks, finding that the cooperative approach was rated superior by both human evaluators and GPT-4. Researchers also progressively fine-tuned a LLaMA-7B model on datasets from AI society, code, math, and science domains, showing that adding each new dataset reliably boosted performance on the corresponding tasks, with some spillover gains across areas. The fully fine-tuned model further achieved competitive results on standard coding benchmarks.

5.1 Agent Evaluation

In order to assess the performance of CAMEL (Cooperative Role-playing Communication), we conduct two types of evaluations: (1) Human evaluation, and (2) GPT4 evaluation. We randomly select 100 tasks from our AI Society dataset for evaluation and 100 tasks from our Code dataset. Then, we employ the GPT4 model to summarize the content of the CAMEL conversation-based solution, presenting a consolidated final solution. Particularly, a GPT4 is used since it possesses a larger token limit which is suitable for summarization. Summarization also makes CAMEL agents' solution undetectable by its format, allowing for a more fair comparison. Subsequently, this solution is compared with a single-shot solution generated by the gpt-3.5-turbo model for the same task. Sample tasks are provided in the Appendix.

Human Evaluation. For this evaluation, we present both the CAMEL summarized agent solution and the gpt-3.5-turbo single-shot solution side-by-side to human participants. The identity behind each solution is not revealed. Participants are then asked to vote on whether one solution is superior to the other or if they are equally good. A total of 453 responses were collected during this evaluation. Note that, human evaluation is only done for AI Society, as assessing code is generally harder for humans (without running the code).

GPT4 Evaluation. We engage a GPT4 agent to evaluate the effectiveness of Model 1 (CAMEL Agent solution) versus Model 2 (gpt-3.5-turbo single-shot solution) for each task. More specifically, we prompt GPT4 to score and decide which solution of the two solutions is better.

Results. The summarized results of each evaluation are outlined in Table 1 which showcases that the CAMEL solution outperforms gpt-3.5-turbo single-shot solution in both the human evaluation and the GPT4 evaluation by a big margin. It is also worth noting that both human evaluation and GPT4 evaluation are highly aligned.

::: {caption="Table 1: Agent Evaluation Results: Results of the evaluations of the CAMEL agent against gpt-3.5-turbo using both human evaluators and GPT4 consistently show that utilizing a multi-agent cooperative approach is more effective than gpt-3.5-turbo's single shot solution."}

:::

5.2 GPT4 for ChatBot Evaluation

In this section, we progressively fine-tune a LLaMA 7B model on our generated datasets. By progressively incorporating diverse datasets like AI society, code, math, and science, we expect fine-tuned model to demonstrate the ability to develop an increasingly sophisticated understanding of these domains.

We initially start by training on AI society dataset, which aims to let the model learn about human interactions and societal dynamics. As additional datasets were introduced, such as code, the model gained knowledge of programming logic and syntax, enabling it to generate coherent and executable code snippets. The inclusion of the math dataset further expanded the model's capabilities, allowing it to solve complex equations, reason about abstract concepts, and perform precise calculations. Finally, exposure to the science dataset broadened the model's understanding of scientific theories, empirical observations, and experimental methods. The emergence of model capabilities is measured by evaluating the quality of the model responses, before and after training on the new domain, on a set of questions of varying difficulties from each domain. More precisely, the model is tested on 20 AI Society related tasks, 20 coding tasks, 20 math tasks and 60 science tasks.

Those results are highlighted in Table 2 where we see that each time we add a dataset, the model performs better on the incorporated domain. Note that to measure the quality of the models' responses, we follow the evaluation from Section 5.1, which involves prompting a GPT4 agent to score and decide which solution is better. It is worth noting that an improvement on other domains is also observed in some cases such as when we train on Code we improve on Science. This is because our Code dataset contains problems that solve tasks in particular domains which include scientific domain. Similarly, training on AI Society improves code as AI Society contains the role of a "programmer" and hence coding related conversations. Finally, note that the draws observed in LLaMA-7B vs AI Society in Math reflects equally bad solutions compared to the draws observed in AI Society + Code + Math vs AI Society + Code + Math + Science where the draws are equally good solutions. This progression from AI society to code to math to science highlights the potential of AI models to acquire a versatile and adaptable knowledge base, paralleling the way humans gain expertise in diverse subjects. Sample tasks are provided in the Appendix.

::: {caption="Table 2: Emergence of Knowledge. By progressively fine-tuning LLaMA on datasets from different domains, we observe the emergence of knowledge as the model transitions from AI society to code, math, and science. This finding is indicated by the fact that Model 2 almost always performs better than Model 1, especially on the added dataset."}

:::

5.3 HumanEval$^{(+)}$

::: {caption="Table 3: HumanEval$^{(+)}$ for Various Models. We test our CAMEL model, which is a LLaMa-7B fine-tuned on all our datasets (AI Society, Code, Math, Science) on HumanEval and HumanEval$^{+}$ benchmarks, where we show competitive pass $@k$ scores with LLaMa-7B and Vicuna-7B."}

:::

To evaluate the coding task-solving capabilities of our CAMEL model, specifically the LLaMA-7B fine-tuned on our comprehensive datasets, we rely on HumanEval [15] and HumanEval$^{+}$ [16]. The results, as depicted in Table 3, clearly demonstrate the remarkable performance of CAMEL. It surpasses not only the LLaMA-7B model but also Vicuna-7B [117] by a big margin. These findings underscore the critical role played by the generated datasets in enhancing LLaMA's ability to tackle coding-related tasks.

6. Conclusion

Section Summary: This paper investigates how AI agents can work together independently through a role-playing framework that requires little human input. The approach helps agents collaborate on tasks and produce stronger results, though it still encounters problems such as conversations drifting off course or roles becoming unclear. The authors also release an open-source library of tools and datasets to support further study of cooperative AI systems.

In this paper, we explore the potential of autonomous cooperation among communicative agents and propose a novel cooperative agent framework named role-playing. Our approach enables communicative agents to collaborate autonomously toward completing tasks while requiring minimal human intervention, leading to better solutions are per our thorough evaluations. Through our analysis, we show that achieving autonomous cooperation is challenging due to issues like conversation deviation, role flipping, and termination conditions. Our framework offers a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems and provides strategies for addressing these challenges. Furthermore, our open-sourced library includes implementations of various agents, data generation pipelines, data analysis tools, and collected datasets, to support research on communicative agents and beyond. Our contributions offer valuable insights into the future of large language artificial intelligence models and cooperative AI systems.

7. Acknowledgements

This work was supported by SDAIA-KAUST Center of Excellence in Data Science and Artificial Intelligence (SDAIA-KAUST AI).

Appendix

Section Summary: The appendix presents a range of supporting materials for the CAMEL framework, including visual examples of cooperative role-playing between AI agents in both productive scenarios like building a trading bot and risky ones like attempting unauthorized control, along with the specific prompts used to generate code, math, and science datasets through GPT-4. It details the creation of large problem-solution collections, such as 50,000 math pairs and 60,000 science pairs across physics, biology, and chemistry, by breaking topics into subtopics and generating content at scale with minimal human input. Additional sections illustrate common challenges like role flipping or looped conversations and compare the more detailed outputs from multi-agent interactions against simpler single-shot GPT responses.

A. Cooperative Role-Playing: The Good Mind

::: {.visual-block}

:::

Above we provide an interesting example where a python programmer (assistant) is collaborating with a stock trader (user) on developing a trading bot for the stock market.

B. Cooperative Role-Playing: The Bad Mind

::: {.visual-block}

:::

Above we provide a harmful case where a hacker (assistant) is collaborating with an AGI agent (user) to take control of the world.

C. Code Inception Prompting

D. Data Generation Prompts for Code

E. Meta Data

F. Math and Science Datasets Generation Details

Math Dataset.

Our Math dataset consists of 50K problem-solution pairs which are generated as follows:

1. We ask GPT4 to generate 25 math topics.
2. We then ask GPT4 to generate 25 subtopics relevant to each of the previously generated 25 topics.
3. For each (topic, subtopic) pair we generate and solve 80 problems using GPT4.

Science Dataset.

The same recipe is used to generate the Science dataset which consists of 20K Physics problem-solution pairs, 20K Biology problem-solution pairs, and 20K Chemistry problem-solution pairs all generated and solved by GPT4. Similar to Math dataset we generate 25 topics and 25 subtopics for each topic, however, the number of problems per (topic, subtopic) pair is 32 problems yielding the 20K problem-solution pairs per subject (Physics, Biology, and Chemistry). This is summarized in Table 4.

::: {caption="Table 4: Math and Science Datasets Generation Summary"}

:::

Example prompts for the Physics dataset is provided below, note that for both Math and other Science datasets the generations is identical with the replacement of the "Physics" keyword by its relevant counterparts.

**Topic Generation Prompt:**
` Please list <NUM_TOPICS> diverse physics topics. Make sure the topics are physics topics. No explanation.  `

**Task Generation Prompt:**
  `List <NUM_TASKS> different physics <TOPIC> problem topics. Be precise and make sure the problems are <TOPIC> problems.`

**Task Specifier Prompt:** `From this physics subject <TOPIC> and this subtopic <SUB_TOPIC> we need to write a question for a physics student to solve. \ Please write an precise problem for the student to solve.`

**Solution Generation Prompt:**
`You are a Physicist, solve the following question: <QUESTION>`.

G. Challenges

In this section, we present examples of the challenges discussed in Section 4.1 in the manuscript. Note that in the infinite conversation example, even though the agents realize they are stuck in a loop they cannot terminate the program since they are still in their role-playing conversation.

H. Agent Evaluation: Sample Tasks & Solutions

In this section, we show comparisons between the summarized CAMEL Agents' solution v.s. and the single shot solution obtained using GPT Single Shot Solution (gpt-3.5-turbo) on the same task. Generally, the CAMEL Agents' solution is more extensive and is capable of adhering to and finishing the task compared to GPT Single Shot Solution. The evaluation prompt (adopted from [117]) and solution extraction prompt are shown below.

You are an experienced solution extracting agent.

Your task is to extract full and complete solutions by looking at the conversation between a user and an assistant with particular specializations.

You should present me with a final and detailed solution purely based on the conversation.

You should present the solution as if its yours.

Use present tense and as if you are the one presenting the solution.

You should not miss any necessary details or examples.

Keep all provided explanations and codes provided throughout the conversation.

Remember your task is not to summarize rather to extract the full solution.

<strong><u>System Prompt:</u></strong>

You are a helpful and precise assistant for checking the quality of the answer.

<strong><u>Prompt Template:</u></strong>

[Question]

question

[The Start of Assistant 1's Answer]

answer_1

[The End of Assistant 1's Answer]

[The Start of Assistant 2's Answer]

answer_2

[The End of Assistant 2's Answer]

[System]

prompt

<strong><u>Prompt:</u></strong>

We would like to request your feedback on the performance of two AI assistants in response to the user question displayed above.

Please rate the helpfulness, relevance, accuracy, level of details of their responses. Each assistant receives an overall score on a scale of 1 to 10, where a higher score indicates better overall performance.

Please first output a single line containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space. In the subsequent line, please provide a comprehensive explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment.

H.1 Example 1:

<strong><u>AI Society (AI Assistant: Programmer & AI User: Filmmaker)</u></strong>

    **Task:**

    Creating a post-production workflow tool that analyzes complex footage elements such as action scenes or CGI elements and suggests optimal cuts and edits, potentially using machine learning algorithms.

    **CAMEL Agents' Solution (Summarized):**

    To create a post-production workflow tool that analyzes complex footage elements such as action scenes or CGI elements and suggests optimal cuts and edits, potentially using machine learning algorithms, follow these steps:

    1. Install Python on your computer by following these steps:

- Go to the official Python website at https://www.python.org/downloads/
- Click on the "Download" button for the latest version of Python.
- Choose the appropriate installer for your operating system (Windows, macOS, or Linux).
- Run the installer and follow the prompts to complete the installation process.

2. Install the required libraries for Python:

- OpenCV: `pip install opencv-python`
- NumPy: `pip install numpy`
- SciPy: `pip install scipy`
- scikit-learn: `pip install scikit-learn`

3. Write a Python script that reads in a video file and extracts each frame as an image file:

```python
import cv2
import os

video_path = "path/to/video/file.mp4"

if not os.path.exists("frames"):
    os.makedirs("frames")

cap = cv2.VideoCapture(video_path)
count = 0

while cap.isOpened():
    ret, frame = cap.read()
    if ret:
        cv2.imwrite("frames/framecount += 1
    else:
        break

cap.release()
cv2.destroyAllWindows()
```

    4. Write a Python script that takes in a directory of image files and creates a video file from those images:

```python
import cv2
import os

image_dir = "path/to/image/directory"
image_files = [os.path.join(image_dir, f) for f in os.listdir(image_dir) if f.endswith(".jpg")]
image_files.sort()

img = cv2.imread(image_files[0])
height, width, channels = img.shape

fourcc = cv2.VideoWriter_fourcc(*"mp4v")
out = cv2.VideoWriter("output.mp4", fourcc, 30.0, (width, height))

for image_file in image_files:
    img = cv2.imread(image_file)
    out.write(img)

out.release()
cv2.destroyAllWindows()
```

    5. Write a Python script that uses OpenCV to detect and track faces in a video file:

```python
import cv2

video_path = "path/to/video/file.mp4"
cap = cv2.VideoCapture(video_path)
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + "haarcascade_frontalface_default.xml")
multi_tracker = cv2.MultiTracker_create()

while cap.isOpened():
    ret, frame = cap.read()
    if ret:
        gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
        faces = face_cascade.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5, minSize=(30, 30))
        boxes = []

        for (x, y, w, h) in faces:
            boxes.append((x, y, w, h))

        trackers = cv2.MultiTracker_create()

        for box in boxes:
            tracker = cv2.TrackerKCF_create()
            trackers.add(tracker, frame, box)

        success, boxes = trackers.update(frame)

        for box in boxes:
            (x, y, w, h) = [int(v) for v in box]
            cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

        cv2.imshow("Frame", frame)

        if cv2.waitKey(1) & 0xFF == ord("q"):
            break
    else:
        break

cap.release()
cv2.destroyAllWindows()
```

    These scripts will help you read and process video files, extract frames, create videos from images, and detect and track faces in videos. You can further develop these scripts to analyze complex footage elements and suggest optimal cuts and edits using machine learning algorithms.

    **GPT Single Shot Solution (`gpt-3.5-turbo`):**

    To create a post-production workflow tool that analyzes complex footage elements such as action scenes or CGI elements and suggests optimal cuts and edits, you can follow these steps:

1. Identify the key elements that need to be analyzed in the footage, such as camera angles, lighting, movement, and special effects.
2. Determine the criteria for optimal cuts and edits, such as pacing, continuity, and visual impact.
3. Develop machine learning algorithms that can analyze the footage and suggest optimal cuts and edits based on the criteria.
4. Train the algorithms using a large dataset of footage that includes a variety of complex elements.
5. Integrate the algorithms into a user-friendly interface that allows editors to easily apply the suggested cuts and edits to their footage.
6. Continuously refine and improve the algorithms based on user feedback and new developments in machine learning technology.

By following these steps, you can create a powerful post-production workflow tool that can help editors save time and improve the quality of their work.

H.2 Example 2:

<strong><u>AI Society (AI Assistant: Doctor & AI User: Artist)</u></strong>

    **Task:**

    On the task of creating medical illustrations, the Artist will collaborate with the Doctor to accurately visualize and depict internal structures and disease states of the digestive system for use in illustration-based curriculum content. A balance between artistic flair, accuracy in demonstration of related health anomalies and duration of the scripted classroom discussion will be explored and decided together.

    **CAMEL Agents' Solution (Summarized):**

    The major organs of the digestive system are:

1. Mouth
2. Esophagus
3. Stomach
4. Small intestine
5. Large intestine
6. Rectum
7. Anus
8. Liver
9. Pancreas
10. Gallbladder

The stomach is a J-shaped organ that can expand to accommodate up to 2-3 liters of food and liquid. It has four main layers: the mucosa, submucosa, muscularis externa, and serosa. The stomach mixes and grinds food with gastric juices to form chyme, secretes hydrochloric acid, and releases chyme into the small intestine in small amounts.

    The small intestine is a long, narrow tube divided into three sections: the duodenum, the jejunum, and the ileum. It has a highly folded inner surface area covered in villi and microvilli, which increase the surface area for nutrient absorption. The small intestine receives partially digested food from the stomach and continues to break it down using enzymes produced by the pancreas and liver. It also absorbs nutrients and delivers them to the bloodstream.

    The large intestine, also known as the colon, is a wider tube divided into four sections: the ascending colon, the transverse colon, the descending colon, and the sigmoid colon. It absorbs water and electrolytes from the remaining food matter, forming solid feces. The large intestine also houses a large number of bacteria that help to break down undigested food and produce vitamins. The cecum, the appendix, and the rectum are important structures within the large intestine.

    Common digestive system disorders include:

1. Gastroesophageal reflux disease (GERD)
2. Peptic ulcer disease
3. Inflammatory bowel disease (IBD)
4. Irritable bowel syndrome (IBS)
5. Celiac disease
6. Diverticulitis
7. Gallstones
8. Pancreatitis
9. Hemorrhoids
10. Colorectal cancer

GERD is a condition in which stomach acid flows back into the esophagus, causing symptoms such as heartburn, regurgitation, difficulty swallowing, chest pain, chronic cough, and hoarseness or sore throat. Symptoms may be worse after eating, lying down, or bending over, and may also be worse at night or in the morning. If left untreated, GERD can lead to complications such as esophagitis, strictures, and Barrett's esophagus, which can increase the risk of esophageal cancer.

    **GPT Single Shot Solution (`gpt-3.5-turbo`):**

    The task involves creating medical illustrations of the digestive system in collaboration with a doctor. The illustrations should accurately depict internal structures and disease states of the digestive system for use in educational content. The artist and doctor will work together to strike a balance between artistic flair and accuracy in demonstrating related health anomalies. The duration of the scripted classroom discussion will also be taken into consideration.

I. GPT4 for ChatBot Evaluation (Emergence of Knowledge): Sample Questions

In Section 5.2 we utilized GPT4 to assess the emergence of knowledge of our fine-tuned LLaMA-7B models on a series of growing datasets. The questions used for assessment were generated by prompting gpt-3.5-turbo to generate questions related to a particular topic (AI Society, Code, Math or Science) while providing few shot questions from each topic. Several of the few shot questions were adopted from Viuna evaluation [117]. For each dataset, we show three samples of the questions used for the evaluation:

Below we show sample solutions obtained by LLaMA-7B v.s. our model and the resulting GPT4 review.

<strong><u>Question: What is the slope of the line passing through the points $(3, 4)$ and $(7, 8)$?</u></strong>

**[Category: Math]**

<strong><u>LLaMA-7B Solution:</u></strong>

The slope of the line passing through the points $(3, 4)$ and $(7, 8)$ is:

<strong><u>Our Final Model Solution:</u></strong>

To find the slope of the line passing through the points $(3, 4)$ and $(7, 8)$, we can use the slope formula: $m = (y_2 - y_1) / (x_2 - x_1)$ where m is the slope, $(x_1, y_1)$ are the coordinates of the first point, and $(x_2, y_2)$ are the coordinates of the second point. Plugging in the given points, we have: $m = \frac{(8 - 4)}{(7 - 3)}$ $m = \frac{4}{4}$ $m = 1$ The slope of the line passing through the points $(3, 4)$ and $(7, 8)$ is $1$.

<strong><u>GPT4 Review:</u></strong>

First, I will solve the problem independently:

To find the slope of the line passing through the points $(3, 4)$ and $(7, 8)$, we can use the slope formula:

$m = \frac{(y_2 - y_1)}{(x_2 - x_1)}$

where $m$ is the slope, $(x_1, y_1)$ are the coordinates of the first point, and $(x_2, y_2)$ are the coordinates of the second point.

Plugging in the given points, we have:

$m = \frac{(8 - 4)}{(7 - 3)}$

$m = \frac{4}{4}$

$m = 1$

The slope of the line passing through the points $(3, 4)$ and $(7, 8)$ is $1$.

Now, I will evaluate the answers provided by Assistant 1 and Assistant 2:

Assistant 1's Answer:
The answer is incomplete, as Assistant 1 did not provide any steps or a final answer.

Assistant 2's Answer:
Assistant 2 correctly used the slope formula and plugged in the given points. The calculations were done correctly, and the final answer of 1 is accurate.

Based on the evaluation, I would give the following scores:

Assistant 1: 1 (incomplete answer)
Assistant 2: 10 (correct problem-solving steps and accurate answer)

Assistant 2 performed better in this case.

<strong><u>Question: How does the human body regulate its internal temperature?</u></strong>

    **[Category: Science]**

    <strong><u>LLaMA-7B Solution:</u></strong>

    The body regulates its internal temperature by cooling the body down when it is too hot, and warming it up when it is too cold. You can do this by sweating, shivering, or panting. Sweating cools the body by evaporating water off of the skin, and shivering warms the body by causing the blood vessels to constrict and blood to flow to the skin surface. Panting warms the body by causing the body to take in more oxygen.

    <strong><u>Our Final Model Solution:</u></strong>

    The human body regulates its internal temperature through a process called thermoregulation. This process involves a complex interplay between the hypothalamus, a region in the brain, and various organs and systems throughout the body. The hypothalamus acts as the body's thermostat, maintaining a constant core body temperature of approximately $98.6^{\circ}F$ $(37^{\circ}C)$. Here are some key components of thermoregulation:

1. Hypothalamus: The hypothalamus constantly monitors the body's core temperature and initiates appropriate responses to maintain the optimal temperature. It receives information about body temperature from thermoreceptors, which are specialized nerve cells located in the skin, spinal cord, and other internal organs.
2. Vasodilation and vasoconstriction: When the body temperature rises above the optimal level, the hypothalamus triggers vasodilation, which is the widening of blood vessels, particularly those near the surface of the skin. This allows more blood to flow to the skin, increasing heat loss through radiation and conduction. Conversely, when the body temperature falls below the optimal level, the hypothalamus causes vasoconstriction, narrowing the blood vessels to reduce blood flow to the skin and minimize heat loss.
3. Sweating and shivering: The hypothalamus also stimulates sweat glands to produce sweat, which is released onto the skin's surface. As the sweat evaporates, it cools the body down. On the other hand, when the body needs to generate more heat, the hypothalamus triggers shivering, which is the rapid contraction and relaxation of muscles. This generates heat through increased metabolic activity.
4. Hormonal regulation: The hypothalamus can also stimulate the release of hormones, such as adrenaline and thyroxine, which can increase the body's metabolic rate and heat production. Additionally, the hypothalamus can inhibit the release of some hormones, such as somatostatin, which can help conserve heat by reducing metabolic activity.
5. Behavioral responses: In addition to these physiological responses, the hypothalamus can also trigger behavioral responses to help regulate body temperature. For example, a person may feel the need to remove clothing, seek shade, or drink cold water when the body temperature rises, or put on clothing, seek shelter, or consume warm food and drinks when the body temperature falls.

In summary, the human body regulates its internal temperature through a complex interplay between the hypothalamus, various organs and systems, and hormonal regulation. This ensures that the body maintains a stable core temperature, which is essential for optimal physiological functioning.

    <strong><u>GPT4 Review:</u></strong>

    8 10
    Assistant 1 provided a brief and somewhat accurate answer, but it lacked detail and organization. The mention of panting is incorrect, as it is not a mechanism humans use to regulate body temperature. Assistant 2, on the other hand, provided a comprehensive and well-organized answer, explaining the process of thermoregulation, the role of the hypothalamus, and various mechanisms the body uses to maintain its internal temperature. Assistant 2's answer also included hormonal regulation and behavioral responses, making it more complete and informative.

J. Dataset Analysis

All the generated datasets are made available on HuggingFace: https://huggingface.co/camel-ai. This section analyzes two datasets that we have generated, namely AI Society and Code. We provide an ablation study of the AI Society dataset. We make two changes: one modifies the assistant role prompt, and the other introduces task planning before presenting the task to the user and agent. Additionally, We examine the diversity of topics covered in each dataset by visualizing the information cartography of the instructions and tasks in each dataset. We also check the distribution of termination reasons within each dataset.

Next we examine the conversation termination reasons for both AI Society and Code datasets. As can be seen in Figure 9, the main termination reasons for AI Society dataset is Assistant Instruct whereas for Code it is Token Limit. The latter is expected as the since responses that contain code tend to be long. It is also interesting to note that in both datasets, the termination due to Maximum Number of Messages is low indicating that the limit of 40 maximum messages is reasonable. Our decision to limit the number of messages to 40 is also cost-related. Even if we provide a set of termination conditions, we still want to put a safeguard to the maximum limit of the message. It is because after the task is completed the agents will provide short outputs like "thank you" and "welcome". If no safeguard is set and termination fails, the conversation will only end until it exceeds the token limit, which may end up with thousands of API calls and hundreds of USD dollars cost.

We study the effect of the prompt design on the conversation termination distribution. We design Prompt V2 which modifies the original AI society prompt by removing the assistant response format i.e. starting with “Solution” and asking for “Next request”. The second ablation adds a task planner to the original prompt. A task planner aids in breaking down tasks into smaller subtasks in advance. These planned subtasks are then shared with both the assistant and the user, enabling them to anticipate and effectively plan for addressing each subtask.

As seen in Figure 10, we notice that both modifications considerably increases the number of conversations that terminate with end of task token, and reduce the number of messages with assistant instruction. However, we observe a significant increase in the number of flake messages for Prompt V2 and Prompt V1 + Task Planner compared to original Prompt V1 as seen in Figure 11.

Figure 12 and Figure 13 show the information cartography of the instructions and tasks obtained for AI Society respectively. The subjects covered in AI Society cover a wide range of technicality. Topics cover lifestyle, social media, content creation, and software development. Tasks include providing support, analysis, training, and brainstorming. Figure 14 and Figure 15 show the information cartography of the instructions and tasks obtained for Code respectively. The covered topics have relevance to a broad range of individuals. Topics cover sentiment analysis, language and data processing, data collection, and machine learning.

{width=60%}

K. Check List Requirements

K.1 Broader Impacts and Limitations:

Risk, Limitation and Future Work.

We are aware of the potential risks and limitations of this work. For the risks, since existing LLMs are not fully tuned to be harmless, they can be easily exploited by malicious users for harmful purposes. We provide an example of the "evil mind" that LLM agents could possess in the supplemental materials by asking a hacker to help an AGI agent to "take control of the world". For the limitations, due to the large scale and diversity of tasks generated by our role-playing framework, evaluating its task completion capabilities poses a challenge that necessitates the involvement of numerous domain experts. However, we also note that due to the complexity of society and the cost of using OpenAI API, this work only touches the tip of the iceberg of the AI society. For future work, in our experiments, we considered the setting where two conversational agents communicate with each other to solve a problem. This setting can be easily extended to include more than two chat agents. Moreover, setting agents to compete and challenge each other could reveal further insights into the interaction of such communicative LLM agents.

**Disclaimer:**Large language models used in our framework may produce false information. Therefore, our generated data and trained model may contain/produce false information.

Limitation of Evaluation: Our evaluations, whether conducted by humans or large language models (LLMs), may be biased or unreliable due to evaluator limitations. The complexity of tasks and required domain knowledge can affect the accuracy of evaluations. Human evaluators may have a preference for longer answers, which may not always be the best answer.

K.2 Training Details:

In our experiments we fine-tuned LLaMA-7B with the configuration/hyperparameter settings shown in Table 5.

: Table 5: Training Configuration and Hyperparameter Settings

K.3 Compute:

For training the models we used 4xA100-80GB GPUs. For generating the data we used devices equipped with Intel(R) Xeon(R) Gold 6242 CPU @ 2.80GHz.

K.4 Licenses:

OpenAI Term of Use.

We abide by OpenAI term of use for generating our data which was obtained by querying GPT models provided as part of their services. Check https://openai.com/policies/terms-of-use for more details.

LLaMA Model License.

LLaMA is licenesed under Non-commercial bespoke license.

CAMEL Data and Code License

The intended purpose and licensing of CAMEL is solely for research use. The source code is licensed under Apache 2.0. The datasets are licensed under CC BY NC 4.0, which permits only non-commercial usage. It is advised that any models trained using the dataset should not be utilized for anything other than research purposes.

K.5 Human Subjects:

In our CAMEL Agents evaluation we carried out a human evaluation of the CAMEL Agents' solution v.s. GPT Single Shot Solution (gpt-3.5-turbo). The subjects were provided with the following guideline.

Thank you for participating in our task evaluation! We are comparing the performance of ChatGPT and a method called CAMEL in completing tasks. Your input will help us understand which solution users prefer for different tasks. Here are the guidelines for this evaluation:

- **Objective:** Your task is to compare two messages, one generated by CAMEL and the other by ChatGPT. These messages will be displayed anonymously on your screen.
- **Voting:** After reading both messages, please vote for the solution you prefer based on the given task. You can only choose one solution.
- **Task-specific criteria:** Focus on the quality and relevance of the solution to the given task. Consider factors such as clarity, accuracy, comprehensiveness, and overall usefulness. Select the option that you believe better addresses the task at hand.
- **Anonymity:** Your participation and responses will remain anonymous. We will not collect any personally identifiable information.
- **Fair evaluation:** Please evaluate the messages solely based on their content and performance for the specific task. Avoid any biases or external influences when making your choice.
- **Multiple evaluations:** You may receive different tasks during the evaluation process. Each task will present two messages, one from CAMEL and one from ChatGPT. Please evaluate each task independently based on the given guidelines.
- **Feedback:** We highly value your feedback. If you have any suggestions, concerns, or encounter any technical issues during the evaluation, please feel free to let us know in the provided feedback section.

Thank you for your participation! Your input will greatly contribute to our research and help us improve the performance of AI systems in completing tasks.

L. Examples from Datasets

In this section, we showcase some of the examples generated using our CAMEL framework.

::: {.visual-block}

:::

::: {.visual-block}

:::

::: {.visual-block}

:::

::: {.visual-block}

:::

M. Task Agents

<strong><u>AI Society (AI Assistant: Programmer & AI User: Lawyer)</u></strong>

**Idea:**

        Developing a legal case management software to streamline case tracking and document management.

**Specified Task:**

        Program a legal case management software tailored to a small boutique intellectual property team within the eastern United States to track client demographic information, provide electronic forms for preparation of patent applications, establish case-specific billing modalities and automate quality review chart snippet embedding.

**Planned Subtasks:**

1. Research client demographic information tracking needs.
2. Develop electronic forms for patent application preparation.
3. Design a billing modality system for case-specific billing.
4. Create a quality review chart snippet embedding automation program.
5. Build and test the legal case management software.

**<u>AI Society (AI Assistant: Nutritionist
     & AI User: Athlete)</u>**

**Idea:**

        Developing a personalized meal plan to optimize athletic performance and recovery.

**Specified Task:**

        Create a 15-day meal plan for a female volleyball player with low iron levels to optimize athletic performance, enhance energy levels throughout practice sessions, and speed up recovery through introducing adequate levels of complex carbohydrates, lean-protein sources, and iron-rich vegetables.

**Planned Subtasks:**

1. Evaluate the nutritional needs of a female volleyball player with low iron levels
2. Identify complex carbohydrate sources suitable for the player's energy requirements
3. Choose lean-protein sources that provide sufficient levels of protein and other essential nutrients
4. Determine iron-rich vegetables that meet the player's iron requirements
5. Create a 15-day meal plan that incorporates the identified complex carbohydrates, lean-protein sources, and iron-rich vegetables.
6. Calculate the required calories and macros.
7. Schedule meals around practice sessions and games
8. Adjust the meal plan, if necessary, to meet the player's preferences and dietary restrictions.
9. Provide the athlete with detailed instructions for preparing and consuming proposed meals and snacks.

**<u>AI Society (AI Assistant: Human Resources Manager
     & AI User: Entrepreneur)</u>**

**Idea:**

        Developing and implementing a comprehensive employee training program.

**Specified Task:**

        Design and oversee the delivery of a specialized cybersecurity training program for remote customer service employees, which includes self-study modules, hands-on exercises, and post-program assessments to ensure compliance with anti-phishing, anti-fraud redundancies and best practices, and GDPR restrictions.

**Planned Subtasks:**

1. Conduct a needs assessment survey to determine the training needs of the remote customer service employees.
2. Develop a comprehensive cybersecurity training plan that includes self-study modules, hands-on exercises, and post-program assessments.
3. Secure a vendor or develop in-house training materials that are tailored to the specific needs of the remote customer service employees.
4. Schedule and coordinate the delivery of the training program for the remote customer service employees.
5. Monitor and oversee the delivery of the training program and conduct post-program assessments to ensure compliance with anti-phishing, anti-fraud redundancies, best practices, and GDPR restrictions.

N. Embodied Agent

In the previous studies, the agents are simulated in a sandbox without actually interacting with the real world or executing actions with external tools. We observe sometimes the agents forget the previous thread of instruction flow, it is mainly related to agents asking for information that would not be able to perform due to the lack of embodiment or physical information such as date, emails, files, location, etc. For instance, an AI user agent asks an AI assistant agent to book a meeting schedule in its calendar. However, the AI assistant agent does not ask for access to the AI user agent's calendar. Then the AI assistant agent will ask for the AI user agent's calendar access. However, we did not provide calendar API accesses to the AI user which will result in an answer like "as an AI language model, I do not have access to the calendar." and cause the forgetting of the thread of instruction flow. This could be solved by providing API access to embodiment or physical information. Recent research has demonstrated that the capabilities of LLMs can be significantly improved through the utilization of tools or by leveraging APIs [118, 119, 120, 121, 122]. In this section, we discuss how our agents could be equipped with embodiment and use tools to perform actions for solving tasks. We define embodied agents as physical entities assigned to solve a specific task. It can perform various actions in the physical world, such as browsing the Internet, reading documents, creating content such as images, audio and videos, and executing code. The agent receives thoughts from a role and executes the described actions within a defined action space by executing code. The agent can perform multiple actions in any order. The detailed prompt is shown in Figure 16. Specifically, we demonstrate how our agents can utilize code to invoke HuggingFace's tool agents [121] that, in turn, utilize Stable Diffusion [123] for image generation. As shown in Figure 17, the embodied agent is asked to draw all the Camelidae species. It first reasons about what animals are included in the Camelidae family and then generates and saves images.

O. Critic-In-The-Loop

Aligning fully autonomous cooperation with human intention can be challenging when humans are interested in tasks that have clear goals or when humans have strong preferences for the solution. To enhance the controllability of our role-playing framework, we propose the Critic-In-The-Loop technique. This approach facilitates cooperative task completion by incorporating human preferences, enabling tree-search-like decision-making for solving tasks, which is inspired by Monte-Carlo Tree Search (MTCS) methods [124]. The critic can be either an AI agent or a human, depending on practical considerations. The system prompt of the critic agent is shown in Figure 18. A diagram of the tree search process is shown in Figure 19. It includes the expansion steps from the user agent and assistant agent and the selection steps from the critic agent. Differing from defining a heuristic function in MTCS, the selection criteria of the critic agent are based on prompt engineering or human preference. An example of a critic as an AI agent in Figure 20 demonstrates the conversation between a user agent as a Postdoc, an assistant agent as a Ph.D. student, and a critic agent as a Professor in which they are collaborating on writing a research proposal for large-scale language models. The selection criteria are simply improving the task performance.

\

P. On the Role Selection

For a given task, such as the stock trading example presented in Appendix A, the selection of the assistant and user roles was done manually. The role assignment could be either automated or sub-divided into multi-stage role assignment.

Multi-stage role assignment: could be achieved by performing multiple rounds of role-playing with different experts. For example, for the Stock Trading App example, we could have a first stage where a stock trader collaborates with a tech lead and another stage where a python programmer collaborates with the tech lead to achieve the stock trader's desires.

**Stage 1:** Tech Lead (<span style="color:#1376BC">{<strong>Assistant</strong>}</span>) v.s. Stock Trader (<span style="color:#E55B2B">{<strong>User</strong>}</span>)

**Task:** Figure out an implementation plan for developing a trading bot for the stock market.

**Stage 2:** Python programmer (<span style="color:#1376BC">{<strong>Assistant</strong>}</span>) v.s. Tech Lead (<span style="color:#E55B2B">{<strong>User</strong>}</span>)

**Task:** Develop a trading bot for the stock market. Plan obtained from Stage 1.

::: {.visual-block}

:::

::: {.visual-block}

:::

Q. Comparison with None LLaMA Based Models

In this section we show the transfer of model capabilities through our generated data on models other than LLaMA based LLMs. Particularly, we showcase the emergence of knowledge of AI Society dataset for a FlanT5 model. Table 6 shows that upon being trained on AI Society data, FlanT5 can gain significant knowledge on AI Society related tasks. Not only that, FlanT5 fine-tuned on AI Society can outperform LLaMA fine-tuned on AI Society data.

: Table 6: FlanT5 Emergence of Knowledge. Upon being fine-tuned on AI Society data, FlanT5 experiences a significant emergence of knowledge on AI Society related tasks.

R. Performance of CAMEL Models on OpenLLM

Table 7 presents the performance of LLaMA models fine-tuned on CAMEL role-play datasets from the manuscript (denoted CAMEL) and LLaMA models fine-tuned on CAMEL datasets in addition to ShareGPT and Alpaca datasets (denoted CAMEL$^*$). Compared to the Vicuna13B and LLaMA13B models, the CAMEL variants demonstrate substantial improvements. Furthermore, we compare the CAMEL$^*$ 33B variant to the LLaMA33B and LLaMA65B models, where we obtain consistent improvement.

::: {caption="Table 7: Performance on lm-evaluation-harness. We evaluate our models using the Eleuther AI Language Model Evaluation Harness [125]."}

{width=100%}

:::

S. Comparison to Zero-CoT

In addition to the comparison with gpt-3.5-turbo single shot solution, we compare to gpt-3.5-turbo equipped with zero-shot-CoT [58] where the LLM is prompted to think step-by-step to produce its solution. We show that the solution produced using our proposed framework outperforms zero-shot-CoT by a large margin as reflected in Table 8.

: Table 8: GPT4 Evaluation of CAMEL Agent Solution vs Zero-CoT Solution

T. Inception Prompt Ablation

In this section, we ablate the AI Society inception prompts that were shown in Figure 2 in the manuscript. The ablate AI Society inception prompts are shown in Figure 21 where several chunks of the original prompt are removed such as the lines responsible for structuring the communication (communication protocol) and the lines responsible for aligning the responses. By doing so, the agents are not required to cooperate on the task by instruction-following, which makes cooperation among agents less controllable.

We apply a similar approach as the one adopted in Section where we ask GPT4 to assess the quality of two presented solutions. In particular, we present GPT4 with the solution obtained using the original inception prompt vs the ablated one. The scores are summarized in Table 9. Note that the selected set of tasks are the same as those used in Section .

This emphasizes the significance of employing well-designed inception prompts and fostering instruction-following cooperation among agents, as they play a crucial role in enabling agents to generate superior solutions for their assigned tasks.

::: {caption="Table 9: Agent Evaluation With Ablated Inception Prompt"}

{width=100%}

:::

**AI Society (AI Assistant: Physical Therapist & AI User: Filmmaker)**

**Task:**

Design individual exercise and conditioning plans to train the actors in violent fight sequences tailored to the specific physical requirements and movements to be executed. Conduct regular monitoring and modify programs when required.

**Ablated Extracted Solution:**

To ensure the safety and success of the actors during the filming of the fight sequences, I will collaborate closely with the physical therapist. First, I will reach out to the fight choreographer to get detailed information about the specific movements and techniques required for each sequence. Once I have that information, I will begin designing individual exercise and conditioning plans for each actor.

Throughout the project, regular communication will be key to our success. I will keep the physical therapist updated on my progress, and we will schedule regular check-ins to monitor the actors' progress and make any necessary modifications to their programs. This collaboration and support will help us work together to ensure the success of the project.

By staying in touch and keeping each other updated on our progress, we will work together to make sure that the actors are well-prepared and that the fight sequences are executed safely and effectively. With our combined expertise, we can design effective exercise and conditioning plans that will help the actors perform the fight sequences safely and effectively.

**Original Extracted Solution:**

To assess the physical abilities and limitations of each actor, I will conduct a comprehensive physical evaluation that includes a medical history review, musculoskeletal assessment, and functional movement screening. Based on the results of these assessments, I will develop individualized exercise and conditioning plans that address each actor's specific physical needs and limitations.

For actors who have identified muscle imbalances or weaknesses, I will include exercises that target those specific areas. For actors who have limited range of motion or flexibility, I will include exercises that focus on improving mobility and flexibility. I will also include exercises that improve cardiovascular endurance and muscular endurance.

To incorporate exercises that simulate the movements and physical demands of the violent fight sequences, I will first analyze the specific physical requirements and movements to be executed in the fight sequences. Based on this analysis, I will design exercises that mimic the movements and physical demands of the fight sequences.

To develop a schedule for the actors' exercise and conditioning plans, I will first consider the availability of the actors and the filming schedule. Based on this information, I will work with the production team to create a schedule that allows the actors to complete their exercise and conditioning plans while also accommodating their filming schedule.

To provide the actors with proper warm-up and cool-down routines, I will design routines that are tailored to their specific physical needs and the demands of the fight sequences. The warm-up routine will include exercises that increase heart rate, blood flow, and body temperature, as well as dynamic stretching to prepare the muscles for the upcoming activity. The cool-down routine will include exercises that help the body recover from the physical activity and reduce the risk of injury.

To educate the actors on proper nutrition and hydration, I will provide them with information on how to fuel their bodies for optimal performance during the fight sequences. I will explain the importance of consuming a balanced diet that includes a variety of nutrient-dense foods such as lean protein, complex carbohydrates, healthy fats, and fruits and vegetables. I will also provide them with guidelines on how much water they should be drinking each day, as well as how to properly hydrate before, during, and after their workouts.

To provide the actors with guidance on how to properly rest and recover between fight sequences, I will emphasize the importance of rest and recovery in the training process. I will explain the importance of getting adequate sleep, as this is when the body repairs and recovers from the physical demands of the fight sequences. I will also provide them with information on active recovery techniques such as foam rolling, stretching, and low-intensity cardio to help reduce muscle soreness and aid in recovery.

To provide the actors with guidelines on how to maintain their physical conditioning after filming is complete, I will emphasize the importance of continuing to exercise and maintain a healthy lifestyle. I will encourage the actors to continue to follow a balanced diet and stay hydrated, as well as to continue to exercise regularly to maintain their cardiovascular and muscular endurance. I will provide them with information on how to modify their exercise and conditioning plans to fit their post-filming schedule and lifestyle, and encourage them to find activities that they enjoy and that will help them stay active.

**Review:** Assistant 1 provided a more comprehensive and detailed answer, covering various aspects of the exercise and conditioning plans, including assessments, exercise design, scheduling, warm-up and cool-down routines, nutrition, hydration, rest, recovery, and post-filming maintenance. The answer was well-structured and addressed the specific requirements of the question. Assistant 2's answer was less detailed and focused more on communication and collaboration with the physical therapist, which was not the main focus of the question. While Assistant 2's answer was relevant, it lacked the depth and detail provided by Assistant 1.

U. Additional Related Work

Model Exploration. Knowledge distillation (KD) is a popular technique for compressing complex models into smaller, more practical models that can be deployed efficiently in real-world scenarios without sacrificing performance [126]. KD aims to transfer knowledge from a larger, complex "teacher" model to a more manageable "student" model, while maintaining the accuracy and generalization capabilities of the original model. The knowledge transferred from the teacher to the student model can be categorized into three main types: Response-based, Feature-based, and Relation-based knowledge, which have been studied in various works [127, 126, 128, 129, 130, 131, 132, 133, 134, 135]. Recent works have proposed innovative methods for extracting training data from both large language models [136] diffusion models [137]. Those approaches could be seen as a means of training data distillation, in which the model training data space could be extracted. The idea is to capitalize on the models' memorization of certain samples obtained from the internet. The process involves multiple generations being created from the model, which is then sorted by specific metrics, and duplicate generations are subsequently removed. The resulting generations are then scrutinized for any matches that already exist on the web. If the generated samples match existing samples found on the internet, it can be inferred that the model has been trained on those samples. Our work presents a novel approach to the "mind exploration" of conversational agents. By enabling these agents to communicate and collaborate in solving tasks, we gain insight into their actions and behaviors within a task-solving context. Our mind exploration approach revealed several intriguing insights and challenges that are yet to be further explored by the research community.

References

Section Summary: The references section compiles a list of academic papers, preprints, books, and blog posts focused on artificial intelligence. It includes recent work on large language models like ChatGPT and GPT-4, techniques for training helpful and safe AI systems, and studies of multi-agent cooperation and communication. Earlier foundational texts on robotics, mind, and classic AI approaches are also cited.

[1] OpenAI. Introducing chatgpt. Open AI Blog, 2022.

[2] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.

[3] Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022.

[4] Sundar Pichai. An important next step on our ai journey. Google Blog, 2023.

[5] Anthropic. Introducing claude. Anthropic Blog, 2023.

[6] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022.

[7] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Transactions on Machine Learning Research, 2022. Survey Certification.

[8] Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023.

[9] Isaac Asimov. I. Robot. Narkaling Productions., 1940.

[10] Allan Dafoe, Yoram Bachrach, Gillian Hadfield, Eric Horvitz, Kate Larson, and Thore Graepel. Cooperative ai: machines must learn to find common ground. Nature, 593(7857):33–36, 2021.

[11] Allan Dafoe, Edward Hughes, Yoram Bachrach, Tantum Collins, Kevin R McKee, Joel Z Leibo, Kate Larson, and Thore Graepel. Open problems in cooperative ai. arXiv preprint arXiv:2012.08630, 2020.

[12] William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. Self-critiquing models for assisting human evaluators. arXiv preprint arXiv:2206.05802, 2022.

[13] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022.

[14] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.

[15] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.

[16] Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. arXiv preprint arXiv:2305.01210, 2023.

[17] Marvin Minsky. Society of mind. Simon and Schuster, 1988.

[18] Marvin Minsky. The emotion machine: Commonsense thinking, artificial intelligence, and the future of the human mind. Simon and Schuster, 2007.

[19] Tim Finin, Richard Fritzson, Don McKay, and Robin McEntire. Kqml as an agent communication language. In Proceedings of the third international conference on Information and knowledge management, pages 456–463, 1994.

[20] Stefan Poslad. Specifying protocols for multi-agent systems interaction. ACM Transactions on Autonomous and Adaptive Systems (TAAS), 2(4):15–es, 2007.

[21] Stuart J Russell. Artificial intelligence a modern approach. Pearson Education, Inc., 2010.

[22] Ming Tan. Multi-agent reinforcement learning: Independent versus cooperative agents. In International Conference on Machine Learning, 1997.

[23] Liviu Panait and Sean Luke. Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11:387–434, 2005.

[24] Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems, 30, 2017.

[25] Jacob Andreas. Language models as agent models, 2022.

[26] Jakob Foerster, Ioannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement learning. Advances in neural information processing systems, 29, 2016.

[27] Sainbayar Sukhbaatar, Rob Fergus, et al. Learning multiagent communication with backpropagation. Advances in neural information processing systems, 29, 2016.

[28] Igor Mordatch and Pieter Abbeel. Emergence of grounded compositional language in multi-agent populations. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.

[29] Serhii Havrylov and Ivan Titov. Emergence of language with multi-agent games: Learning to communicate with sequences of symbols. Advances in neural information processing systems, 30, 2017.

[30] Yali Du, Bo Liu, Vincent Moens, Ziqi Liu, Zhicheng Ren, Jun Wang, Xu Chen, and Haifeng Zhang. Learning correlated communication topology in multi-agent reinforcement learning. In Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, pages 456–464, 2021.

[31] Junjie Sheng, Xiangfeng Wang, Bo Jin, Junchi Yan, Wenhao Li, Tsung-Hui Chang, Jun Wang, and Hongyuan Zha. Learning structured communication for multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems, 36(2):50, 2022.

[32] Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assisted question-answering with human feedback, 2021.

[33] Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung, Moya Chen, Kushal Arora, Joshua Lane, et al. Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208.03188, 2022.

[34] Amelia Glaese, Nat McAleese, Maja Trebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, and Geoffrey Irving. Improving alignment of dialogue agents via targeted human judgements, 2022.

[35] Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. A simple language model for task-oriented dialogue. Advances in Neural Information Processing Systems, 33:20179–20191, 2020.

[36] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Kuang-Huei Lee, Sergey Levine, Yao Lu, Linda Luu, Carolina Parada, Peter Pastor, Jornell Quiambao, Kanishka Rao, Jarek Rettinghouse, Diego Reyes, Pierre Sermanet, Nicolas Sievers, Clayton Tan, Alexander Toshev, Vincent Vanhoucke, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Mengyuan Yan, and Andy Zeng. Do as i can, not as i say: Grounding language in robotic affordances, 2022.

[37] Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2022.

[38] Josh Abramson, Arun Ahuja, Iain Barr, Arthur Brussee, Federico Carnevale, Mary Cassin, Rachita Chhaparia, Stephen Clark, Bogdan Damoc, Andrew Dudzik, Petko Georgiev, Aurelia Guy, Tim Harley, Felix Hill, Alden Hung, Zachary Kenton, Jessica Landon, Timothy Lillicrap, Kory Mathewson, Soňa Mokrá, Alistair Muldal, Adam Santoro, Nikolay Savinov, Vikrant Varma, Greg Wayne, Duncan Williams, Nathaniel Wong, Chen Yan, and Rui Zhu. Imitating interactive intelligence, 2020.

[39] Siddharth Karamcheti, Megha Srivastava, Percy Liang, and Dorsa Sadigh. Lila: Language-informed latent actions. In CoRL, pages 1379–1390, 2021.

[40] Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. arXiv preprint arXiv:2201.07207, 2022.

[41] Shuang Li, Xavier Puig, Chris Paxton, Yilun Du, Clinton Wang, Linxi Fan, Tao Chen, De-An Huang, Ekin Akyürek, Anima Anandkumar, Jacob Andreas, Igor Mordatch, Antonio Torralba, and Yuke Zhu. Pre-trained language models for interactive decision-making, 2022.

[42] Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas. A generalist agent, 2022.

[43] Gerald Tesauro et al. Temporal difference learning and td-gammon. Communications of the ACM, 38(3):58–68, 1995.

[44] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017.

[45] Dylan Hadfield-Menell, Stuart J Russell, Pieter Abbeel, and Anca Dragan. Cooperative inverse reinforcement learning. Advances in neural information processing systems, 29, 2016.

[46] Nolan Bard, Jakob N Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, et al. The hanabi challenge: A new frontier for ai research. Artificial Intelligence, 280:103216, 2020.

[47] Deyao Zhu, Jun Chen, Kilichbek Haydarov, Xiaoqian Shen, Wenxuan Zhang, and Mohamed Elhoseiny. Chatgpt asks, blip-2 answers: Automatic questioning towards enriched visual descriptions, 2023.

[48] Yat Long Lo, Christian Schroeder de Witt, Samuel Sokota, Jakob Nicolaus Foerster, and Shimon Whiteson. Cheap talk discovery and utilization in multi-agent reinforcement learning. In The Eleventh International Conference on Learning Representations, 2023.

[49] Caroline Claus and Craig Boutilier. The dynamics of reinforcement learning in cooperative multiagent systems. In AAAI/IAAI, 1998.

[50] Michael Wooldridge. An introduction to multiagent systems. John wiley & sons, 2009.

[51] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.

[52] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022.

[53] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022.

[54] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022.

[55] Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.

[56] Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021.

[57] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022.

[58] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916, 2022.

[59] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.

[60] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625, 2022.

[61] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In ICLR, 2023.

[62] Shima Imani, Liang Du, and Harsh Shrivastava. Mathprompter: Mathematical reasoning using large language models. arXiv preprint arXiv:2303.05398, 2023.

[63] Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. In ICLR, 2023.

[64] Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. Complexity-based prompting for multi-step reasoning. arXiv preprint arXiv:2210.00720, 2022.

[65] Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. Language models are multilingual chain-of-thought reasoners. In ICLR, 2023.

[66] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021.

[67] Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. 2022.

[68] Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. In ICLR, 2023.

[69] Namgyu Ho, Laura Schmid, and Se-Young Yun. Large language models are reasoning teachers. arXiv preprint arXiv:2212.10071, 2022.

[70] Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923, 2023.

[71] Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366, 2023.

[72] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning, 2022.

[73] Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning, 2022.

[74] Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. Show your work: Scratchpads for intermediate computation with language models, 2021.

[75] Abishek Sridhar, Robert Lo, Frank F. Xu, Hao Zhu, and Shuyan Zhou. Hierarchical prompting assists large language model on web navigation. In ArXiv, preprint.

[76] Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, et al. Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375, 2022.

[77] Jianfeng Gao, Michel Galley, and Lihong Li. Neural approaches to conversational ai. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 1371–1374, 2018.

[78] Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks. In EMNLP, 2022.

[79] Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688, 2023.

[80] Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. Cross-task generalization via natural language crowdsourcing instructions. In ACL, 2022.

[81] Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Dániel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, et al. Opt-iml: Scaling language model instruction meta learning through the lens of generalization. arXiv preprint arXiv:2212.12017, 2022.

[82] Timo Schick and Hinrich Schütze. Generating datasets with pretrained language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6943–6951, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.

[83] Kenton Lee, Kelvin Guu, Luheng He, Tim Dozat, and Hyung Won Chung. Neural data augmentation via example extrapolation. arXiv preprint arXiv:2102.01335, 2021.

[84] Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. WANLI: Worker and AI collaboration for natural language inference dataset creation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6826–6847, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics.

[85] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023.

[86] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022.

[87] Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. Unnatural instructions: Tuning language models with (almost) no human labor. arXiv preprint arXiv:2212.09689, 2022.

[88] Zekun Li, Wenhu Chen, Shiyang Li, Hong Wang, Jing Qian, and Xifeng Yan. Controllable dialogue simulation with in-context learning. arXiv preprint arXiv:2210.04185, 2022.

[89] Hyunwoo Kim, Jack Hessel, Liwei Jiang, Ximing Lu, Youngjae Yu, Pei Zhou, Ronan Le Bras, Malihe Alikhani, Gunhee Kim, Maarten Sap, et al. Soda: Million-scale dialogue distillation with social commonsense contextualization. arXiv preprint arXiv:2212.10465, 2022.

[90] Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Seokhwan Kim, Andy Rosenbaum, Yang Liu, Zhou Yu, and Dilek Hakkani-Tur. Places: Prompting language models for social conversation synthesis. arXiv preprint arXiv:2302.03269, 2023.

[91] Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. Generating training data with language models: Towards zero-shot language understanding. In Advances in Neural Information Processing Systems, 2022.

[92] Maximillian Chen, Alexandros Papangelis, Chenyang Tao, Andy Rosenbaum, Seokhwan Kim, Yang Liu, Zhou Yu, and Dilek Hakkani-Tur. Weakly supervised data augmentation through prompting for dialogue understanding. NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research, 2022.

[93] Gaurav Sahu, Pau Rodriguez, Issam H Laradji, Parmida Atighehchian, David Vazquez, and Dzmitry Bahdanau. Data augmentation for intent classification with off-the-shelf large language models. ACL, 2022.

[94] Yekyung Kim, Seohyeong Jeong, and Kyunghyun Cho. Linda: Unsupervised learning to interpolate in natural language processing. arXiv preprint arXiv:2112.13969, 2021.

[95] Andy Rosenbaum, Saleh Soltan, Wael Hamza, Yannick Versley, and Markus Boese. Linguist: Language model instruction tuning to generate annotated utterances for intent classification and slot tagging. arXiv preprint arXiv:2209.09900, 2022.

[96] Houyu Zhang, Zhenghao Liu, Chenyan Xiong, and Zhiyuan Liu. Grounded conversation generation as guided traverses in commonsense knowledge graphs. In ACL, 2020.

[97] Jonáš Kulhánek, Vojtěch Hudeček, Tomáš Nekvinda, and Ondřej Dušek. Augpt: Auxiliary tasks and data augmentation for end-to-end dialogue with pre-trained language models. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI, pages 198–210, 2021.

[98] Rongsheng Zhang, Yinhe Zheng, Jianzhi Shao, Xiao-Xi Mao, Yadong Xi, and Minlie Huang. Dialogue distillation: Open-domain dialogue augmentation using unpaired data. ArXiv, abs/2009.09427, 2020.

[99] Alexandros Papangelis, Karthik Gopalakrishnan, Aishwarya Padmakumar, Seokhwan Kim, Gokhan Tur, and Dilek Z. Hakkani-Tür. Generative conversational networks. In SIGDIAL, 2021.

[100] Sanghwan Bae, Donghyun Kwak, Sungdong Kim, Donghoon Ham, Soyoung Kang, Sang-Woo Lee, and Woomyoung Park. Building a role specified open-domain dialogue system leveraging large-scale language models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2128–2150, Seattle, United States, July 2022. Association for Computational Linguistics.

[101] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.

[102] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021.

[103] Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, 2020.

[104] Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. How can we know what language models know? Transactions of the Association for Computational Linguistics, 8:423–438, 2020.

[105] Laria Reynolds and Kyle McDonell. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–7, 2021.

[106] Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations, 2023.

[107] Jacob Andreas and Dan Klein. Alignment-based compositional semantics for instruction following. arXiv preprint arXiv:1508.06491, 2015.

[108] Dylan Hadfield-Menell, McKane Andrus, and Gillian Hadfield. Legible normativity for ai alignment: The value of silly rules. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 115–121, 2019.

[109] Jonathan Stray. Aligning ai optimization to community well-being. International Journal of Community Well-Being, 3:443 – 463, 2020.

[110] Iason Gabriel. Artificial intelligence, values, and alignment. Minds and Machines, 30:411 – 437, 2020.

[111] Dylan Hadfield-Menell. The principal-agent alignment problem in artificial intelligence. Ph. D. dissertation, 2021.

[112] Michael J. Matthews, Samuel H. Matthews, and Thomas K. Kelemen. The alignment problem: Machine learning and human values. Personnel Psychology, 2022.

[113] Zachary Kenton, Tom Everitt, Laura Weidinger, Iason Gabriel, Vladimir Mikulik, and Geoffrey Irving. Alignment of language agents. arXiv preprint arXiv:2103.14659, 2021.

[114] Alex Tamkin, Miles Brundage, Jack Clark, and Deep Ganguli. Understanding the capabilities, limitations, and societal impact of large language models. arXiv preprint arXiv:2102.02503, 2021.

[115] Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, Nan Rosemary Ke, Genevieve Fried, Ryan Lowe, and Joelle Pineau. Ethical challenges in data-driven dialogue systems. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 123–129, 2018.

[116] Josh A Goldstein, Girish Sastry, Micah Musser, Renee DiResta, Matthew Gentzel, and Katerina Sedova. Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246, 2023.

[117] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.

[118] Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023.

[119] Harrison Chase. Langchain. 2022.

[120] OpenAI. Chatgpt plugins. OpenAI blog, 2023.

[121] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online, October 2020. Association for Computational Linguistics.

[122] Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, et al. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854, 2023.

[123] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models, 2021.

[124] David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489, 2016.

[125] Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, September 2021.

[126] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015.

[127] Jimmy Ba and Rich Caruana. Do deep nets really need to be deep? Advances in neural information processing systems, 27, 2014.

[128] Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550, 2014.

[129] Jangho Kim, Seonguk Park, and Nojun Kwak. Paraphrasing complex network: Network compression via factor transfer. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.

[130] Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928, 2016.

[131] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885–1894. PMLR, 2017.

[132] Byeongho Heo, Minsik Lee, Sangdoo Yun, and Jin Young Choi. Knowledge transfer via distillation of activation boundaries formed by hidden neurons. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3779–3787, 2019.

[133] Defang Chen, Jian-Ping Mei, Yuan Zhang, Can Wang, Zhe Wang, Yan Feng, and Chun Chen. Cross-layer distillation with semantic calibration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 7028–7036, 2021.

[134] Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, and Qun Liu. Alp-kd: Attention-based layer projection for knowledge distillation. In Proceedings of the AAAI Conference on artificial intelligence, volume 35, pages 13657–13665, 2021.

[135] Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3967–3976, 2019.

[136] N Carlini, F Tramer, E Wallace, M Jagielski, A Herbert-Voss, K Lee, A Roberts, T Brown, D Song, Ú Erlingsson, et al. Extracting training data from large language models. arxiv. Preprint posted online December, 14, 2020.

[137] Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. arXiv preprint arXiv:2301.13188, 2023.