WizardLM: Empowering Large Pre-Trained Language Models to Follow Complex Instructions
Can Xu$^{1*}$
$^{1}$Microsoft
Qingfeng Sun$^{1*}$
$^{1}$Microsoft
Kai Zheng$^{1*}$
$^{1}$Microsoft
Xiubo Geng$^{1}$
$^{1}$Microsoft
Pu Zhao$^{1}$
$^{1}$Microsoft
Jiazhan Feng$^{2\dagger}$
$^{2}$Peking University
Chongyang Tao$^{1}$
$^{1}$Microsoft
Qingwei Lin$^{1}$
$^{1}$Microsoft
Daxin Jiang$^{1\ddagger}$
$^{1}$Microsoft
$^{*}$ Equal contribution.
$^{\dagger}$ Work done during the internship at Microsoft.
$^{\ddagger}$ Corresponding author.
Abstract
Training large language models (LLMs) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Both automatic and human evaluations consistently indicate that WizardLM outperforms baselines such as Alpaca (trained from Self-Instruct) and Vicuna (trained from human-created instructions). The experimental results demonstrate that the quality of instruction-following dataset crafted by Evol-Instruct can significantly improve the performance of LLMs.
Executive Summary: Training large language models to reliably handle complex, real-world instructions has traditionally required massive human annotation efforts. However, human curation is expensive, slow, and tends to produce predominantly simple or moderate tasks, leaving models poorly equipped for high-difficulty challenges. The article addresses this operational bottleneck by presenting an automated pipeline, called Evol-Instruct, which uses language models instead of human annotators to generate diverse, highly complex training instructions at scale.
The main objective of the article is to demonstrate that automatically evolving simple instructions into more complex variants substantially improves the performance of fine-tuned language models across open-domain benchmarks. The approach utilizes an evolutionary loop that increases instruction difficulty through depth-based modifications (adding constraints, deepening concepts, increasing reasoning steps, and complicating inputs) and breadth-based mutations (generating rare, topically diverse prompts), followed by automated quality filtering. The authors used this method to expand a 52,000-sample dataset into 250,000 instructions, subsequently fine-tuning open-source foundation models into a new model family called WizardLM.
The findings show that WizardLM significantly outperforms leading open-source models trained on human or self-instructed datasets, such as Vicuna and Alpaca. Across standard academic, coding, and mathematical benchmarks, WizardLM-13b achieved an overall average score of 58.96 compared to Alpaca-13b's 43.44 and Vicuna-13b's 54.60. Most notably, in math reasoning on the GSM8k benchmark, WizardLM reached 37.15% accuracy, outperforming Vicuna's 24.34% and more than quadrupling Alpaca's 8.35%. In coding evaluation (HumanEval), WizardLM scored 24.0% versus Vicuna's 12.5% and Alpaca's 9.2%. Both GPT-4 assessments and blind human pairwise evaluations confirmed that WizardLM consistently delivers higher quality, more accurate, and more relevant responses. Furthermore, ablation experiments showed that performance gains directly correlated with increasing instruction complexity across evolution cycles.
These results carry major strategic implications for organizations deploying artificial intelligence. High-performing models can be developed with minimal manual annotation, dramatically lowering training costs and shortening development cycles. The findings prove that instruction complexity and diversity, rather than raw data volume or human labeling alone, drive instruction-following capabilities. The framework also demonstrates flexibility across model architectures and sizes, scaling effectively from 7-billion to 70-billion parameter models and working with open-source evolution engines as well as proprietary ones.
Decision-makers aiming to improve internal model performance should adopt automated instruction evolution as a core data-generation strategy. Teams can lower compute and API costs by using strong open-source models (such as LLaMA-2) to drive the evolution process instead of proprietary APIs. However, leadership should note limitations surrounding automated evaluation reliability and potential boundary conditions across untested industry domains. Organizations should conduct targeted pilot tests within their specific domain tasks to confirm that evolved synthetic data aligns with specialized operational requirements before full production deployment.
1. Introduction
Section Summary: Large language models excel at generating fluent text but often fail to follow complex or varied user instructions effectively. Human-created instruction datasets are costly to produce, tend to skew toward simple tasks, and lack sufficient diversity or difficulty. This paper presents Evol-Instruct, an automated technique that uses language models themselves to iteratively evolve simpler instructions into more challenging and diverse ones, enabling the fine-tuned WizardLM model to outperform prior systems like Vicuna on a range of benchmarks.
Large-scale language models (LLMs) have become the go-to approach for numerous natural language processing tasks ([1, 2, 3]). LLMs are trained on large volumes of text data to predict the subsequent tokens, enabling them to generate coherent and fluent text in response to various inputs. However, these models often struggle to follow instructions or goals specified by users, which limits their usefulness and applicability in real-world scenarios.
The NLP community has recently witnessed many endeavors to train LLMs to follow instructions better and be more helpful ([4, 5, 6, 7]). Initial attempts ([8, 9, 10, 11, 12]) to train instruction-following language models are based on a collection of various NLP tasks, with a small amount of hand-written instructions. These closed-domain instructions suffer from two main drawbacks: first, all the samples in an NLP dataset share only a few common instructions, severely limiting their diversity; second, the instructions usually only ask for one task. But in real life, human instructions often have multiple and varied task demands. By using open-domain instruction data generated by real human users, OpenAI's LLMs (e.g., InstructGPT ([2]) and ChatGPT^1) have achieved great success. These open-domain instructions can fully unleash the unlimited potential of LLMs ([13, 14, 15, 16]) and enable them to perform more complex and diverse tasks. However, using humans to create open-domain instruction datasets like OpenAI did will encounter the following challenges. The whole annotating process is extremely expensive and time-consuming ([17, 18, 19, 20]). On the other hand, the difficulty level distribution of human-created instructions is skewed towards being easy or moderate, with fewer difficult ones (according to the difficulty statistics of ShareGPT ([21]) from Figure 5a). Human annotators are prone to fatigue and cannot sustain high-intensity work to produce a sufficient proportion of high-difficulty instructions ([22, 23, 24, 25]). Based on these issues, developing an automatic method that can mass-produce open-domain instructions (especially the more difficult ones) at a relatively low cost becomes the key to further advancing instruction-tuned language models ([26, 27, 28, 29]).
In this work, we introduce Evol-Instruct, a novel method using LLMs instead of humans to automatically mass-produce open-domain instructions of various difficulty levels, to improve the performance of LLMs. Figure 1 shows the running examples of Evol-Instruct. Starting from a simple initial instruction “1+1=?”, our method randomly selects In-depth Evolving (blue direction line) or In-breadth Evolving (red direction line) to upgrade the simple instruction to a more complex one or create a new one (to increase diversity). The In-depth Evolving includes five types of operations: add constraints, deepening, concretizing, increase reasoning steps, and complicate input. The In-breadth Evolving is mutation, i.e., generating a completely new instruction based on the given instruction. These six operations are implemented by prompting an LLM with specific prompts. Since the evolved instructions are generated from LLMs, sometimes the evolving will fail. We adopt an instruction eliminator to filter the failed instructions, which is called Elimination Evolving. We repeat this evolutionary process for several rounds to obtain enough instruction data containing various complexities.
In order to verify the effectiveness of Evol-Instruct and whether the instructions it creates for fine-tuning surpass those created by humans, we evolve the instructions from Aplaca ([30]) data (created by machine), fine-tune the LLaMA ([3]) model, and comprehensively compare the fine-tuned model WizardLM with Vicuna ([21]) trained on ShareGPT (instructions are created by human). Alpaca data has a total of $52k$ samples and is generated using self-instruct ([31]) from only $175$ human-created seed instructions. We choose Alpaca data as the initial data for evolution, which can ensure that the training instructions of WizardLM have almost no direct human participation in annotations. We execute four epochs of evolution using OpenAI ChatGPT API[^2] and finally obtain $250k$ instructions. To ensure a fair comparison with Vicuna's $70k$ real user data, we sampled $70k$ from the full $250k$ data and fine-tuned the LLaMA 13B model. Because the original Alpaca data only has $52k$ samples, we used its self-instruct method to generate an additional $18k$ data, and retrained the LLaMA 13B model with its code^3 to get Alpaca 13B as our baseline. Due to the low proportion of difficult instructions in the previous instruction-following test dataset, we manually created a new difficulty-balanced test dataset, named WizardEval. We evaluate Alpaca, Vicuna, ChatGPT, and WizardLM on a wide range of LLM benchmarks (covering reasoning, code, mathematics, general conversation, etc.). Our main findings are as follows:
- We introduce Evol-Instruct, a novel approach that enhances the performance of the open-source LLMs by a large margin via automatically mass-producing open-domain instructions of various topics and difficulty levels.
- We develop WizardLM model, which significantly surpasses typical open-source LLMs such as Alpaca and Vicuna in a series of benchmarks. Notably, WizardLM outperforms baselines by a substantial margin in terms of code, math, GPT-4 and human evaluations.
- We have undertaken a preliminary investigation that underscores the importance of instruction complexity in attaining outstanding performance in supervised fine-tuning large pre-trained language models.
[^2]: gpt-3.5-turbo from https://oai.azure.com/portal
2. Related Work
Section Summary: Early work on instruction tuning focused on training language models across many standard NLP tasks, such as question answering or summarization, by converting them into a shared text format and scaling up the number of tasks to thousands. These closed-domain efforts improved performance on new tasks but often failed to handle the varied, open-ended requests people make in real life. More recent open-domain approaches, including InstructGPT, Alpaca, and Vicuna, instead use diverse human-written or AI-generated instructions to better match everyday use cases, with the present work advancing this line through controlled increases in instruction difficulty.
Closed domain instruction tuning
Early instruction-following training work ([9, 32]) concerns cross task generalization in LMs, where LMs are fine-tuned on a broad range of public NLP datasets and evaluated on a different set of NLP tasks. T5 [33] made the earliest attempt by training natural language processing (NLP) tasks such as question answering, document summarization, and sentiment classification together using a unified text-to-text format. Works such as FLAN [9], ExT5 [8], T0 [11], and KnowDA [34] increased the number of NLP tasks to around one hundred, with several instructions carefully designed for each task [35, 36, 37, 38]. Furthermore, works such as ZeroPrompt [10] and FLAN-T5 [12] raised the number of tasks to the thousands. These studies consistently show that fine-tuning LMs with diverse NLP task instructions enhances their performance on new tasks. However, LLMs trained with these closed-form instructions (i.e., instructions are often only for a single NLP task, and the input data form is simple) tend to fail in real-world user scenarios.
Open domain instruction tuning
Our work belongs to this research line. OpenAI has hired many annotators and written many instructions with corresponding correct responses. These human-created instructions have diverse forms and rich task types. Based on this dataset, OpenAI trained GPT-3 [1] into InstructGPT [2], which can process a variety of real user instructions and led to the success of ChatGPT. Orca [39] learns not only the superficial response text from LLMs, but also captures complex reasoning process signals. Since these outstanding works from OpenAI were not open-sourced, Alpaca [30] and Vicuna [21] subsequently actively explored open-domain instruction fine-tuning based on the open-source LLM LLaMA [3]. Alpaca used a dataset of 50k instructions generated from a limited (e.g., 175 samples) seed set of manually-written instructions. Our work is different from InstructGPT and Vicuna in that we use AI-generated data for instruction fine-tuning. Unlike Alpaca’s self-instruct [31] generation method, Evol-Instruct can control the difficulty and complexity level of the generated instructions.
3. Approach
Section Summary: The Evol-Instruct approach begins with an initial set of instructions and responses and iteratively upgrades them by prompting a large language model to create either more complex versions of existing instructions or entirely new ones that increase diversity. Complexity is raised through targeted changes such as adding constraints, deepening reasoning, or complicating inputs, while breadth is expanded by generating fresh instructions on new topics; an eliminator step discards any evolutions that fail to meet quality rules. After each round the model produces fresh responses for the updated instructions, yielding a richer dataset after several iterations.
In this section, we elaborate on the details of the proposed Evol-Instruct. As illustrated in Figure 2, the pipeline mainly contains two components: Instruction Evolver and Instruction Eliminator. The details of these compoents will be presented in Section 3.2 and instruction fine-tuning method will be described in Section 3.3.
3.1 Definition of Instruction Data Evolution
We start the evolution from a given initial instruction dataset $D^{(0)}={(I_k^{(0)},R_k^{(0)})}_{1\leq k\leq N}$, where $I_k^{(0)}$ is the $k$-th instruction in $D^{(0)}$, $R_k^{(0)}$ is the corresponding response for the $k$-th instruction, and $N$ is the number of samples in $D^{(0)}$. In each evolution, we upgrade all the $I^{(t)}$ in $D^{(t)}$ to $I^{(t+1)}$ by prompting a LLM with Evol-Instruct prompt, and then use the LLM to generate corresponding responses $R^{t+1}$ for the newly evolved $I^{t+1}$. Thus, we obtain an evolved instruction dataset $D^{t+1}$. By iteratively performing $M$ evolutions, we can sequentially obtain $M$ evolution datasets $[D^{(1)} \cdots D^{(M)}]$. Our work focuses on open-domain instruction data, where instructions have varying inputs and tasks without a clear distinction between the instruction part and the input.
3.2 Automatic Instruction Data Evolution
Our pipeline for instruction evolution consists of three steps: 1) instruction evolving, 2) response generation, and 3) elimination evolving, i.e., filtering intructions that fails to evolve.
Instruction Evolution.
We found that LLMs can make given instructions more complex and difficult using specific prompts. Additionally, they can generate entirely new instructions that are equally complex but completely different. Using this discovery, we can iteratively evolve an initial instruction dataset, improving difficulty level and expanding its richness and diversity. We initiate the instruction pool with the given initial instruction dataset $D^{(0)}$. In each evolution epoch, upgraded instructions from the previous epoch are taken out from the pool. Then we leverage the instruction evolver to evolve each fetched instruction, and the instruction eliminator to check whether the evolution fails. Successful evolved instructions are added to the pool, while unsuccessful ones are placed back as they are, with the hope of upgrading them successfully in the next evolution epoch.
Instruction Evolver.
The Instruction Evolver is an LLM that uses Evol-Instruct prompts to evolve instructions, with two types: in-depth evolving and in-breadth evolving.
In-Depth Evolving enhances instructions by making them more complex and difficult through five types of prompts: add constraints, deepening, concretizing, increased reasoning steps, and complicating input. The core part of In-Depth Evolving's prompt is "Your objective is to rewrite a given prompt into a more complex version to make those famous AI systems (e.g., ChatGPT and GPT4 ([40])) a bit harder to handle. But the rewritten prompt must be reasonable, understood, and responded to by humans". We require the LLM to create challenging instructions that are reasonable and not arbitrarily imagined by AI. A gradual difficulty increase is necessary to avoid filling the instruction set with extremely complex instructions, which would harm the generalization performance of trained models. To control difficulty increase, we make each evolution "a bit harder" and restrict adding a maximum of 10 to 20 words. Among the five mentioned evolving, all can be implemented without any in-context examples except for complicating input. We show the prompt of add constraints as follows (the prompts of deepening, concretizing and increased reasoning steps will be detailed in the Appendix A-C).
I want you act as a Prompt Rewriter.
Your objective is to rewrite a given prompt into a more complex version to make those famous AI systems (e.g., ChatGPT and GPT4) a bit harder to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
Your rewriting cannot omit the non-text parts such as the table and code in \#Given Prompt\#:. Also, please do not omit the input in \#Given Prompt\#.
You SHOULD complicate the given prompt using the following method:
**Please add one more constraints/requirements into \#Given Prompt\#**
You should try your best not to make the \#Rewritten Prompt\# become verbose, \#Rewritten Prompt\# can only add 10 to 20 words into \#Given Prompt\#. '\#Given Prompt\#', '\#Rewritten Prompt\#', 'given prompt' and 'rewritten prompt' are not allowed to appear in \#Rewritten Prompt\#
**\#Given Prompt\#:**
Here is instruction.
**\#Rewritten Prompt\#:**
For complicating input, we will use in-context demonstration. Due to the lengthy demonstrations, we will provide a brief template below, with the full prompt detailed in the Appendix D.
I want you act as a Prompt Rewriter.
Your objective is to rewrite a given prompt into a more complex version to make those famous AI systems (e.g., ChatGPT and GPT4) a bit harder to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add **[XML data]** format data as input data in [Rewritten Prompt]
**\#Given Prompt\#:**
Here is instruction of Example 1.
**\#Rewritten Prompt\#:**
Here is rewritten instruction of Example 1.
**... N -1 Examples ...**
You must add **[\#Given Dataformat\#]** format data as input data in [Rewritten Prompt]
**\#Given Prompt\#:**
Here is instruction of Example N.
**\#Rewritten Prompt\#:**
In-Breadth Evolving aims to enhance topic coverage, skill coverage, and overall dataset diversity. Open-domain instruction finetune datasets (e.g., Alpaca, ShareGPT, etc.) are typically small in scale, lacking topic and skill diversity. To solve this problem, we designed a prompt to generate a completely new instruction based on the given instruction, requiring the new instruction to be more long-tailed. Our In-Breadth Evolving prompt is as follows:
I want you act as a Prompt Creator.
Your goal is to draw inspiration from the \#Given Prompt\# to create a brand new prompt.
This new prompt should belong to the same domain as the \#Given Prompt\# but be even more rare.
The LENGTH and difficulty level of the \#Created Prompt\# should be similar to that of the \#Given Prompt\#.
The \#Created Prompt\# must be reasonable and must be understood and responded by humans.
'\#Given Prompt\#', '\#Created Prompt\#', 'given prompt' and 'created prompt' are not allowed to appear in \#Created Prompt\#.
**\#Given Prompt\#:**
Here is instruction.
**\#Created Prompt\#:**
Response Generation.
We use the same LLM as for evolving to generate the corresponding responses for the evolved instructions. The generation prompt is "Here is instruction.", we feed it into the request of the ChatGPT-3.5 and parse the returned text body as the response.
Elimination Evolving.
We classify the following four situations as instruction evolution failure:
- The evolved instruction does not provide any information gain compared to the original one. We use ChatGPT to make this determination, details please refer to Appendix G.
- The evolved instruction makes it difficult for the LLM to generate a response. We found that when the generated response contains "sorry" and is relatively short in length (i.e., less than 80 words), it often indicates that the LLM struggles to respond to the evolved instruction. So we can use this rule to make a judgment.
- The response generated by the LLM only contains punctuation and stop words.
- The evolved instruction obviously copies some words from the evolving prompt, such as "given prompt", "rewritten prompt", "#Rewritten Prompt#", etc.
3.3 Finetuning the LLM on the Evolved Instructions
Once all evolutions are done, we will merge the initial instruction dataset with evolved instruction data from all epochs and randomly shuffle the samples to create the fine-tuning dataset. This processing ensures even distribution of instructions of varying difficulty levels in the dataset, maximizing model fine-tuning smoothness. To prove that the performance gain is not due to the increased amount of data after merging, but from our proposed novel method Evol-Instruct, we randomly sample an equal amount of data the same with training baselines (e.g., Vicuna) from this merged data as our final fine-tuning data. We choose Vicuna's prompt as the prompt for our fine-tuning, the specific format is "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: Hi ASSISTANT: Hello. USER: Who are you? ASSISTANT: I am WizardLM ......."
4. Experiment
Section Summary: The experiments evaluate WizardLM, a model fine-tuned from LLaMA 13B on 70,000 evolved instructions, against baselines including an enhanced Alpaca, Vicuna, and ChatGPT through both automated benchmarks and human review. Automatic tests covered academic knowledge, coding, math reasoning, and GPT-4 judged tasks, where WizardLM showed clear gains over other open-source models of similar size, especially on harder problems. Human evaluators performed blind comparisons on a new test set of 218 real-world instructions and generally rated WizardLM higher than the baselines for relevance and quality.
We assess WizardLM, Alpaca, Vicuna, and ChatGPT using both automatic and human evaluations.
4.1 Baselines
(1) ChatGPT is an AI chatbot developed by OpenAI that can interact with users in a natural and engaging way. It is built on top of LLMs like GPT-3.5 and GPT-4, trained on vast internet text data.
(2) Alpaca is an open-source instruction-following model developed by Stanford University. For a fair comparison, we expanded the number of instructions from $52k$ to $70k$ using self-Instruct adopted by Alpaca and replaced the original Davici-003 responses with ChatGPT's responses. We re-trained Alpaca 13B from LLaMA 13B([3]) based on this new Alpaca data.
(3) Vicuna is based on LLaMA and fine-tuned on $70k$ user-shared conversations collected from ShareGPT. It is one of the most advanced and versatile open instruction-following models available today. We use the 13B-v1.1 model from FastChat^4.
(4) Open-source models trained from Llama 13B, including Baize ([41]), CAMEL ([42]), and Tulu ([43])
4.2 Experiment detail
To construct the dataset, we initialize it with the $52k$ instruction dataset of Alpaca and iteratively perform $M$ evolutions, where $M=4$. For each instruction in each round of evolution, we randomly select one evolving prompt from total six prompts (i.e., five from in-depth evolving and one from in-breadth evolving) with equal probability. We execute above process using Azure OpenAI ChatGPT API[^5]. Then, we leverage ChatGPT to generate responses. Finally, we obtain $250k$ instructions. For a fair comparison, we randomly sample $70k$ data from $250k$ data with equal probability as the final training data for WizardLM, the same as the amount of training data for Vicuna. We use a temperature of 1 to generate response and set the maximum number of tokens for generation to $2048$. Additionally, we set the frequency penalty to zero and top-p to $0.9$. Totally, we request the API $52k$ $\times$ $4$ $\times$ $3 = 624k$ times to construct the full dataset. We use pre-trained LLaMA 13B ([3]) to initialize our model. We adopt Adam optimizer with an initial learning rate of $2 \times 10^{-5}$, a maximum number of tokens $2048$, and the batch size is $4$ for each GPU. We train our model on 8 V100 GPUs with Deepspeed Zero-3 for $140$ hours on $3$ epochs. For inference, we use greedy search for WizardLM and baseline models, and set the maximum generation length to $2048$.
[^5]: gpt-3.5-turbo from https://oai.azure.com/portal
4.3 Automatic Evaluation
To present a comprehensive overview of the performance of our WizardLM, we conduct a comparative comparison between our model and the established baselines across a range of LLM benchmarks.
OpenLLM Leaderboard of HuggingFace ([44]) includes MMLU ([45]), ARC ([46]), HellaSwag ([47]), and TruthfulQA ([48]). MMLU consists of a range of multiple-choice academic questions. ARC is a set of grade-school science questions. HellaSwag is a test of commonsense inference. TruthfulQA measures a model’s propensity to reproduce falsehoods. We adopt the evaluate code ([49]) from OpenLLM.
Code Generation We use the extensively utilized HumanEval ([50]) benchmark consisting of 164 coding problems to evaluate LLMs’ code writing capabilities at the function level by reporting the pass@1 metric.
Math Reasoning We use GSM8k ([51]) to evaluate mathematical abilities of models, GSM8k contains 1319 grade school math test data. We adopt 4-shot testing and report pass@1.
GPT-4 Evaluation We employ two widely recognized GPT-4 evaluation benchmarks, including AlpacaEval ([52]) and MT-Bench ([53]). We also use GPT-4 to judge LLMs on our following proposed WizardEval.
::: {caption="Table 1: Performance comparison of ChatGPT-3.5, open-source baselines, and WizardLM-13b."}
:::
As shown in Figure 3 and Table 1, compared with other same-sized open-sourced models, WizardLM has a remarkable performance advantage in most benchmarks. Especially in math, code, and GPT-4 evaluations, it achieves significant improvement over Alpaca, Vicuna, Baize, CAMEL, and Tulu.
4.4 Human evaluation
To evaluate WizardLM, we conduct human evaluation on our crafted testbed WizardEval, which includes 218 real-world human instructions from diverse sources such as online opensource projects (Github, ShareGPT), platforms (Twitter), and forums (Reddit, Discord). The data contains 29 skills and domains that represent the main requirements of humanity, such as Coding Generation, Math, Reasoning, Complex Formats, Writing, Extensive Disciplines, and so on. As shown in Figure 4a and Appandix Figure 6, we also analyse the difficulty and skills distribution of WizardEval respectively, which indicate that WizadEval is able to handle the evaluation on more complex and demanding scenarios than Self-Instruct and Vicuna testset.
We perform a blind pairwise comparison between WizardLM-13b and baselines. Specifically, we recruit 10 well-educated annotators. To each annotator, four responses from Alpaca-13b, Vicuna-13b, WizardLM and ChatGPT are presented, which are randomly shuffled to hide their sources. The annotators then judge which response is better following criterion (for detailed definition, please refer to Appendix K): (1) Relevance, (2) Knowledgeable, (3) Reasoning, (4) Calculation, and (5) Accuracy.
Then they should rank the four responses from 1 to 5 (1 means best), and allowing equal scores for comparable instances. To estimate the win rate, we compare the frequency of win, lost, and tie between each pair of models. As shown in Figure 4 (b). WizardLM achieved significantly better results than Alpaca and Vicuna, which demonstrates the effectiveness of Evol-Instruct method. All of the Kappa scores are greater than 0.6, which indicates the good agreement among the annotators.
4.5 Ablation Study
Training with different data (seed, size), evol model, and base model size. In order to study the impact of different data seeds, Evol models, scale of evolved dataset, pre-trained models on our proposed method, we conducted the following experiments: a) Using 70k ShareGPT as the seed data to obtain WizardLM-13b (ShareGPT Seed); b) Using LlaMA-2-70B-Chat to replace ChatGPT as the evolutionary execution model to obtain WizardLM-13b (LlaMA-2-70B-Chat Evol); c) We train on larger size pre-trained models Llama-1 65B and Llama-2 70B to obtain WizardLM-65b and WizardLM-70b respectively; d) Using the complete 250k evolved data to obtain WizardLM-13b (250K); e) Using a completely different base from the LlaMA family, Mistral-7B, to obtain WizardLM-7b (Mistral); f) In order to compare more diverse instruction data, we choose Supernatural Instructions([54]) and randomly extract 70k data to train llama-13b to obtain LlaMA-13b (SNI). The full results are shown in the Table 2. To investigate the reason of why does WizardLM-13b (ShareGPT Seed) performs worse on GSM8k, we random sample 2000 instructions from ShareGPT and Alpaca data respectively, then use ChatGPT to judge (prompt please refer to Appendix G) whether an instruction is "math" related, we find that the ShareGPT only contains 4.3% math data, and Alpaca data contains 11.8% math data, thus we think that less math data results in worse GSM8k performance of WizardLM-13b (ShareGPT Seed).
::: {caption="Table 2: WizardLM with different data seed, data size, evol model, and base model size."}
:::
The results indicate that (i) the ShareGPT is a better seed for evol-instruct than Alpaca, (ii) larger evolved data size can improve model capacity, and (iii) our proposed Evol-Instruct method is not dependent on ChatGPT, other strong open source model such as Llama-2 is also a good substitute for ChatGPT, (iv) our evloved data also shows better finetune performance than Supernatural Instructions. Futhermore, the results on different pre-trained bases (e.g., Llama-1 65B, Llama-2, Mistral-7B) indicate that our Evol-Instruct can be widely applied to various pre-trained models.
Analysis of In-depth Evolving. The Figure 5a and Figure 5b presents an ablation study investigating the impact of the number of data evolution rounds. To study the depth of the evolving process, we use ChatGPT to judge the difficulty level of instruction. The used prompt please refer to Appendix E.
Figure 5b shows the average scores (on nine automatic benchmarks in Section 4.3) of the models fine-tuned with the data from each evolution round. Each round of data from $C0$ to $C4$ is about $52k$. From the trend of this figure, it can be seen that as the complexity of the training instruction data gradually increases, the performance of the fine-tuned models also improves synchronously. To investigate the correctness of the difficulty score by ChatGPT, we also use GPT-4 and human to measure the instructions difficulty, the detailed results in the Table 3 of Appendix I indicate the good agreement among the ChatGPT, GPT-4 and human annotators.
Analysis of In-breadth Evolving. We aims to examine the semantic breadth of instructions. We use t-SNE [55] and the k-means [56] algorithm to partition instructions BERT embeddings into 20 clusters. Figure 6 in Appendix F displays clusters, highlighting our method's superior dispersion compared to ShareGPT and Alpaca, indicating greater topic diversity in our instructions.
5. Conclusions
Section Summary: The paper introduces an evolutionary method called Evol-Instruct that automatically creates varied and challenging training instructions, enabling the resulting WizardLM model to outperform earlier open-source systems on standard tests, with especially strong gains in coding and math tasks. The authors also note that their reliance on automated and human judgments for evaluation has drawbacks in scale and consistency. They further caution that their test examples may not capture every possible real-world use of such models.
This paper presented Evol-Instruct, an evolutionary algorithm that generates diverse and complex instruction data for LLM. Comprehensive experiments demonstrate that WizardLM significantly surpasses typical open-source LLMs such as Alpaca and Vicuna in a wide range of well-recognized benchmarks. Notably, WizardLM outperforms baselines by a substantial margin in terms of code, math, GPT-4 and human evaluations.
Limitations. This paper acknowledges the limitations of our automatic GPT-4 and human evaluation methods. This method poses challenges for scalability and reliability. Moreover, our test set may not represent all the scenarios or domains where LLM can be applied or compared with other methods.
Appendix
Section Summary: The appendix presents several template-based methods for rewriting simple prompts into more complex versions that remain understandable to humans but pose greater challenges for AI systems. These approaches include expanding the depth and breadth of questions, substituting general concepts with specific ones, requiring explicit multi-step reasoning, and embedding detailed input data in formats such as XML or SQL. Each example shows the original prompt alongside instructions that limit additions to roughly 10–20 words while preserving non-text elements.
A. Deepening Prompt
I I want you act as a Prompt Rewriter.
Your objective is to rewrite a given prompt into a more complex version to make those famous AI systems (e.g., ChatGPT and GPT4) a bit harder to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
Your rewriting cannot omit the non-text parts such as the table and code in \#Given Prompt\#:. Also, please do not omit the input in \#Given Prompt\#.
You SHOULD complicate the given prompt using the following method:
**If \#Given Prompt\# contains inquiries about certain issues, the depth and breadth of the inquiry can be increased.**
You should try your best not to make the \#Rewritten Prompt\# become verbose, \#Rewritten Prompt\# can only add 10 to 20 words into \#Given Prompt\#. '\#Given Prompt\#', '\#Rewritten Prompt\#', 'given prompt' and 'rewritten prompt' are not allowed to appear in \#Rewritten Prompt\#
**\#Given Prompt\#:**
Here is instruction.
**\#Rewritten Prompt\#:**
B. Concretizing Prompt
I I want you act as a Prompt Rewriter.
Your objective is to rewrite a given prompt into a more complex version to make those famous AI systems (e.g., ChatGPT and GPT4) a bit harder to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
Your rewriting cannot omit the non-text parts such as the table and code in \#Given Prompt\#:. Also, please do not omit the input in \#Given Prompt\#.
You SHOULD complicate the given prompt using the following method:
**Please replace general concepts with more specific concepts.**
You should try your best not to make the \#Rewritten Prompt\# become verbose, \#Rewritten Prompt\# can only add 10 to 20 words into \#Given Prompt\#. '\#Given Prompt\#', '\#Rewritten Prompt\#', 'given prompt' and 'rewritten prompt' are not allowed to appear in \#Rewritten Prompt\#
**\#Given Prompt\#:**
Here is instruction.
**\#Rewritten Prompt\#:**
C. Increased Reasoning Steps Prompt
I I want you act as a Prompt Rewriter.
Your objective is to rewrite a given prompt into a more complex version to make those famous AI systems (e.g., ChatGPT and GPT4) a bit harder to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
Your rewriting cannot omit the non-text parts such as the table and code in \#Given Prompt\#:. Also, please do not omit the input in \#Given Prompt\#.
You SHOULD complicate the given prompt using the following method:
**If \#Given Prompt\# can be solved with just a few simple thinking processes, you can rewrite it to explicitly request multiple-step reasoning.**
You should try your best not to make the \#Rewritten Prompt\# become verbose, \#Rewritten Prompt\# can only add 10 to 20 words into \#Given Prompt\#. '\#Given Prompt\#', '\#Rewritten Prompt\#', 'given prompt' and 'rewritten prompt' are not allowed to appear in \#Rewritten Prompt\#
**\#Given Prompt\#:**
Here is instruction.
**\#Rewritten Prompt\#:**
D. Complicate Input Prompt
[]{Prompt for Complicate Input of Evolving}
I I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat to make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [XML data] format text as input data in [Rewritten Prompt]
**\#Given Prompt\#:**
I'm using this php code to fetch the xml data
**\#Rewritten Prompt\#:**
I have this xml and i want to get the xml data to auto populate HTML table, the codes works but it makes duplicate on the table content
Here is the xml data:
```
<root>
<stats>
<item>
<day>2017-11-01</day>
<impressions>2192</impressions>
<money>1.96790003</money>
</item>
<item>
<day>2017-11-02</day>
<impressions>2824</impressions>
<money>3.208500033</money>
</item>
<item>
<day>2017-11-03</day>
<impressions>3680</impressions>
<money>3.321799981</money>
</item>
</stats>
<total>
<impressions>8696</impressions>
<money>8.498200044</money>
</total>
<filter>
<dateFrom>2017-11-01</dateFrom>
<dateTo>2017-11-03</dateTo>
<groupBy>day</groupBy>
<format>xml</format>
</filter>
</root>
```
` I'm using this php code to fetch the xml data but this code fetching from whole xml data which makes duplicate field table `
```
<?php
\$dom = new DOMDocument;
\$dom -> load('http://example.com/' . \$dateselected . '&dateTo=' . \$dateselected2 . '&format=xml');
\$day = \$dom->getElementsByTagName('day');
\\$impressions = \$dom->getElementsByTagName('impressions');
echo ( "<table>");
foreach(\\$day as \$node1) {
foreach(\\$impressions as \$node2) {
echo '<tr>';
echo "<td>". \\$node1 -> textContent . "\mathrel{<}td\mathrel{>}";
echo "\mathrel{<}td\mathrel{>}". \$node2 -> textContent . "<td>";
echo "<td>". \\$node2 -> textContent *0.5/1000 ."<td>";
echo '</tr>';
}
}
echo( "</table>");
?>
```
`Could anyone give a hint how I can fix this? thank you\\\ ####`
I I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat to make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [SQL database] format text as input data in [Rewritten Prompt]
**\#Given Prompt\#:**
achieve the SQL query result
**\#Rewritten Prompt\#**
(MUST contain a specific SQL database as input):
There is a table messages that contains data as shown below:
```
Id Name Other_Columns
-------------------------
1 A A_data_1
2 A A_data_2
3 A A_data_3
4 B B_data_1
5 B B_data_2
6 C C_data_1
```
I If I run a query select \* from messages group by name, I will get the result as:
```
1 A A_data_1
4 B B_data_1
6 C C_data_1
```
`What query will return the following result?`
```
3 A A_data_3
5 B B_data_2
6 C C_data_1
```
`That is, the last record in each group should be returned.`
`At present, this is the query that I use:`
```
SELECT
*
FROM (SELECT
*
FROM messages
ORDER BY id DESC) AS x
GROUP BY name
```
`But this looks highly inefficient. Any other ways to achieve the same result?\ \ ####`
I I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat to make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [python code] format text as input data in [Rewritten Prompt]
**\#Given Prompt\#:**
Transformat python code
**\#Rewritten Prompt\#**
(MUST contain a specific python code as input):
I have the following Python code:
```
cursor.execute("INSERT INTO table VALUES var1, var2, var3,")
```
`where var1 is an integer, var2 and var3 are strings.\ How can I write the variable names without Python including them as part of the query text?\\\ ####`
I I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat to make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [HTML page] format text as input data in [Rewritten Prompt]
**\#Given Prompt\#:**
scroll through the whole HTML page
**\#Rewritten Prompt\#**
(MUST contain a specific HTML page as input):
I want to be able to scroll through the whole page, but without the scrollbar being shown.
In Google Chrome it's:
```
::-webkit-scrollbar display: none;
```
`But Mozilla Firefox and Internet Explorer don't seem to work like that.\ I also tried this in CSS:`
```
overflow: hidden;
```
`That does hide the scrollbar, but I can't scroll any more. Is there a way I can remove the scrollbar while still being able to scroll the whole page?\ \ With just CSS or HTML, please.\ ####`
I I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat to make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [Shell cmd] format text as input data in [Rewritten Prompt]
**\#Given Prompt\#:**
Shell scp file
**\#Rewritten Prompt\#**
(MUST contain a specific Shell cmd as input):
I'm trying to scp a file from a remote server to my local machine. Only port 80 is accessible.
I tried:
scp -p 80 [email protected]:/root/file.txt .
but got this error: cp: 80: No such file or directory
How do I specify the port number in a scp command?
I I want you act as a Prompt Rewriter. Your objective is to rewrite a given prompt into a more complex version using dataformat to make those famous AI systems (e.g., chatgpt and GPT4) more difficult to handle. But the rewritten prompt must be reasonable and must be understood and responded by humans.
You must add [JSON data] format data as input data, add [JSON data] code as input code in [Rewritten Prompt]
Rewrite prompt must be a question style instruction
**\#Given Prompt\#:**
Given a JSON dataset of customer purchase history, how can we calculate the probability of a customer making a repeat purchase from the same store? Can we utilize the formula for conditional probability: $P(A|B)=P(A \cap B) / P(B)$ where A represents the event of a customer making a repeat purchase and B represents the event of a customer making a purchase from the same store again? Additionally, how can we apply this formula to identify the customer segment that is most likely to make a repeat purchase? Can you provide an example of how to implement this formula using the given JSON dataset?
Rewritten prompt must be a question style instruction
**\#Rewritten Prompt\#**
(MUST contain a specific JSON data as input):
E. Difficulty Judge Prompt
W We would like you to evaluate and rate the difficulty and complexity of the following question. You should give an overall score on a scale of 1 to 10, where a higher score indicates higher difficulty and complexity. You must just give a score without any other reasons.
**\#\# Question:**
Here is instruction.
**\#\# Score:**
F. Equal Prompt
H Here are two Instructions to ChatGPT AI, do you think they are equal to each other, which meet the following requirements:
1. They have same constraints and requirments.
2. They have same depth and breadth of the inquiry.
The First Prompt: Here is first instruction.
The Second Prompt: Here is second instruction.
Your Judgement (Just answer: Equal or Not Equal. No need to explain the reason.):
G. Math Judgement Prompt
P Please judge whether the following question is a math problem, and only return True or False without providing any explanation.
Question: instruction
H. WizardEval Analysis
We collected our Evol-Instruct testset that includes real-world human instructions from diverse sources such as online opensource projects, platforms, and forums. We analyzed the data and identified 29 distinct skills that represent the main requirements of humanity, such as Coding Generation
amp;$ Debugging, Math, Reasoning, Complex Formats, Writing, Extensive Disciplines, and so on. Figure 6 illustrates the distribution of the instances and skills in our test set. Our test set consists of 218 instances, each of which is an instruction for a specific skill. We compared our test set with Vicuna’s test set, which is a benchmark dataset for evaluating instruction following models. We found that Vicuna’s test set only 80 instances and 9 skills and is much smaller and less diverse than ours. Figure 4a shows how the difficulty and complexity of the test data vary across different instances. Our test data has a more uniform distribution, meaning that it contains instructions with different levels of difficulty and complexity. On the other hand, Vicuna and Alpaca have a skewed distribution, meaning that they mostly contain instructions with low difficulty and complexity. This indicates that these two corpus are not able to handle the evaluation on more complex and demanding scenarios.
I. Different difficulty Annotators
We just use the ChatGPT to post analyse the"difficult" distribution of the generated instructions, but we do not use this analysis results to guide the data generation or model training. In order to explore the ability of ChatGPT to perform difficulty analysis, we sample 600 instructions and use the more powerful GPT4 model and 5 well-educated human annotators together for difficulty assessment. The assessment results are in the Table 3. The results show that ChatGPT, GPT4, and manual annotation show a high degree of consistency in the trend of difficulty changes.
::: {caption="Table 3: Use ChatGPT, GPT-4, human to measure the instruction difficulty."}
:::
To investigate the correctness of the difficulty score by ChatGPT, we add a new experiment to measure agreement of difficulty judge between ChatGPT and humans: We randomly select two instructions from the six datasets - Alpaca, ShareGPT, C1 to C4 - with equal probability each time, forming a pair. In total, we have selected 300 instruction pairs. Then, we ask ChatGPT and 5 well-educated human annotators to judge which one is more difficulty in one instruction pair, the Kappa score between humans is 0.68, and the Kappa between ChatGPT and human (majority voting) is 0.66, which indicates the good agreement among the ChatGPT and human annotators.
J. Cluster Scatter Plot
In-breadth Evolving aims to enhance topic coverage, skill coverage, and overall dataset diversity. To examine (qualitative analysis) the breadth (diversity) of different dataset, we firstly use BERT to encode each instruction and get its embedding with 768 dimensions, then use a dimension reduction algorithm named t-SNE to reduce embedding dimension to 2, finally we apply a clustering algorithm k-means to partition the instructions of each dataset into 20 clusters for an intuitive visualization. As shown in the Figure 7, the data points of our dataset are more dispersed than ShareGPT and Alpaca (Self-Instruct), which indicates the better topic diversity in our instructions.
K. Human Evaluation Aspects
The annotators then judge which response is better from five aspects:
(1) Relevance: Assessing the model’s ability to correctly interpret the semantic meaning of the context and questions.
(2) Knowledgeable: Whether the model can accurately use various and detailed knowledge for problem-solving.
(3) Reasoning: Assessing the model’s ability to execute correct reasoning processes or devise valid reasoning concepts to solve problems.
(4) Calculation: Evaluating whether the model can perform accurate mathematical computations of the provided formulas in the domains of math, biology, chemistry and physics.
(5) Accuracy: Evaluating whether the model can perform correctly in the corresponding for a given instruction.
L. Performance details of different checkpoints
In this paper, we train our model with 3 epochs and only reported the performance of the final checkpoint in the above "Section 4 Experiment" to align with previous works.
As shown in the following Table 4, we report the model checkpoints performance on different epochs (2.5, 2,75, 3). For 13B models, we can see that the best performance always appears on WizardLM-13b (ShareGPT Seed) for each benchmark except GSM8k. And for 65b/70b models, we also see that the WizardLM-70b is the best one on all the benchmarks. Therefore, we think this is mainly caused by the fluctuations on some benchmarks in model training.
::: {caption="Table 4: Performance details of different checkpoints."}
:::
References
Section Summary: This section consists of a numbered bibliography citing over twenty recent academic papers and preprints focused on large language models. The works address topics such as model training with human feedback, few-shot and zero-shot learning, open-source foundations like Llama, instruction tuning, and techniques for alignment and efficiency. Most references date from 2020 to 2023 and come from venues including NeurIPS, ICLR, and arXiv.
[1] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020.
[2] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
[3] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023.
[4] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Z. Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jianyun Nie, and Ji rong Wen. A survey of large language models. ArXiv, abs/2303.18223, 2023.
[5] Xingwei He, Zheng-Wen Lin, Yeyun Gong, Alex Jin, Hang Zhang, Chen Lin, Jian Jiao, Siu Ming Yiu, Nan Duan, and Weizhu Chen. Annollm: Making large language models to be better crowdsourced annotators. ArXiv, abs/2303.16854, 2023.
[6] Zhen Guo, Peiqi Wang, Yanwei Wang, and Shangdi Yu. Dr. llama: Improving small language models in domain-specific qa via generative data augmentation. 2023.
[7] Jia Li, Ge Li, Yongming Li, and Zhi Jin. Enabling programming thinking in large language models toward code generation. 2023b.
[8] Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Gupta, Kai Hui, Sebastian Ruder, and Donald Metzler. Ext5: Towards extreme multi-task scaling for transfer learning. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=Vzh1BFUCiIX.
[9] Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021.
[10] Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, and Zhilin Yang. Zeroprompt: Scaling prompt-based pretraining to 1,000 tasks improves zero-shot generalization. arXiv preprint arXiv:2201.06910, 2022.
[11] Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. Multitask prompted training enables zero-shot task generalization. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=9Vrb9D0WI4.
[12] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022.
[13] Ziyang Luo, Can Xu, Pu Zhao, Xiubo Geng, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Augmented large language models with parametric knowledge guiding. ArXiv, abs/2305.04757, 2023.
[14] Huan Ma, Changqing Zhang, Yatao Bian, Lemao Liu, Zhirui Zhang, Peilin Zhao, Shu Zhang, H. Fu, Qinghua Hu, and Bing Wu. Fairness-guided few-shot prompting for large language models. ArXiv, abs/2303.13217, 2023.
[15] Zhiqiang Hu, Yihuai Lan, Lei Wang, Wanyu Xu, Ee-Peng Lim, Roy Ka-Wei Lee, Lidong Bing, and Soujanya Poria. Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models. ArXiv, abs/2304.01933, 2023.
[16] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. ArXiv, abs/2304.10592, 2023.
[17] Andreas Kopf, Yannic Kilcher, Dimitri von Rutte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Rich'ard Nagyfi, ES Shahul, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. Openassistant conversations - democratizing large language model alignment. ArXiv, abs/2304.07327, 2023.
[18] Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, Jianquan Li, Xiang Wan, Benyou Wang, and Haizhou Li. Phoenix: Democratizing chatgpt across languages. ArXiv, abs/2304.10453, 2023.
[19] Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David D. Cox, Yiming Yang, and Chuang Gan. Principle-driven self-alignment of language models from scratch with minimal human supervision. ArXiv, abs/2305.03047, 2023.
[20] Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Feiran Huang. Rrhf: Rank responses to align language models with human feedback without tears. ArXiv, abs/2304.05302, 2023.
[21] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL https://vicuna.lmsys.org.
[22] Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mi Zhou. Automl-gpt: Automatic machine learning with gpt. ArXiv, abs/2305.02499, 2023.
[23] Wen Xiao, Yujia Xie, Giuseppe Carenini, and Pengcheng He. Chatgpt-steered editing instructor for customization of abstractive summarization. ArXiv, abs/2305.02483, 2023.
[24] Potsawee Manakul, Adian Liusie, and Mark John Francis Gales. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. ArXiv, abs/2303.08896, 2023.
[25] Shan Zhong, Zhongzhan Huang, Wushao Wen, Jinghui Qin, and Liang Lin. Sur-adapter: Enhancing text-to-image pre-trained diffusion models with large language models. ArXiv, abs/2305.05189, 2023.
[26] Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. Tallrec: An effective and efficient tuning framework to align large language model with recommendation. ArXiv, abs/2305.00447, 2023.
[27] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. ArXiv, abs/2304.08485, 2023.
[28] Ning Bian, Pei Yu Liu, Xianpei Han, Hongyu Lin, Yaojie Lu, Ben He, and Le Sun. A drop of ink may make a million think: The spread of false information in large language models. ArXiv, abs/2305.04812, 2023.
[29] Vivien A. Cabannes, Léon Bottou, Yann LeCun, and Randall Balestriero. Active self-supervised learning: A few low-cost relationships are all you need. ArXiv, abs/2303.15256, 2023.
[30] Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford_alpaca, 2023.
[31] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022a.
[32] Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688, 2023.
[33] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. URL http://jmlr.org/papers/v21/20-074.html.
[34] Yufei Wang, Jiayi Zheng, Can Xu, Xiubo Geng, Tao Shen, Chongyang Tao, and Daxin Jiang. Knowda: All-in-one knowledge mixture model for data augmentation in few-shot nlp. arXiv preprint arXiv:2206.10265, 2022c.
[35] Adrian de Wynter, Xun Wang, Alex Sokolov, Qilong Gu, and Si-Qing Chen. An evaluation on large language model outputs: Discourse and memorization. ArXiv, abs/2304.08637, 2023.
[36] Ekaterina Svikhnushina and Pearl Pu. Approximating human evaluation of social chatbots with prompting. ArXiv, abs/2304.05253, 2023.
[37] Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jia-Bin Huang, Jinglin Liu, Yixiang Ren, Zhou Zhao, and Shinji Watanabe. Audiogpt: Understanding and generating speech, music, sound, and talking head. ArXiv, abs/2304.12995, 2023.
[38] Xiang Yue, Boshi Wang, Kai Zhang, Zi-Yuan Chen, Yu Su, and Huan Sun. Automatic evaluation of attribution by large language models. ArXiv, abs/2305.06311, 2023.
[39] Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4, 2023.
[40] OpenAI. Gpt-4 technical report, 2023.
[41] Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with parameter-efficient tuning on self-chat data, 2023.
[42] Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large language model society, 2023a.
[43] Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. How far can camels go? exploring the state of instruction tuning on open resources, 2023.
[44] Edward Beeching, Clémentine Fourrier, Nathan Habib, Sheon Han, Nathan Lambert, Nazneen Rajani, Omar Sanseviero, Lewis Tunstall, and Thomas Wolf. Open llm leaderboard. https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard, 2023.
[45] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020.
[46] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge, 2018.
[47] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019.
[48] Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods, 2022.
[49] Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, Jason Phang, Laria Reynolds, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, September 2021. URL https://doi.org/10.5281/zenodo.5371628.
[50] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Joshua Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. CoRR, abs/2107.03374, 2021. URL https://arxiv.org/abs/2107.03374.
[51] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
[52] Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval, 2023c.
[53] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685, 2023.
[54] Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint arXiv:2204.07705, 2022b.
[55] Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(86):2579–2605, 2008. URL http://jmlr.org/papers/v9/vandermaaten08a.html.
[56] J. A. Hartigan and M. A. Wong. A k-means clustering algorithm. JSTOR: Applied Statistics, 28(1):100–108, 1979.