Back

Calibrate Before Use: Improving Few-Shot Performance of Language Models

Tony Z. ZhaoEric WallaceShi FengDan KleinSameer Singh

article2021ICML2,062 citations

Proposes contextual calibration, an approach that corrects language model output biases using content-free inputs to substantially reduce prompt variance and boost few-shot accuracy across diverse tasks.

few-shot learningnatural language questionsstatistical language modelsTraining ExamplesAccuracyLanguage Models
arXiv: 2102.09690
  • Paper: Language Models are Few-Shot Learners, T. B. Brown et al. (2020). It introduced the paradigm of few-shot prompting in large autoregressive models like GPT-3, establishing the foundational in-context learning framework whose instability and bias the source specifically investigates and mitigates.
  • Paper: On Calibration of Modern Neural Networks, Chuan Guo et al. (2017). It provides the foundational analysis of probability miscalibration in modern neural networks and temperature scaling techniques that underpin the post-hoc calibration methods adapted in the source.
  • Paper: Language Models are Unsupervised Multitask Learners, Alec Radford et al. (2019). It establishes the underlying GPT-2 architecture and the zero-shot multitask evaluation framework that the source's few-shot calibration experiments build upon.
  • Paper: When Does Label Smoothing Help?, Rafael Müller et al. (2019). It offers core theoretical and empirical insights into how target probability distributions impact model overconfidence and calibration error.
Cover for Calibrate Before Use: Improving Few-Shot Performance of Language Models

Abstract

GPT-3 can perform numerous tasks when provided a natural language prompt that contains a few training examples. We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art. We demonstrate that this instability arises from the bias of language models towards predicting certain answers, e.g., those that are placed near the end of the prompt or are common in the pre-training data. To mitigate this, we first estimate the model's bias towards each answer by asking for its prediction when given the training prompt and a content-free test input such as "N/A". We then fit calibration parameters that cause the prediction for this input to be uniform across answers. On a diverse set of tasks, this contextual calibration procedure substantially improves GPT-3 and GPT-2's average accuracy (up to 30.0% absolute) and reduces variance across different choices of the prompt.

Topics

few-shot learningnatural language questionsstatistical language modelscontext-freetraining programs

Keywords

Training ExamplesAccuracyLanguage ModelsPromptFew-ShotImproving Few-Shot PerformanceCalibrationUseGPT-3Movie

Table of Contents

  • 1 Introduction
  • 2 Background and Experimental Setup
  • 2.1 Datasets and Prompt Formats
  • 2.2 Model Details
  • 3 Accuracy Varies Highly Across Prompts
  • 4 What Causes the High Variance?
  • 5 Contextual Calibration
  • 5.1 Results for Contextual Calibration
  • 5.2 Ablations on Contextual Calibration
  • 6 Discussion
  • 7 Related Work
  • 8 Conclusion and Future Work
  • References
  • A Additional Results on Variance and Calibration
  • B Prompt Formats Used

Knowls

  1. Knowl 1 — Contextual Calibration for In-Context Few-Shot Predictions

    algorithm

    Contextual calibration is a data-free post-processing procedure designed to remove systematic prediction biases (such as recency, majority label, and common token biases) in autoregressive language models performing few-shot in-context learning.

    Input: Language model LMLM, prompt template TT containing kk demonstration exemplars, candidate label set Y={y1,y2,…,yK}\mathcal{Y} = \{y_1, y_2, \dots, y_K\}, test input xx
    Output: Calibrated label prediction y^∈Y\hat{y} \in \mathcal{Y}
    Construct content-free inputs xcf∈{"N/A","[MASK]",""}x_{\text{cf}} \in \{\text{"N/A"}, \text{"[MASK]"}, \text{""}\}
    for each xcfx_{\text{cf}} do
        Form null prompt Pcf=T(xcf)P_{\text{cf}} = T(x_{\text{cf}})
        Query LMLM to obtain raw next-token probabilities p(yj∣Pcf)p(y_j \mid P_{\text{cf}}) for all yj∈Yy_j \in \mathcal{Y}
    end for
    Compute average content-free distribution p^cf∈RK\hat{p}_{\text{cf}} \in \mathbb{R}^K where (p^cf)j=13∑xcfp(yj∣T(xcf))∑m=1Kp(ym∣T(xcf))(\hat{p}_{\text{cf}})_j = \frac{1}{3} \sum_{x_{\text{cf}}} \frac{p(y_j \mid T(x_{\text{cf}}))}{\sum_{m=1}^K p(y_m \mid T(x_{\text{cf}}))}
    Set diagonal weight matrix W=diag(p^cf)−1∈RK×KW = \text{diag}(\hat{p}_{\text{cf}})^{-1} \in \mathbb{R}^{K \times K} and bias vector b=0∈RKb = \mathbf{0} \in \mathbb{R}^K
    Form test prompt Ptest=T(x)P_{\text{test}} = T(x)
    Obtain normalized test label distribution p^∈RK\hat{p} \in \mathbb{R}^K where p^j=p(yj∣Ptest)∑m=1Kp(ym∣Ptest)\hat{p}_j = \frac{p(y_j \mid P_{\text{test}})}{\sum_{m=1}^K p(y_m \mid P_{\text{test}})}
    Compute calibrated logits s^=Wp^+b\hat{s} = W \hat{p} + b
    return y^=arg⁡max⁡yj∈Ys^j\hat{y} = \arg\max_{y_j \in \mathcal{Y}} \hat{s}_j

    For open-ended single-token generation tasks where the dimensionality of the candidate space is large (e.g., the full vocabulary ≈50,000\approx 50{,}000), an alternate parameterization sets W=IW = I (the identity matrix) and bias b=−p^cfb = -\hat{p}_{\text{cf}}, subtracting the content-free bias vector directly from unnormalized next-token output probabilities before taking the arg⁡max⁡\arg\max.

  2. Knowl 2 — Affine Transformation for Output Probability Calibration

    equation

    To counteract output distribution shifts induced by prompt design and language model priors without parameter finetuning, the model's raw probability vector p^∈RK\hat{p} \in \mathbb{R}^K over KK candidate completion classes is adjusted via an affine transformation:

    q^=softmax(Wp^+b)\hat{q} = \text{softmax}(W\hat{p} + b)

    where p^\hat{p} is the vector of raw next-token probabilities assigned by the autoregressive language model to candidate tokens (renormalized to sum to 1 over the KK candidate classes), W∈RK×KW \in \mathbb{R}^{K \times K} is a diagonal scaling matrix defined by:

    W=diag(p^cf)−1W = \text{diag}(\hat{p}_{\text{cf}})^{-1}

    b∈RKb \in \mathbb{R}^K is an offset vector (set to 0\mathbf{0} for classification or −p^cf-\hat{p}_{\text{cf}} when W=IW=I), and p^cf∈RK\hat{p}_{\text{cf}} \in \mathbb{R}^K is the probability distribution output by the model when evaluated on the identical prompt format with a content-free placeholder string (e.g., "N/A"). The class prediction is obtained via arg⁡max⁡i(Wp^+b)i\arg\max_{i} (W\hat{p} + b)_i.

  3. Knowl 3 — Majority Label, Recency, and Common Token Biases in Few-Shot Language Models

    definition

    The prediction instability and high variance of autoregressive language models in few-shot in-context learning are driven by three systematic biases:

    1. Majority Label Bias: The tendency of the language model to favor classes that appear more frequently among the in-context demonstration examples. In 1-shot learning, this often causes the model to simply echo the single demonstrated class on all test inputs, causing accuracy drops relative to 0-shot baselines.
    2. Recency Bias: The tendency of the language model to disproportionately predict answer labels that appear near the end of the prompt sequence (i.e., in the final exemplars). This effect can override majority label bias; for example, placing a negative example at the final position of a 4-shot sentiment prompt containing three positive examples can cause nearly 90% of test predictions to be labeled negative.
    3. Common Token Bias: The tendency of the language model to predict tokens that occur with high frequency in its pre-training corpus (e.g., preferring common entity names like "United States" over rare entities, or predicting common category labels such as "book" at eleven times the rate of "artist").

  4. Knowl 4 — Instability of Few-Shot In-Context Accuracy Across Example Permutations and Formats

    empirical result

    In-context few-shot learning with autoregressive language models exhibits severe variance depending on prompt formatting, demonstration exemplar selection, and demonstration ordering:

    • Ordering / Permutation Sensitivity: Keeping the set of few-shot training examples fixed while simply altering their ordering/permutation in the prompt causes accuracy on SST-2 (4-shot, GPT-3 2.7B) to fluctuate between near random chance (54.3%) and near state-of-the-art (93.4%). Similarly, on a 2-shot prompt, swapping the order of two examples drops accuracy from 88.5% to 51.3%.
    • Prompt Format Sensitivity: Evaluating across 15 different natural language prompt formats for the same task and training examples reveals large average performance spreads and high intra-format variance across exemplar draws.
    • Persistence Across Scale and Sample Count: Increasing model parameter scale (from GPT-3 2.7B to GPT-3 175B) or increasing the number of prompt exemplars up to 16 does not eliminate this variance, and moving from 0-shot to 1-shot often reduces average accuracy (e.g., DBPedia 0-shot accuracy drops from 36.0% to 25.9% in 1-shot).

  5. Knowl 5 — Few-Shot Accuracy Comparison Between Uncalibrated Greedy Decoding and Contextual Calibration Across GPT-3 Scales

    data/table

    Contextual calibration substantially improves average accuracy (by up to 30.0% absolute) and reduces the standard deviation across different random training exemplar draws across text classification, fact retrieval, and information extraction tasks.

    Dataset LM 0-shot 1-shot 4-shot 8-shot
    Baseline Ours Baseline Ours Baseline Ours Baseline Ours
    AGNews 2.7B 44.7 63.2 33.0 59.6 43.3 71.1 50.8 72.7
    AGNews 175B 43.9 73.9 62.1 77.1 61.0 85.9 79.1 84.3
    TREC 2.7B 31.0 38.8 24.3 36.8 25.8 38.6 29.3 44.3
    TREC 175B 47.4 57.4 57.7 75.7 60.2 69.7 45.6 66.9
    CB 2.7B 44.6 50.0 33.8 33.0 43.5 54.2 43.9 53.0
    CB 175B 30.4 48.2 50.9 51.8 45.2 60.7 59.6 65.0
    RTE 2.7B 44.8 49.5 49.6 50.4 44.0 54.5 49.2 54.8
    RTE 175B 57.8 57.8 62.9 62.8 58.7 60.4 66.2 65.5
    SST-2 2.7B 57.2 71.4 67.3 79.1 59.1 79.9 54.0 82.0
    SST-2 175B 71.6 75.8 93.3 94.7 93.6 94.3 95.6 95.3
    DBPedia 2.7B 36.0 38.7 25.9 61.6 61.0 66.0 72.6 74.8
    DBPedia 175B 22.0 59.7 79.3 85.3 84.6 86.9 82.3 86.9
    LAMA 2.7B 14.0 22.7 29.7 31.6 35.8 37.4 42.5 42.5
    LAMA 175B 23.5 30.1 48.9 49.0 62.0 61.8 63.8 63.6
    MIT-Genre 2.7B 5.0 5.7 26.7 37.9 53.1 54.7 59.0 59.1
    MIT-Genre 13B 15.0 18.7 47.3 52.0 57.9 58.9 59.0 59.1
    MIT-Director 2.7B 46.3 47.0 42.0 53.5 73.5 74.1 75.3 75.1
    MIT-Director 13B 36.3 38.7 58.6 72.8 75.4 75.9 77.8 77.8
    ATIS-Airline 2.7B 10.8 14.0 29.8 33.1 43.0 47.3 55.6 58.8
    ATIS-Airline 13B 49.5 52.7 69.6 71.8 67.5 69.6 63.4 64.5
    ATIS-Date 2.7B 6.4 12.9 42.3 65.6 75.0 83.4 81.0 88.3
    ATIS-Date 13B 4.0 5.0 97.9 95.5 98.0 97.8 98.8 98.8

    The table demonstrates that contextual calibration consistently improves mean accuracy across shots and scales. In several configurations, contextual calibration enables the smaller GPT-3 2.7B model to surpass the uncalibrated baseline performance of the 50x larger GPT-3 175B model.

  6. Knowl 6 — Data-Free Contextual Calibration Competitiveness with Labeled Oracle Calibration

    empirical result

    Contextual calibration, which estimates the diagonal scaling matrix WW entirely without labeled training data via content-free input queries, achieves accuracy closely matching an "oracle" calibration procedure that optimizes the diagonal elements of WW directly on a labeled validation set.

    On the 4-way AGNews classification benchmark with GPT-3 175B across 0, 1, 4, 8, and 16 exemplar shots, the mean accuracy curve of contextual calibration tracks within 1–2% of oracle calibration across all shot regimes, while both substantially outperform the uncalibrated baseline (e.g., improving from ~61% uncalibrated to ~86% calibrated at 4 shots).

  7. Knowl 7 — Effectiveness of Contextual Calibration on GPT-2 Models

    empirical result

    Contextual calibration applies directly to earlier autoregressive language models such as GPT-2 XL (1.5B parameters), confirming that prompt bias and output distribution shifts are general properties of autoregressive language models rather than specific to GPT-3.

    On GPT-2 XL:

    • 0-shot Accuracy: SST-2 accuracy improves from 60.0% to 82.0%; AGNews improves from 44.0% to 60.0%; TREC improves from 24.0% to 37.3%; LAMA fact retrieval improves from 14.0% to 22.7%.
    • 1-shot Accuracy: AGNews improves from 45.4% (±8.4\pm 8.4) to 67.9% (±5.7\pm 5.7); DBPedia improves from 33.6% (±18.9\pm 18.9) to 69.5% (±9.4\pm 9.4); MIT-Director improves from 26.2% (±10.5\pm 10.5) to 58.8% (±4.8\pm 4.8).
    • Variance Reduction: Standard deviations across different randomly sampled training sets are reduced substantially across 1-shot, 4-shot, and 8-shot settings.

  8. Knowl 8 — Impact of Content-Free Null Input Representations on Calibration Accuracy

    empirical result

    Evaluating various choices of content-free inputs xcfx_{\text{cf}} for contextual calibration on 1-shot SST-2 and 0-shot AGNews shows that while calibration accuracy is somewhat sensitive to the choice of null string, a wide range of dummy tokens reliably outperform the uncalibrated baseline:

    • Uncalibrated Baseline: 66.5% (SST-2), 48.5% (AGNews)
    • "N/A": 74.2% (SST-2), 64.5% (AGNews)
    • "[MASK]": 74.5% (SST-2), 63.8% (AGNews)
    • Empty String "": 72.9% (SST-2), 64.7% (AGNews)
    • Ensemble Average of {"N/A", "[MASK]", ""}: 79.0% (SST-2), 66.5% (AGNews)
    • Generic / Random Strings (e.g., "the", "abc", "the man.", gibberish "dasjhasjkdhjskdhds"): 69.1%–79.4% (SST-2), 57.3%–65.5% (AGNews)

    Averaging the output distributions across "N/A", "[MASK]", and the empty string provides the most consistent performance improvements across datasets.

Coverage note — None was omitted; all key contributed methods, theoretical bias definitions, algorithmic procedures, and empirical results across datasets, model sizes, and ablations are covered.

References

  1. 1.Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., and Choi, Y. COMET: Commonsense transformers for automatic knowledge graph construction. In ACL, 2019.
  2. 2.Braverman, M., Chen, X., Kakade, S., Narasimhan, K., Zhang, C., and Zhang, Y. Calibration, entropy rates, and memory in language models. In ICML, 2020.
  3. 3.Brier, G. W. Verification of forecasts expressed in terms of probability. Monthly Weather Review, 1950.
  4. 4.Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In NeurIPS, 2020.
  5. 5.Dagan, I., Glickman, O., and Magnini, B. The PASCAL recognising textual entailment challenge. In Machine Learning Challenges Workshop, 2005.
  6. 6.de Marneffe, M.-C., Simons, M., and Tonhauser, J. The CommitmentBank: Investigating projection in naturally occurring discourse. In Sinn und Bedeutung, 2019.
  7. 7.Devlin, J., Chang, M., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
  8. 8.Gao, T., Fisch, A., and Chen, D. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723, 2020.
  9. 9.Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In ICML, 2017.
  10. 10.Hemphill, C. T., Godfrey, J. J., and Doddington, G. R. The ATIS spoken language systems pilot corpus. In Speech and Natural Language Workshop, 1990.
  11. 11.Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y. The curious case of neural text degeneration. In ICLR, 2020.
  12. 12.Jiang, Z., Araki, J., Ding, H., and Neubig, G. How can we know when language models know? arXiv preprint arXiv:2012.00955, 2020a.
  13. 13.Jiang, Z., Xu, F. F., Araki, J., and Neubig, G. How can we know what language models know? In TACL, 2020b.
  14. 14.Khandelwal, U., He, H., Qi, P., and Jurafsky, D. Sharp nearby, fuzzy far away: How neural language models use context. In ACL, 2018.
  15. 15.Lake, B. M., Salakhutdinov, R., and Tenenbaum, J. B. Human-level concept learning through probabilistic program induction. In Science, 2015.
  16. 16.Li, J., Galley, M., Brockett, C., Gao, J., and Dolan, B. A diversity-promoting objective function for neural conversation models. In NAACL, 2016.
  17. 17.Liu, J., Cyphers, S., Pasupat, P., McGraw, I., and Glass, J. A conversational movie search system based on conditional random fields. In INTERSPEECH, 2012.
  18. 18.Logan, R. L., Liu, N. F., Peters, M. E., Gardner, M., and Singh, S. Barack’s wife Hillary: Using knowledge-graphs for fact-aware language modeling. In ACL, 2019.
  19. 19.Paulus, R., Xiong, C., and Socher, R. A deep reinforced model for abstractive summarization. In ICLR, 2018.
  20. 20.Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A. H., and Riedel, S. Language models as knowledge bases? In EMNLP, 2019.
  21. 21.Platt, J. C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers, 1999.
  22. 22.Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. Improving language understanding by generative pre-training. Technical Report, 2018.
  23. 23.Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. Technical Report, 2019.
  24. 24.Ravfogel, S., Goldberg, Y., and Linzen, T. Studying the inductive biases of RNNs with synthetic variations of natural languages. In NAACL, 2019.
  25. 25.Schick, T. and Schütze, H. It’s not just size that matters: Small language models are also few-shot learners. arXiv preprint arXiv:2009.07118, 2020.
  26. 26.Schick, T. and Schütze, H. Exploiting cloze questions for few-shot text classification and natural language inference. In EACL, 2021.
  27. 27.Schwartz, R., Sap, M., Konstas, I., Zilles, L., Choi, Y., and Smith, N. A. The effect of different writing tasks on linguistic style: A case study of the ROC story cloze task. In ACL, 2017.
  28. 28.Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. AutoPrompt: Eliciting knowledge from language models with automatically generated prompts. In EMNLP, 2020.
  29. 29.Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, 2013.
  30. 30.Trinh, T. H. and Le, Q. V. A simple method for common-sense reasoning. arXiv preprint arXiv:1806.02847, 2018.
  31. 31.Voorhees, E. M. and Tice, D. M. Building a question answering test collection. In SIGIR, 2000.
  32. 32.Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In NeurIPS, 2019.
  33. 33.Yogatama, D., d’Autume, C. d. M., Connor, J., Kocisky, T., Chrzanowski, M., Kong, L., Lazaridou, A., Ling, W., Yu, L., Dyer, C., et al. Learning and evaluating general linguistic intelligence. arXiv preprint arXiv:1901.11373, 2019.
  34. 34.Zhang, X., Zhao, J., and LeCun, Y. Character-level convolutional networks for text classification. In NeurIPS, 2015.

Citation

MLA
Zhao, T. Z., et al. “Calibrate Before Use: Improving Few-Shot Performance of Language Models”. arXiv, 2021, http://arxiv.org/abs/2102.09690v2.
APA
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021). Calibrate Before Use: Improving Few-Shot Performance of Language Models. arXiv. http://arxiv.org/abs/2102.09690v2
Chicago
Zhao, T. Z., E. Wallace, S. Feng, D. Klein, and S. Singh. 2021. “Calibrate Before Use: Improving Few-Shot Performance of Language Models”. arXiv. http://arxiv.org/abs/2102.09690v2.
Harvard
Zhao, T.Z. et al. (2021) “Calibrate Before Use: Improving Few-Shot Performance of Language Models”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2102.09690v2.
Vancouver
1. Zhao TZ, Wallace E, Feng S, Klein D, Singh S (2021) Calibrate Before Use: Improving Few-Shot Performance of Language Models. arXiv

BibTeX

@article{zhao2021calibrate,
  title = {Calibrate Before Use: Improving Few-Shot Performance of Language Models},
  author = {Zhao, Tony Z. and Wallace, Eric and Feng, Shi and Klein, Dan and Singh, Sameer},
  year = {2021},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2102.09690v2},
  eprint = {2102.09690}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/