Back

Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting

Zahra FatemiChen XingWenhao LiuCaiming Xiong

article2023ACL46 citations

Presents GEnder Equality Prompt (GEEP), a prompt-tuning method that freezes base model parameters and trains dedicated profession embeddings on gender-neutral data to mitigate gender bias while preventing catastrophic forgetting on general NLP benchmarks.

Listen

Large pre-trained language models routinely absorb and amplify societal gender biases from web-scale data, often associating professions like "nurse" with women and "doctor" with men. While standard remediation methods train existing models on smaller, balanced datasets, this secondary training causes "catastrophic forgetting," where models lose general language knowledge and suffer major performance declines on downstream tasks.

The article demonstrates the extent of this forgetting problem and introduces a lightweight debiasing framework called GEnder Equality Prompt (GEEP) to reduce profession-based gender bias while preserving general model capabilities.

To evaluate debiasing methods, the authors constructed a 6.1-gigabyte gender-neutral dataset from Wikipedia by swapping gendered terms across profession-related sentences. Standard fine-tuning across all model parameters was compared against the proposed method, which freezes the original model and introduces newly initialized, trainable embeddings for 303 profession terms. Evaluations were conducted primarily on RoBERTa across standard fairness benchmarks—such as pronoun-profession resolution tasks—and the eight-task General Language Understanding Evaluation (GLUE) benchmark.

Updating all model parameters reduced overall language understanding on GLUE from 86.5 to 80.2 points, with steep drops of over 18 to 25 points on sensitive linguistic tasks. In contrast, the prompt-based method retained strong general capability, scoring 83.3 on GLUE while outperforming standard debiasing across fairness benchmarks. Specifically, it boosted profession pronoun resolution accuracy on the Winogender dataset to 64.5%, compared to 57.3% for standard debiasing and 50.9% for the baseline model. Furthermore, on direct pronoun prediction across 303 professions, the proposed method reduced gender bias scores by 70% compared to base BERT, and achieved near-peak fairness performance in one-fifth of the training iterations required by traditional retraining.

These findings indicate that updating entire neural networks on narrow debiasing data introduces severe operational risks by degrading broader linguistic competence. The proposed method resolves this trade-off by adding only 232,000 parameters (a 0.21% increase), providing a cost-effective, scalable way to debias language models rapidly without requiring full retraining.

Organizations seeking to improve artificial intelligence fairness should adopt parameter-efficient prompt tuning rather than full-model retraining. Future work should extend this approach beyond binary profession terms to address broader non-binary gender identities, intersectional fairness, and other demographic attributes before deploying models in high-stakes environments.

SalesforceUniversity of Illinois Chicagoethnicitystatistical language modelsword sense disambiguationgender fairnesspre-trained language modelscatastrophic forgetting
arXiv: 2110.05367
Cover for Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting

Abstract

Existing studies addressing gender bias of pre-trained language models, usually build a small gender-neutral data set and conduct a second phase pre-training on the model with such data. However, given the limited size and concentrated focus of the gender-neutral data, catastrophic forgetting would occur during second-phase pre-training. Forgetting information in the original training data may damage the model’s downstream performance by a large margin. In this work, we empirically show that catastrophic forgetting occurs in such methods by evaluating them with general NLP tasks in GLUE. Then, we propose a new method, GEnder Equality Prompt (GEEP), to improve gender fairness of pre-trained models with less forgetting. GEEP freezes the pre-trained model and learns gender-related prompts with gender-neutral data. Empirical results show that GEEP not only achieves SOTA performances on gender fairness tasks, but also forgets less and performs better on GLUE by a large margin.

Topics

ethnicitystatistical language modelsword sense disambiguationintegrated data

Keywords

gender fairnesspre-trained language modelscatastrophic forgettingpronoun prediction biasgender neutralizationgender equality promptspronoun coreference resolution taskspre-trained RoBERTa-baseBERT-base models

Table of Contents

  • 1 Introduction
  • 2 Related Work
  • 3 Improving Gender Fairness without Forgetting
  • 3.1 Profession-Related Gender-Neutral Data Collection
  • 3.2 Gender Equality Prompt Approach
  • 4 Experiments
  • 4.1 Experimental Setup
  • 4.2 Evaluation Tasks
  • 4.3 Results
  • 5 Closing Remarks
  • References
  • A Limitations
  • B Appendix
  • B.1 Hyper-parameters for SPPA and GEEP
  • B.2 Pronoun Coreference Resolution Experiment Setup
  • B.3 GLUE Experiment Setup
  • B.4 Pronoun Prediction Experiment Setup and Results
  • B.5 Analysis regarding SPPA’s performance drop on GLUR
  • B.6 Discussions on non-binary gender identities
  • B.7 The capacity increase of GEEP compared to SPPA
  • B.8 Quality of gender-neutral data
  • B.9 Experiment Results on BERT

Knowls

  1. Knowl 1 — GEEP freezes the language model and learns replacement profession embeddings

    model/method

    GEnder Equality Prompt (GEEP) reduces gender bias while limiting changes to a pre-trained language model by freezing all its pre-trained parameters and adding trainable embeddings for the profession names being debiased. If the original vocabulary has nn tokens and embedding dimension dd, its embedding matrix has shape n×dn\times d; GEEP adds an m×dm\times d matrix for mm professions and concatenates it along the token-vocabulary dimension, leaving the model's hidden dimension unchanged. The new profession embeddings are randomly initialized, and GEEP uses them instead of the original embeddings whenever those professions occur during debiasing, later fine-tuning, or inference. The original model parameters remain frozen throughout. For the 303 professions and 768-dimensional embeddings used with RoBERTa-base, the addition is 232,000 parameters, or 0.21% of the model's 110 million parameters.

  2. Knowl 2 — Profession-related gender-neutral training data

    data/table

    The authors construct profession-related gender-neutral text by filtering English Wikipedia for sentences containing professions regarded as gender-neutral but stereotypically gendered, then creating counterparts by swapping gendered terms such as “he” and “she” or “man” and “woman.” The training collection contains both each original sentence and its gender-swapped counterpart. The resulting corpus is 6.1 GB, compared with the 160 GB of text reported for RoBERTa's original pre-training data; it is therefore much smaller and less diverse.

    Corpus Text size
    Profession-related gender-neutral data 6.1 GB
    RoBERTa original pre-training data 160 GB

  3. Knowl 3 — GEEP training objective and main training configuration

    experimental setup

    GEEP trains its new profession embeddings on gender-neutral text using the masked-language-modeling objective. For a sequence xx with NmaskN_{\mathrm{mask}} masked positions, let xtx_t be the original token at masked position tt, and let xcontextx_{\mathrm{context}} denote the sequence context supplied to the model. The model minimizes the mean negative log probability of the original masked tokens; the pre-trained parameters θ\theta stay fixed, while the new profession embedding matrix WpW_p is trainable:

    L(Wp)=−1Nmask∑t=1Nmasklog⁡pθ,Wp(xt∣xcontext).\mathcal{L}(W_p)=-\frac{1}{N_{\mathrm{mask}}}\sum_{t=1}^{N_{\mathrm{mask}}}\log p_{\theta,W_p}(x_t\mid x_{\mathrm{context}}).

    The main RoBERTa-base experiments train for 100,000 steps with AdamW, learning rate 1×10−51\times10^{-5}, maximum sequence length 128, and batch size 256. Each profession embedding is initialized from a normal distribution with standard deviation 0.2. The same gender-neutral data is used for the SPPA baseline, which instead updates all model parameters.

  4. Knowl 4 — GEEP improves pronoun coreference accuracy over RoBERTa and SPPA

    data/table

    After fine-tuning on GAP, the models are evaluated for pronoun coreference on Winogender, WSC, and DPR/WSCR. The reported accuracies show that GEEP outperforms both the original RoBERTa-base model and SPPA on all three datasets; its largest gain over SPPA is on Winogender, whose examples feature professions. The task scores are percentages.

    Dataset RoBERTa SPPA GEEP
    Winogender 50.9 57.3 64.5
    WSC 50.1 50.9 52.7
    DPR/WSCR 50.8 51.1 53.6

    For this evaluation, models are fine-tuned on GAP for one epoch with batch size 64 and learning rate 5.0×10−65.0\times10^{-6}. Winogender has 1,584 sentences, WSC has 273, and the DPR test set has 131 sentences.

  5. Knowl 5 — GEEP retains more GLUE performance than full-model second-phase pre-training

    data/table

    The authors use GLUE performance after second-phase pre-training to assess forgetting. Relative to RoBERTa-base, SPPA's average across eight reported tasks falls from 86.5 to 80.2, while GEEP reaches 83.3. GEEP scores higher than SPPA on seven tasks and ties it on QNLI; it mitigates, but does not eliminate, the performance loss relative to the original model. CoLA and RTE show particularly large losses for SPPA. Scores are the task metrics reported by the paper: Matthews correlation for CoLA, Pearson correlation for STS-B, and accuracy for the other tasks.

    Task RoBERTa SPPA GEEP
    MNLI 87.7 87.2 87.7
    QNLI 92.4 92.4 92.4
    QQP 91.8 91.3 91.7
    SST-2 95.4 94.7 95.4
    CoLA 64.1 38.9 50.5
    MRPC 91.4 88.8 89.8
    RTE 78.4 60.2 68.7
    STS-B 90.7 88.3 89.9
    Average 86.5 80.2 83.3

  6. Knowl 6 — GEEP reaches strong coreference results in fewer training steps

    data/table

    A training-duration comparison shows that GEEP improves coreference accuracy substantially earlier than SPPA. At 20,000 steps, GEEP already exceeds SPPA's 100,000-step score on all three coreference datasets: 64.3 versus 57.3 on Winogender, 52.1 versus 50.9 on WSC, and 52.1 versus 51.1 on DPR/WSCR. GEEP's scores are near their 100,000-step values by 50,000 steps, whereas SPPA remains below its final Winogender and WSC results at that point. The table also reports average GLUE scores at each training duration.

    Dataset RoBERTa SPPA-20k GEEP-20k SPPA-50k GEEP-50k SPPA-100k GEEP-100k
    Winogender 50.9 51.6 64.3 54.6 64.5 57.3 64.5
    WSC 50.1 50.1 52.1 50.5 52.3 50.9 52.7
    DPR/WSCR 50.8 50.9 52.1 51.1 53.4 51.1 53.6
    Average GLUE 86.5 82.7 85.9 80.7 84.5 80.2 83.3

    The configurations use the same gender-neutral data and training setup; “20k,” “50k,” and “100k” indicate second-phase pre-training steps.

  7. Knowl 7 — Controlled variants implicate both data selection and gender neutralization in SPPA's GLUE drop

    empirical result

    The authors compare standard SPPA and GEEP with variants trained on the same Wikipedia subset without gender neutralization, and with an SPPA variant that adds new profession embeddings but still updates all parameters. The results show that SPPA without gender neutralization performs better than standard SPPA but worse than the original RoBERTa, indicating that both the small subset and its gender-neutralization processing contribute to SPPA's GLUE loss. GEEP without gender neutralization reaches an average of 85.1, compared with 81.4 for SPPA without gender neutralization, suggesting that freezing the pre-trained parameters also helps when the data is small but not debiased. Adding profession embeddings alone does not reproduce GEEP's retention: SPPA-with-NPE averages 80.4, versus 83.3 for GEEP. The table gives all reported GLUE metrics; GN means gender neutralization, and NPE means new profession embeddings.

    Task RoBERTa SPPA GEEP SPPA-without-GN GEEP-without-GN SPPA-with-NPE
    MNLI 87.7 87.2 87.7 87.3 87.7 87.2
    QNLI 92.4 92.4 92.4 92.3 92.4 92.3
    QQP 91.8 91.3 91.7 91.4 91.8 91.5
    SST-2 95.4 94.7 95.4 95.0 95.4 94.7
    CoLA 64.1 38.9 50.5 40.2 59.6 39.3
    MRPC 91.4 88.8 89.8 88.8 90.5 88.8
    RTE 78.4 60.2 68.7 66.4 73.1 61.0
    STS-B 90.7 88.3 89.9 89.5 90.4 88.5
    Average 86.5 80.2 83.3 81.4 85.1 80.4

    SPPA-with-NPE updates both the original model parameters and the added profession embeddings; GEEP freezes the original parameters and updates only the new embeddings.

  8. Knowl 8 — GEEP reduces masked-pronoun gender bias in BERT

    empirical result

    In a direct pronoun-prediction probe, a model is given a template containing a profession and a masked pronoun. The bias score for a profession is the probability of predicting “he” minus the probability of predicting “she”; values closer to zero indicate a smaller preference between those two predictions. Across 303 professions, the average absolute score is 0.44 for BERT-base, 0.16 for BERT-SPPA, and 0.13 for GEEP. The authors report that GEEP reduces this average absolute bias by 70% relative to BERT-base. The plotted examples for 60 professions also illustrate the change: for “nurse,” the score shifts from about −0.7-0.7 for BERT-base to −0.5-0.5 for BERT-SPPA and 0.10.1 for GEEP.

    bias(p)=P(“he”∣p)−P(“she”∣p),\text{bias}(p)=P(\text{“he”}\mid p)-P(\text{“she”}\mid p),

    where pp is a profession in the masked-pronoun template.

  9. Knowl 9 — Preliminary BERT experiments also show coreference gains and reduced forgetting with GEEP

    data/table

    The authors also test GEEP and SPPA on released BERT, using 10,000 second-phase training steps. In this setting the gender-neutral corpus is 7.1 GB and the original BERT pre-training data is 16 GB, both from Wikipedia, so the SPPA forgetting effect is less pronounced than in the RoBERTa experiments. GEEP obtains the highest reported coreference accuracy on all three datasets. On GLUE, its average is 81.6, between BERT-base at 82.0 and BERT-SPPA at 81.3; it improves over SPPA on most listed tasks but not RTE or STS-B. Scores and task metrics are reproduced as reported.

    GLUE task BERT-base BERT-SPPA GEEP
    MNLI 84.3 84.0 84.1
    QNLI 91.4 90.0 91.3
    QQP 90 90.1 90.4
    SST-2 93 92.2 92.4
    CoLA 54.0 52.0 53.0
    MRPC 85.7 84.1 84.9
    RTE 69.4 69.8 69.1
    STS-B 88.0 88.0 87.0
    Average 82.0 81.3 81.6

  10. Knowl 10 — Evaluation is limited to profession-related bias and mostly binary pronoun tasks

    limitation

    The main fairness experiments focus on profession-related gender bias because it is comparatively well studied, and the coreference benchmarks used in those experiments involve binary-gender pronouns. The paper therefore does not establish that GEEP addresses other aspects of gender fairness or represents all gender identities. The authors also limit the study to gender fairness rather than other fairness concerns; applying GEEP to other concerns is presented as a possible future direction, contingent on identifying the relevant biased words.

Coverage note — Omitted the sampled sentence-level grammar audit and the per-profession bias plots beyond their summarized examples; these provide supporting diagnostics rather than additional central results.

References

  1. 1.Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29:4349–4357.
  2. 2.Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165.
  3. 3.Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186.
  4. 4.Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. 2020. Fairfil: Contrastive neural debiasing method for pretrained text encoders. In International Conference on Learning Representations.
  5. 5.Brandon Darr and Tyler Kibbey. 2016. Pronouns and thoughts on neutrality: Gender concerns in modern grammar. Pursuit-The Journal of Undergraduate Research at the University of Tennessee, 7(1):10.
  6. 6.Daniel de Vassimon Manela, David Errington, Thomas Fisher, Boris van Breugel, and Pasquale Minervini. 2021. Stereotype and skew: Quantifying gender bias in pre-trained and fine-tuned language models. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2232–2242.
  7. 7.Madeline B Deutsch and David Buchholz. 2015. Electronic health records and transgender patients—practical recommendations for the collection of gender identity data. Journal of general internal medicine, 30(6):843–847.
  8. 8.J. Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL.
  9. 9.Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16):E3635–E3644.
  10. 10.Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 609–614.
  11. 11.Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. 2020. Spanbert: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77.
  12. 12.James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526.
  13. 13.V Kocijan, O-M Camburu, A-M Cretu, Y Yordanov, P Blunsom, and T Lukasiewicz. 2019. Wikicrem: A large unsupervised corpus for coreference resolution. volume D19-1, page 4294–4303. Association for Computational Linguistics.
  14. 14.Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. In Proceedings of the First Workshop on Gender Bias in Natural Language Processing, pages 166–172.
  15. 15.Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691.
  16. 16.Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The winograd schema challenge. In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning.
  17. 17.Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692.
  18. 18.Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119.
  19. 19.Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543.
  20. 20.Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227–2237.
  21. 21.Yusu Qian, Urwa Muaz, Ben Zhang, and Jae Won Hyun. 2019. Reducing gender bias in word-level language models with a gender-equalizing loss function. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 223–228.
  22. 22.Altaf Rahman and Vincent Ng. 2012. Resolving complex cases of definite pronouns: the winograd schema challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 777–789.
  23. 23.Christina Richards, Walter Pierre Bouman, Leighton Seal, Meg John Barker, Timo O Nieder, and Guy T’Sjoen. 2016. Non-binary or genderqueer genders. International Review of Psychiatry, 28(1):95–102.
  24. 24.Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana. Association for Computational Linguistics.
  25. 25.Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1630–1640.
  26. 26.Tony Sun, Kellie Webster, Apu Shah, William Yang Wang, and Melvin Johnson. 2021. They, them, theirs: Rewriting with gender-neutral english. arXiv preprint arXiv:2102.06788.
  27. 27.Yi Chern Tan and L. Elisa Celis. 2019. Assessing social and intersectional biases in contextualized word representations. In NeurIPS.
  28. 28.Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388–12401. Curran Associates, Inc.
  29. 29.Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355, Brussels, Belgium. Association for Computational Linguistics.
  30. 30.Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the gap: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605–617.
  31. 31.Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032.
  32. 32.Qiyu Wu, Chen Xing, Yatao Li, Guolin Ke, Di He, and Tie-Yan Liu. 2020. Taking notes on the fly helps language pre-training. In International Conference on Learning Representations.
  33. 33.Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 629–634.
  34. 34.Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 15–20, New Orleans, Louisiana. Association for Computational Linguistics.
  35. 35.Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning gender-neutral word embeddings. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4847–4853.

Citation

MLA
Fatemi, Z., et al. “Improving Gender Fairness of Pre-Trained Language Models Without Catastrophic Forgetting”. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2023, pp. 1249–62, https://doi.org/10.18653/v1/2023.acl-short.108.
APA
Fatemi, Z., Xing, C., Liu, W., & Xiong, C. (2023). Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 1249–1262. https://doi.org/10.18653/v1/2023.acl-short.108
Chicago
Fatemi, Z., C. Xing, W. Liu, and C. Xiong. 2023. “Improving Gender Fairness of Pre-Trained Language Models Without Catastrophic Forgetting”. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 1249–62. https://doi.org/10.18653/v1/2023.acl-short.108.
Harvard
Fatemi, Z. et al. (2023) “Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting”, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, pp. 1249–1262. Available at: https://doi.org/10.18653/v1/2023.acl-short.108.
Vancouver
1. Fatemi Z, Xing C, Liu W, Xiong C (2023) Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, pp 1249–1262

BibTeX

@inproceedings{fatemi-etal-2023-improving,
    title = "Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting",
    author = "Fatemi, Zahra  and
      Xing, Chen  and
      Liu, Wenhao  and
      Xiong, Caimming",
    editor = "Rogers, Anna  and
      Boyd-Graber, Jordan  and
      Okazaki, Naoaki",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-short.108/",
    doi = "10.18653/v1/2023.acl-short.108",
    pages = "1249--1262"
}
Metadata:ACL Anthology

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF
License: https://creativecommons.org/licenses/by/4.0/