Back

Language (Technology) is Power: A Critical Survey of “Bias” in NLP

Su Lin BlodgettSolon BarocasHal Daum'eHanna M. Wallach

article2020ACL1,832 citationsBest Paper Award (WNGT @ ACL 2020)

Reveals pervasive conceptual weaknesses across 146 natural language processing bias studies and delivers essential guidelines for aligning technical mitigation methods with normative reasoning and social power dynamics.

Listen

As automated language technologies become increasingly integrated into high-stakes applications such as hiring, moderation, and search, concerns regarding algorithmic fairness have surged. The article addresses the foundational problem that current technical research on "bias" in natural language processing (NLP) often lacks conceptual clarity and rigorous ethical grounding. Without a structured understanding of what makes a system's behavior harmful and to whom, organizations and technologists risk developing superficial technical fixes that fail to prevent real-world harms or protect vulnerable user populations.

The article evaluates how "bias" is conceptualized, motivated, and operationalized across the language processing literature. To achieve this, it presents a comprehensive survey of 146 research publications released prior to May 2020 that focus on analyzing, measuring, or mitigating social bias in written language technologies. The analysis categorizes these works using an established taxonomy of harms, specifically distinguishing between representational harms—such as stereotyping, denigration, and demographic performance disparities—and allocational harms, which involve the unfair distribution of resources or opportunities.

The analysis reveals three central shortcomings across the surveyed literature. First, research motivations are frequently ill-defined and lack normative reasoning: approximately 33% of analyzed works present multiple disparate motivations, 16% offer purely vague motivations or none at all, and 32% frame their work around technical model performance rather than ethical or normative justifications. Second, there is a severe mismatch between stated goals and the quantitative methods used to address them. While 21% of the publications cite allocational harms—such as unfair resume screening—to justify their work, only four papers actually measure or mitigate allocational outcomes directly, focusing instead on narrow token-level associations. Third, the literature largely fails to engage with established scholarship outside of computer science, such as sociolinguistics and critical race studies, resulting in inconsistent definitions of bias even among systems designed for the exact same task.

These findings indicate that existing technical benchmarks provide an incomplete and potentially misleading picture of system fairness. Focusing solely on convenient mathematical formulations without understanding how language reinforces societal inequalities creates significant organizational and compliance risks. For example, language models may inadvertently penalize non-standard dialects, such as African-American English, mischaracterizing benign communication as toxic or low quality. Relying on current debiasing techniques without deeper normative clarity leaves organizations vulnerable to deploying systems that actively perpetuate social marginalization.

To establish a more rigorous and effective path forward, the article outlines three key recommendations. First, research and development must ground technical evaluations in interdisciplinary literature that investigates the relationship between language and social power dynamics, treating representational harms as consequential in their own right. Second, researchers and practitioners must explicitly define their conceptualization of bias, clearly articulating what system behaviors are considered harmful, which groups are affected, and the specific ethical values underlying those determinations. Third, organizations must directly engage the lived experiences of affected communities through participatory methods, critically assessing power dynamics and evaluating whether certain automated systems should be built at all.

The conclusions of the article are based on a systematic evaluation of written text processing across standard academic and industry venues. While the findings are limited by the historical scope ending in early 2020 and do not evaluate spoken language systems, confidence in the diagnosed conceptual and methodological shortcomings remains exceptionally high, warranting deliberate caution among leaders relying on off-the-shelf debiasing benchmarks.

College of Information and Computer SciencesCornell UniversityMicrosoftUniversity of MarylandUniversity of Massachusetts Amherstnatural language processingreasoningrecommendationsocial biasesracial biaslinguistic discrimination
arXiv: 2005.14050
  • Paper: Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings, Tolga Bolukbasi et al. (2016). This paper establishes the foundational methods for quantifying and mitigating geometric gender bias in word embeddings, providing the primary technical paradigm evaluated and critiqued in the survey.
  • Paper: Semantics derived automatically from language corpora contain human-like biases, Aylin Caliskan et al. (2016). This foundational work introduces the Word Embedding Association Test (WEAT) to demonstrate human-like prejudice in text embeddings, serving as a primary target of the survey's critique regarding vague conceptualizations of bias.
  • Paper: Fairness and Abstraction in Sociotechnical Systems, Andrew D. Selbst et al. (2019). This paper identifies key abstraction traps in fair machine learning, supplying the sociotechnical critique of mathematical formalisms that directly motivates the survey's normative analysis.
  • Paper: A Survey on Bias and Fairness in Machine Learning, Ninareh Mehrabi et al. (2019). This survey provides a comprehensive taxonomy of definitions and metrics for bias in machine learning, offering the broader fair-ML context that the survey analyzes within natural language processing.
  • Paper: Big Data's Disparate Impact, Solon Barocas et al. (2016). This work establishes the legal and normative foundations of algorithmic disparate impact, supplying the interdisciplinary reasoning that the survey argues NLP literature urgently lacks.
  • Paper: Model Cards for Model Reporting, Margaret Mitchell et al. (2019). This paper establishes standardized model documentation and disaggregated demographic reporting, laying the practical groundwork for interrogating model harms across affected communities.
  • Paper: Fairness through awareness, Cynthia Dwork et al. (2012). This landmark paper formalizes individual fairness and task-specific similarity metrics, framing the algorithmic non-discrimination theory that underpins downstream bias research in language systems.
Cover for Language (Technology) is Power: A Critical Survey of “Bias” in NLP

Abstract

We survey 146 papers analyzing "bias" in NLP systems, finding that their motivations are often vague, inconsistent, and lacking in normative reasoning, despite the fact that analyzing "bias" is an inherently normative process. We further find that these papers' proposed quantitative techniques for measuring or mitigating "bias" are poorly matched to their motivations and do not engage with the relevant literature outside of NLP. Based on these findings, we describe the beginnings of a path forward by proposing three recommendations that should guide work analyzing "bias" in NLP systems. These recommendations rest on a greater recognition of the relationships between language and social hierarchies, encouraging researchers and practitioners to articulate their conceptualizations of "bias"---i.e., what kinds of system behaviors are harmful, in what ways, to whom, and why, as well as the normative reasoning underlying these statements---and to center work around the lived experiences of members of communities affected by NLP systems, while interrogating and reimagining the power relations between technologists and such communities.

Topics

natural language processingreasoningrecommendation

Keywords

social biasesracial biaslinguistic discriminationlanguage ideologiessociolinguistic justiceabusive language detectionword representationslanguage modelsNLP systems

Table of Contents

  • 1 Introduction
  • 2 Method
  • 3 Findings
  • 3.1 Motivations
  • 3.2 Techniques
  • 4 A path forward
  • 4.1 Language and social hierarchies
  • 4.2 Conceptualizations of “bias”
  • 4.3 Language use in practice
  • 5 Case study
  • 6 Conclusion
  • References
  • A Appendix
  • A.1 Categorization details
  • A.2 Full categorization: Motivations
  • B Full categorization: Techniques

Knowls

  1. Knowl 1 — Taxonomy of Harms for NLP Bias Research

    definition

    An analytical taxonomy categorizes the motivations and quantitative techniques found in natural language processing (NLP) research on social bias into distinct categories of harm:

    • Allocational Harms: Harms that occur when an automated system unfairly distributes resources (such as loans or financial credit) or opportunities (such as employment or housing) among different social groups.
    • Representational Harms: Harms that occur when a system portrays social groups unfavorably, demeans them, or ignores their existence. In NLP bias literature, this is further decomposed into:
      • Stereotyping: System behaviors or representations that reproduce and propagate negative or reductive generalizations about specific social groups.
      • Other Representational Harms: System behaviors resulting in disparate performance across social groups, text generation that misrepresents demographic population distributions, or outputs containing denigrating content.
      • Questionable Correlations: Spurious associations learned by models between system predictions/behaviors and linguistic features or identity tokens commonly associated with particular demographic groups.
    • Vague or Unstated Descriptions: Formulations where the nature of the bias, the mechanism of harm, or the affected population is undefined or presented as self-evident without explicit criteria.
    • Surveys, Frameworks, and Meta-Analyses: Conceptual syntheses, taxonomies, and evaluative reviews of the NLP bias literature.

  2. Knowl 2 — Corpus Distribution Across NLP Tasks in Bias Research

    data/table

    A comprehensive survey of 146 papers published prior to May 2020 analyzing social "bias" in written natural language processing systems reveals the distribution of tasks studied across the discipline:

    NLP Task Papers
    Embeddings (type-level or contextualized) 54
    Coreference resolution 20
    Language modeling or dialogue generation 17
    Hate-speech detection 17
    Sentiment analysis 15
    Machine translation 8
    Tagging or parsing 5
    Surveys, frameworks, and meta-analyses 20
    Other 22

    The total count across categories exceeds 146 because individual papers often evaluate bias across multiple tasks simultaneously (for example, testing debiased word embeddings within a downstream sentiment analysis pipeline). The corpus was gathered from the ACL Anthology, major machine learning and human-computer interaction venues (including ICML, NeurIPS, AIES, FAccT, CHI, and WWW), and arXiv categories (cs.CL and cs.CY), restricted exclusively to written text and social biases.

  3. Knowl 3 — Quantitative Comparison of Motivations and Mitigation Techniques in NLP Bias Research

    data/table

    Across 146 surveyed papers on social bias in NLP, papers exhibit a pronounced discrepancy between the social harms used to motivate the research and the quantitative techniques actually implemented to measure or mitigate bias:

    Harm Category Motivation Papers Technique Papers
    Allocational harms 30 4
    Stereotyping 50 58
    Other representational harms 52 43
    Questionable correlations 47 42
    Vague/unstated 23 0
    Surveys, frameworks, and meta-analyses 20 20

    While 30 papers (20.5%) justify their work by invoking high-stakes allocational harms (such as automated resume filtering or credit scoring disparities), only 4 papers actually evaluate or propose methods targeting allocational harms. Instead, quantitative interventions concentrate overwhelmingly on stereotyping (58 papers) and questionable correlations (42 papers). Furthermore, 23 papers (15.8%) fail to specify any concrete motivation or definition of harm for their technical debiasing proposals.

  4. Knowl 4 — Deficiencies in Normative Reasoning and Harm Motivations in NLP Bias Literature

    empirical result

    An evaluation of stated motivations in 146 NLP bias papers reveals three pervasive conceptual shortcomings:

    1. Absence of Normative Reasoning: 32% of surveyed papers present no normative justification for their work, framing debiasing exclusively as an optimization or error-reduction problem (e.g., maximizing accuracy across datasets) rather than explaining why specific model behaviors constitute social harms.
    2. Conflation of Immediate and Distant Harms: 16% of papers motivate their research by citing distant allocational harms (such as employment discrimination via resume screeners) as downstream consequences of representational harms (such as embedding space geometry), without measuring or establishing causal links to those downstream applications.
    3. Ambiguous and Inconsistent Problem Definitions: 16% of papers provide only vague motivations (e.g., asserting that systems must not "discriminate" or contribute to "social injustice" without defining either term). Additionally, papers addressing the exact same NLP task frequently adopt contradictory, unstated assumptions about what constitutes bias and which stakeholders are harmed.

  5. Knowl 5 — Methodological Shortcomings in Quantitative Bias Techniques

    empirical result

    Quantitative techniques proposed to measure and mitigate bias in NLP systems exhibit three critical limitations:

    1. Isolation from Non-NLP Literature: The majority of quantitative methods fail to engage with established scholarship in sociolinguistics, linguistic anthropology, sociology, or critical race theory. Rare exceptions include adaptations of the Implicit Association Test to word embeddings (the Word Embedding Association Test) and operationalizations of sociological concepts such as the "Angry Black Woman" stereotype, the "double bind" faced by professional women, and Crenshaw's intersectionality framework.
    2. Mismatch with Motivations: Although allocational harms are frequently cited in introductory motivations, 87% of such papers propose techniques that measure only representational disparities or correlation statistics, leaving allocational mechanisms unaddressed.
    3. Narrow Focus on System Predictions: Nearly all proposed methods localize bias exclusively in final model predictions or dataset co-occurrence distributions. They routinely ignore normative decisions embedded across other lifecycle phases, including task formalization, annotation protocols, annotator subjectivity, and evaluation metric design.

  6. Knowl 6 — Recommendation 1: Grounding Bias Analysis in Language Ideologies and Socio-Technical Coproduction

    model/method

    Research on bias in NLP systems should be grounded in interdisciplinary literature (sociolinguistics, linguistic anthropology, and sociology) exploring how language maintains social hierarchies and how technology and social systems are coproduced. This recommendation requires researchers to investigate the following core dimensions:

    • Coproduction Analysis: Investigating how existing racial and social hierarchies drive NLP design decisions and how deployed systems in turn reinforce those hierarchies and ideologies.
    • Lifecycle Assumptions: Auditing the normative assumptions embedded across the development lifecycle, including which dialect or linguistic variety is treated as the default, unmarked "standard"; how task definitions categorize social groups and identities; how annotator demographics and priming impact labeling; and how evaluation metrics hide representational disparities.
    • Language Ideologies: Analyzing whether NLP systems penalize non-standard language practices by labeling them as "noisy text" needing "normalization," thereby legitimizing linguistic deficit perspectives.
    • Intrinsic Representational Harms: Treating representational harms (e.g., misrepresentation, stereotyping, erasure) as harms in their own right, rather than reducing them solely to intermediate precursors of allocational harms.

  7. Knowl 7 — Recommendation 2: Explicit Normative Grounding and Conceptualization of Bias

    model/method

    Work analyzing bias in NLP systems must explicitly state and justify its underlying conceptualization of bias. Because determining what system behaviors are harmful is inherently normative, researchers and practitioners must provide clear answers to three foundational questions:

    1. Target Behaviors and Sources: What specific system behaviors are characterized as "bias," and where do they originate within the pipeline (e.g., general modeling assumptions, task definitions, training corpus selection, annotation schemas)?
    2. Harm Mechanisms and Stakeholders: In what specific ways are these behaviors harmful, to which specific groups or individuals are they harmful, and through what real-world mechanisms does the harm occur?
    3. Underlying Normative Values: What explicit social, political, and ethical values justify designating these behaviors as objectionable?

    Explicitly articulating these choices prevents the conflation of divergent phenomena under ambiguous labels such as "gender bias" or "racial bias" and enables critical comparison of debiasing techniques.

  8. Knowl 8 — Recommendation 3: Centering Lived Experiences and Interrogating Power Relations

    model/method

    NLP bias research must shift away from purely technical, top-down mitigations by centering the lived experiences of communities affected by NLP technologies and restructuring the power dynamics between technologists and these communities:

    • Centering Marginalized Experiences: Because language use is socially situated, system evaluations must prioritize the lived experiences of populations subjected to intersecting axes of oppression.
    • Interrogating System Legitimacy: Rather than assuming that automated systems should always be deployed with technical patches (which preserves developer authority), researchers must critically assess whether certain applications should be designed or built at all.
    • Adopting Participatory Methodologies: Applying established frameworks from Participatory Design, Value-Sensitive Design, Participatory Action Research, and indigenous language reclamation to establish co-equal research partnerships.
    • Power Auditing: Systematically determining whether an NLP application shifts institutional power toward oppressive entities (via surveillance, censorship, or automated gatekeeping) or toward affected communities, and assessing the uncompensated costs borne by groups for whom systems fail.

  9. Knowl 9 — Case Study on African-American English: Raciolinguistic Ideologies and Systemic Harms

    empirical result

    A critical case study of NLP research on African-American English (AAE)—encompassing part-of-speech tagging, parsing, language identification, and toxicity detection—demonstrates how technical debiasing fails when detached from raciolinguistic context:

    • Deficit Framing and De-contextualization: Earlier NLP studies treated lower accuracy on AAE text merely as an out-of-domain performance gap on non-standard text, ignoring the sociolinguistic reality that AAE is a systematic, rule-governed variety and that its speakers experience systemic anti-Black racism.
    • Raciolinguistic Harms in Toxicity Detection: Automated toxicity systems systematically misclassify AAE syntactic and lexical features as toxic or offensive. This creates three distinct harms:
      1. The reinforcement of raciolinguistic ideologies that depict AAE and its speakers as uneducated, improper, or profane;
      2. The disproportionate algorithmic suppression and censorship of Black users on digital platforms; and
      3. The imposition of linguistic accommodation costs, forcing AAE speakers to alter their natural communication to avoid penalization.

    Without directly addressing systemic raciolinguistic ideologies and redistributing power, algorithmic debiasing methods remain unable to prevent technology from reproducing racial hierarchies.

Coverage note — None was omitted; all survey corpus properties, taxonomic categorizations, empirical findings, recommendations, and case study analyses are captured in the knowls.

References

  1. 1.Artem Abzaliev. 2019. On GAP coreference resolution shared task: insights from the 3rd place solution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 107–112, Florence, Italy.
  2. 2.ADA. 2018. Guidelines for Writing About People With Disabilities. ADA National Network. https://bit.ly/2KREbkB.
  3. 3.Oshin Agarwal, Funda Durupinar, Norman I. Badler, and Ani Nenkova. 2019. Word embeddings (also) encode human personality stereotypes. In Proceedings of the Joint Conference on Lexical and Computational Semantics, pages 205–211, Minneapolis, MN.
  4. 4.H. Samy Alim. 2004. You Know My Steez: An Ethnographic and Sociolinguistic Study of Styleshifting in a Black American Speech Community. American Dialect Society.
  5. 5.H. Samy Alim, John R. Rickford, and Arnetha F. Ball, editors. 2016. Raciolinguistics: How Language Shapes Our Ideas About Race. Oxford University Press.
  6. 6.Sandeep Attree. 2019. Gendered ambiguous pronouns shared task: Boosting model confidence by evidence pooling. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, Florence, Italy.
  7. 7.Pinkesh Badjatiya, Manish Gupta, and Vasudeva Varma. 2019. Stereotypical bias removal for hate speech detection task using knowledge-based generalizations. In Proceedings of the International World Wide Web Conference, pages 49–59, San Francisco, CA.
  8. 8.Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. 2019. Differential Privacy Has Disparate Impact on Model Accuracy. In Proceedings of the Conference on Neural Information Processing Systems, Vancouver, Canada.
  9. 9.April Baker-Bell. 2019. Dismantling anti-black linguistic racism in English language arts classrooms: Toward an anti-racist black language pedagogy. Theory Into Practice.
  10. 10.David Bamman, Sejal Popat, and Sheng Shen. 2019. An annotated dataset of literary entities. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 2138–2144, Minneapolis, MN.
  11. 11.Xingce Bao and Qianqian Qiao. 2019. Transfer Learning from Pre-trained BERT for Pronoun Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 82–88, Florence, Italy.
  12. 12.Shaowen Bardzell and Jeffrey Bardzell. 2011. Towards a Feminist HCI Methodology: Social Science, Feminism, and HCI. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), pages 675–684, Vancouver, Canada.
  13. 13.Solon Barocas, Asia J. Biega, Benjamin Fish, J˛edrzej Niklas, and Luke Stark. 2020. When Not to Design, Build, or Deploy. In Proceedings of the Conference on Fairness, Accountability, and Transparency, Barcelona, Spain.
  14. 14.Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The Problem With Bias: Allocative Versus Representational Harms in Machine Learning. In Proceedings of SIGCIS, Philadelphia, PA.
  15. 15.Christine Basta, Marta R. Costa-jussà, and Noe Casas. 2019. Evaluating the underlying gender bias in contextualized word embeddings. In Proceedings of the Workshop on Gender Bias for Natural Language Processing, pages 33–39, Florence, Italy.
  16. 16.John Baugh. 2018. Linguistics in Pursuit of Justice. Cambridge University Press.
  17. 17.Emily M. Bender. 2019. A typology of ethical risks in language technology with an eye towards where transparent documentation can help. Presented at The Future of Artificial Intelligence: Language, Ethics, Technology Workshop. https://bit.ly/2P9t9M6.
  18. 18.Ruha Benjamin. 2019. Race After Technology: Abolitionist Tools for the New Jim Code. John Wiley & Sons.
  19. 19.Ruha Benjamin. 2020. 2020 Vision: Reimagining the Default Settings of Technology & Society. Keynote at ICLR.
  20. 20.Cynthia L. Bennett and Os Keyes. 2019. What is the Point of Fairness? Disability, AI, and The Complexity of Justice. In Proceedings of the ASSETS Workshop on AI Fairness for People with Disabilities, Pittsburgh, PA.
  21. 21.Camiel J. Beukeboom and Christian Burgers. 2019. How Stereotypes Are Shared Through Language: A Review and Introduction of the Social Categories and Stereotypes Communication (SCSC) Framework. Review of Communication Research, 7:1–37.
  22. 22.Shruti Bhargava and David Forsyth. 2019. Exposing and Correcting the Gender Bias in Image Captioning Datasets and Models. arXiv preprint arXiv:1912.00578.
  23. 23.Jayadev Bhaskaran and Isha Bhallamudi. 2019. Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 62–68, Florence, Italy.
  24. 24.Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. Demographic Dialectal Variation in Social Media: A Case Study of African-American English. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 1119–1130, Austin, TX.
  25. 25.Su Lin Blodgett and Brendan O’Connor. 2017. Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English. In Proceedings of the Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML), Halifax, Canada.
  26. 26.Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. Twitter Universal Dependency Parsing for African-American and Mainstream American English. In Proceedings of the Association for Computational Linguistics (ACL), pages 1415–1425, Melbourne, Australia.
  27. 27.Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016a. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. In Proceedings of the Conference on Neural Information Processing Systems, pages 4349–4357, Barcelona, Spain.
  28. 28.Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016b. Quantifying and reducing stereotypes in word embeddings. In Proceedings of the ICML Workshop on #Data4Good: Machine Learning in Social Good Applications, pages 41–45, New York, NY.
  29. 29.Shikha Bordia and Samuel R. Bowman. 2019. Identifying and reducing gender bias in word-level language models. In Proceedings of the NAACL Student Research Workshop, pages 7–15, Minneapolis, MN.
  30. 30.Marc-Etienne Brunet, Colleen Alkalay-Houlihan, Ashton Anderson, and Richard Zemel. 2019. Understanding the Origins of Bias in Word Embeddings. In Proceedings of the International Conference on Machine Learning, pages 803–811, Long Beach, CA.
  31. 31.Mary Bucholtz, Dolores Inés Casillas, and Jin Sook Lee. 2016. Beyond Empowerment: Accompaniment and Sociolinguistic Justice in a Youth Research Program. In Robert Lawson and Dave Sayers, editors, Sociolinguistic Research: Application and Impact, pages 25–44. Routledge.
  32. 32.Mary Bucholtz, Dolores Inés Casillas, and Jin Sook Lee. 2019. California Latinx Youth as Agents of Sociolinguistic Justice. In Netta Avineri, Laura R. Graham, Eric J. Johnson, Robin Conley Riner, and Jonathan Rosa, editors, Language and Social Justice in Practice, pages 166–175. Routledge.
  33. 33.Mary Bucholtz, Audrey Lopez, Allina Mojarro, Elena Skapoulli, Chris VanderStouwe, and Shawn Warner-Garcia. 2014. Sociolinguistic Justice in the Schools: Student Researchers as Linguistic Experts. Language and Linguistics Compass, 8:144–157.
  34. 34.Kaylee Burns, Lisa Anne Hendricks, Kate Saenko, Trevor Darrell, and Anna Rohrbach. 2018. Women also Snowboard: Overcoming Bias in Captioning Models. In Procedings of the European Conference on Computer Vision (ECCV), pages 793–811, Munich, Germany.
  35. 35.Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334).
  36. 36.Kathryn Campbell-Kibler. 2009. The nature of sociolinguistic perception. Language Variation and Change, 21(1):135–156.
  37. 37.Yang Trista Cao and Hal Daumé, III. 2019. Toward gender-inclusive coreference resolution. arXiv preprint arXiv:1910.13913.
  38. 38.Rakesh Chada. 2019. Gendered pronoun resolution using bert and an extractive question answering formulation. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 126–133, Florence, Italy.
  39. 39.Kaytlin Chaloner and Alfredo Maldonado. 2019. Measuring Gender Bias in Word Embedding across Domains and Discovering New Gender Bias Word Categories. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 25–32, Florence, Italy.
  40. 40.Anne H. Charity Hudley. 2017. Language and Racialization. In Ofelia García, Nelson Flores, and Massimiliano Spotti, editors, The Oxford Handbook of Language and Society. Oxford University Press.
  41. 41.Won Ik Cho, Ji Won Kim, Seok Min Kim, and Nam Soo Kim. 2019. On measuring gender bias in translation of gender-neutral pronouns. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 173–181, Florence, Italy.
  42. 42.Shivang Chopra, Ramit Sawhney, Puneet Mathur, and Rajiv Ratn Shah. 2020. Hindi-English Hate Speech Detection: Author Profiling, Debiasing, and Practical Perspectives. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), New York, NY.
  43. 43.Marika Cifor, Patricia Garcia, T.L. Cowan, Jasmine Rault, Tonia Sutherland, Anita Say Chan, Jennifer Rode, Anna Lauren Hoffmann, Niloufar Salehi, and Lisa Nakamura. 2019. Feminist Data Manifest-No. Retrieved from https://www.manifestno.com/.
  44. 44.Patricia Hill Collins. 2000. Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment. Routledge.
  45. 45.Justin T. Craft, Kelly E. Wright, Rachel Elizabeth Weissler, and Robin M. Queen. 2020. Language and Discrimination: Generating Meaning, Perceiving Identities, and Discriminating Outcomes. Annual Review of Linguistics, 6(1).
  46. 46.Kate Crawford. 2017. The Trouble with Bias. Keynote at NeurIPS.
  47. 47.Kimberle Crenshaw. 1989. Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrmination Doctrine, Feminist Theory and Antiracist Politics. University of Chicago Legal Forum.
  48. 48.Amanda Cercas Curry and Verena Rieser. 2018. #MeToo: How Conversational Systems Respond to Sexual Harassment. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 7–14, New Orleans, LA.
  49. 49.Karan Dabas, Nishtha Madaan, Gautam Singh, Vijay Arya, Sameep Mehta, and Tanmoy Chakraborty. 2020. Fair Transfer of Multiple Style Attributes in Text. arXiv preprint arXiv:2001.06693.
  50. 50.Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. Racial bias in hate speech and abusive language detection datasets. In Proceedings of the Workshop on Abusive Language Online, pages 25–35, Florence, Italy.
  51. 51.Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. 2019. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 120–128, Atlanta, GA.
  52. 52.Sunipa Dev, Tao Li, Jeff Phillips, and Vivek Srikumar. 2019. On Measuring and Mitigating Biased Inferences of Word Embeddings. arXiv preprint arXiv:1908.09369.
  53. 53.Sunipa Dev and Jeff Phillips. 2019. Attenuating Bias in Word Vectors. In Proceedings of the International Conference on Artificial Intelligence and Statistics, pages 879–887, Naha, Japan.
  54. 54.Mark Díaz, Isaac Johnson, Amanda Lazar, Anne Marie Piper, and Darren Gergle. 2018. Addressing age-related bias in sentiment analysis. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Montréal, Canada.
  55. 55.Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2019. Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation. arXiv preprint arXiv:1911.03842.
  56. 56.Carl DiSalvo, Andrew Clement, and Volkmar Pipek. 2013. Communities: Participatory Design for, with and by communities. In Jesper Simonsen and Toni Robertson, editors, Routledge International Handbook of Participatory Design, pages 182–209. Routledge.
  57. 57.Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and mitigating unintended bias in text classification. In Proceedings of the Conference on Artificial Intelligence, Ethics, and Society (AIES), New Orleans, LA.
  58. 58.Jacob Eisenstein. 2013. What to do about bad language on the Internet. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 359–369.
  59. 59.Kawin Ethayarajh. 2020. Is Your Classifier Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds. In Proceedings of the Association for Computational Linguistics (ACL).
  60. 60.Kawin Ethayarajh, David Duvenaud, and Graeme Hirst. 2019. Understanding Undesirable Word Embedding Assocations. In Proceedings of the Association for Computational Linguistics (ACL), pages 1696–1705, Florence, Italy.
  61. 61.Joseph Fisher. 2019. Measuring social bias in knowledge graph embeddings. arXiv preprint arXiv:1912.02761.
  62. 62.Nelson Flores and Sofia Chaparro. 2018. What counts as language education policy? Developing a materialist Anti-racist approach to language activism. Language Policy, 17(3):365–384.
  63. 63.Omar U. Florez. 2019. On the Unintended Social Bias of Training Language Generation Models with Data from Local Media. In Proceedings of the NeurIPS Workshop on Human-Centric Machine Learning, Vancouver, Canada.
  64. 64.Joel Escudé Font and Marta R. Costa-jussà. 2019. Equalizing gender biases in neural machine translation with word embeddings techniques. In Proceedings of the Workshop on Gender Bias for Natural Language Processing, pages 147–154, Florence, Italy.
  65. 65.Batya Friedman and David G. Hendry. 2019. Value Sensitive Design: Shaping Technology with Moral Imagination. MIT Press.
  66. 66.Batya Friedman, Peter H. Kahn Jr., and Alan Borning. 2006. Value Sensitive Design and Information Systems. In Dennis Galletta and Ping Zhang, editors, Human-Computer Interaction in Management Information Systems: Foundations, pages 348–372. M.E. Sharpe.
  67. 67.Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018. Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypes. Proceedings of the National Academy of Sciences, 115(16).
  68. 68.Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. 2019. Counterfactual fairness in text classification through robustness. In Proceedings of the Conference on Artificial Intelligence, Ethics, and Society (AIES), Honolulu, HI.
  69. 69.Aparna Garimella, Carmen Banea, Dirk Hovy, and Rada Mihalcea. 2019. Women’s syntactic resilience and men’s grammatical luck: Gender bias in part-of-speech tagging and dependency parsing data. In Proceedings of the Association for Computational Linguistics (ACL), pages 3493–3498, Florence, Italy.
  70. 70.Andrew Gaut, Tony Sun, Shirlyn Tang, Yuxin Huang, Jing Qian, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2020. Towards Understanding Gender Bias in Relation Extraction. In Proceedings of the Association for Computational Linguistics (ACL).
  71. 71.R. Stuart Geiger, Kevin Yu, Yanlai Yang, Mindy Dai, Jie Qiu, Rebekah Tang, and Jenny Huang. 2020. Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From? In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 325–336.
  72. 72.Oguzhan Gencoglu. 2020. Cyberbullying Detection with Fairness Constraints. arXiv preprint arXiv:2005.06625.
  73. 73.Alexandra Reeve Givens and Meredith Ringel Morris. 2020. Centering Disability Perspecives in Algorithmic Fairness, Accountability, and Transparency. In Proceedings of the Conference on Fairness, Accountability, and Transparency, Barcelona, Spain.
  74. 74.Hila Gonen and Yoav Goldberg. 2019. Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 609–614, Minneapolis, MN.
  75. 75.Hila Gonen and Kellie Webster. 2020. Automatically Identifying Gender Issues in Machine Translation using Perturbations. arXiv preprint arXiv:2004.14065.
  76. 76.Ben Green. 2019. “Good” isn’t good enough. In Proceedings of the AI for Social Good Workshop, Vancouver, Canada.
  77. 77.Lisa J. Green. 2002. African American English: A Linguistic Introduction. Cambridge University Press.
  78. 78.Anthony G. Greenwald, Debbie E. McGhee, and Jordan L.K. Schwartz. 1998. Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6):1464–1480.
  79. 79.Enoch Opanin Gyamf, Yunbo Rao, Miao Gou, and Yanhua Shao. 2020. deb2viz: Debiasing gender in word embedding data using subspace visualization. In Proceedings of the International Conference on Graphics and Image Processing.
  80. 80.Foad Hamidi, Morgan Klaus Scheuerman, and Stacy M. Branham. 2018. Gender Recognition or Gender Reductionism? The Social Implications of Automatic Gender Recognition Systems. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Montréal, Canada.
  81. 81.Alex Hanna, Emily Denton, Andrew Smart, and Jamila Smith-Loud. 2020. Towards a Critical Race Methodology in Algorithmic Fairness. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 501–512, Barcelona, Spain.
  82. 82.Madeline E. Heilman, Aaaron S. Wallen, Daniella Fuchs, and Melinda M. Tamkins. 2004. Penalties for Success: Reactions to Women Who Succeed at Male Gender-Typed Tasks. Journal of Applied Psychology, 89(3):416–427.
  83. 83.Jane H. Hill. 2008. The Everyday Language of White Racism. Wiley-Blackwell.
  84. 84.Dirk Hovy, Federico Bianchi, and Tommaso Fornaciari. 2020. Can You Translate that into Man? Commercial Machine Translation Systems Include Stylistic Biases. In Proceedings of the Association for Computational Linguistics (ACL).
  85. 85.Dirk Hovy and Anders Søgaard. 2015. Tagging Performance Correlates with Author Age. In Proceedings of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pages 483–488, Beijing, China.
  86. 86.Dirk Hovy and Shannon L. Spruit. 2016. The social impact of natural language processing. In Proceedings of the Association for Computational Linguistics (ACL), pages 591–598, Berlin, Germany.
  87. 87.Po-Sen Huang, Huan Zhang, Ray Jiang, Robert Stanforth, Johannes Welbl, Jack W. Rae, Vishal Maini, Dani Yogatama, and Pushmeet Kohli. 2019. Reducing Sentiment Bias in Language Models via Counterfactual Evaluation. arXiv preprint arXiv:1911.03064.
  88. 88.Xiaolei Huang, Linzi Xing, Franck Dernoncourt, and Michael J. Paul. 2020. Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition. In Proceedings of the Language Resources and Evaluation Conference (LREC), Marseille, France.
  89. 89.Christoph Hube, Maximilian Idahl, and Besnik Fetahu. 2020. Debiasing Word Embeddings from Sentiment Associations in Names. In Proceedings of the International Conference on Web Search and Data Mining, pages 259–267, Houston, TX.
  90. 90.Ben Hutchinson, Vinodkumar Prabhakaran, Emily Denton, Kellie Webster, Yu Zhong, and Stephen Denuyl. 2020. Social Biases in NLP Models as Barriers for Persons with Disabilities. In Proceedings of the Association for Computational Linguistics (ACL).
  91. 91.Matei Ionita, Yury Kashnitsky, Ken Krige, Vladimir Larin, Dennis Logvinenko, and Atanas Atanasov. 2019. Resolving gendered ambiguous pronouns with BERT. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 113–119, Florence, Italy.
  92. 92.Hailey James-Sorenson and David Alvarez-Melis. 2019. Probabilistic Bias Mitigation in Word Embeddings. In Proceedings of the Workshop on Human-Centric Machine Learning, Vancouver, Canada.
  93. 93.Shengyu Jia, Tao Meng, Jieyu Zhao, and Kai-Wei Chang. 2020. Mitigating Gender Bias Amplification in Distribution by Posterior Regularization. In Proceedings of the Association for Computational Linguistics (ACL).
  94. 94.Taylor Jones, Jessica Rose Kalbfeld, Ryan Hancock, and Robin Clark. 2019. Testifying while black: An experimental study of court reporter accuracy in transcription of African American English. Language, 95(2).
  95. 95.Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2015. Challenges of studying and processing dialects in social media. In Proceedings of the Workshop on Noisy User-Generated Text, pages 9–18, Beijing, China.
  96. 96.Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016. Learning a POS tagger for AAVE-like language. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 1115–1120, San Diego, CA.
  97. 97.Pratik Joshi, Sebastian Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The State and Fate of Linguistic Diversity and Inclusion in the NLP World. In Proceedings of the Association for Computational Linguistics (ACL).
  98. 98.Jaap Jumelet, Willem Zuidema, and Dieuwke Hupkes. 2019. Analysing Neural Language Models: Contextual Decomposition Reveals Default Reasoning in Number and Gender Assignment. In Proceedings of the Conference on Natural Language Learning, Hong Kong, China.
  99. 99.Marie-Odile Junker. 2018. Participatory action research for Indigenous linguistics in the digital age. In Shannon T. Bischoff and Carmen Jany, editors, Insights from Practices in Community-Based Research, pages 164–175. De Gruyter Mouton.
  100. 100.David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. 2017. Incorporating Dialectal Variability for Socially Equitable Language Identification. In Proceedings of the Association for Computational Linguistics (ACL), pages 51–57, Vancouver, Canada.
  101. 101.Masahiro Kaneko and Danushka Bollegala. 2019. Gender-preserving debiasing for pre-trained word embeddings. In Proceedings of the Association for Computational Linguistics (ACL), pages 1641–1650, Florence, Italy.
  102. 102.Saket Karve, Lyle Ungar, and João Sedoc. 2019. Conceptor debiasing of word representations evaluated on WEAT. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 40–48, Florence, Italy.
  103. 103.Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron Tam, Corinne Bintz, Danielle Raz, and P.M. Krafft. 2020. Toward situated interventions for algorithmic equity: lessons from the field. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 45–55, Barcelona, Spain.
  104. 104.Stephen Kemmis. 2006. Participatory action research and the public sphere. Educational Action Research, 14(4):459–476.
  105. 105.Os Keyes. 2018. The Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW).
  106. 106.Os Keyes, Josephine Hoy, and Margaret Drouhard. 2019. Human-Computer Insurrection: Notes on an Anarchist HCI. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Glasgow, Scotland, UK.
  107. 107.Jae Yeon Kim, Carlos Ortiz, Sarah Nam, Sarah Santiago, and Vivek Datta. 2020. Intersectional Bias in Hate Speech and Abusive Language Datasets. In Proceedings of the Association for Computational Linguistics (ACL).
  108. 108.Svetlana Kiritchenko and Saif M. Mohammad. 2018. Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems. In Proceedings of the Joint Conference on Lexical and Computational Semantics, pages 43–53, New Orleans, LA.
  109. 109.Moshe Koppel, Shlomo Argamon, and Anat Rachel Shimoni. 2002. Automatically Categorizing Written Texts by Author Gender. Literary and Linguistic Computing, 17(4):401–412.
  110. 110.Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W. Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. In Proceedings of the Workshop on Gender Bias for Natural Language Processing, pages 166–172, Florence, Italy.
  111. 111.Sonja L. Lanehart and Ayesha M. Malik. 2018. Black Is, Black Isn’t: Perceptions of Language and Blackness. In Jeffrey Reaser, Eric Wilbanks, Karissa Wojcik, and Walt Wolfram, editors, Language Variety in the New South. University of North Carolina Press.
  112. 112.Brian N. Larson. 2017. Gender as a variable in natural-language processing: Ethical considerations. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 30–40, Valencia, Spain.
  113. 113.Anne Lauscher and Goran Glavaš. 2019. Are We Consistently Biased? Multidimensional Analysis of Biases in Distributional Word Vectors. In Proceedings of the Joint Conference on Lexical and Computational Semantics, pages 85–91, Minneapolis, MN.
  114. 114.Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Ivan Vulic. ´ 2019. A General Framework for Implicit and Explicit Debiasing of Distributional Word Vector Spaces. arXiv preprint arXiv:1909.06092.
  115. 115.Christopher A. Le Dantec, Erika Shehan Poole, and Susan P. Wyche. 2009. Values as Lived Experience: Evolving Value Sensitive Design in Support of Value Discovery. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Boston, MA.
  116. 116.Nayeon Lee, Andrea Madotto, and Pascale Fung. 2019. Exploring Social Bias in Chatbots using Stereotype Knowledge. In Proceedings of the Workshop on Widening NLP, pages 177–180, Florence, Italy.
  117. 117.Wesley Y. Leonard. 2012. Reframing language reclamation programmes for everybody’s empowerment. Gender and Language, 6(2):339–367.
  118. 118.Paul Pu Liang, Irene Li, Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2019. Towards Debiasing Sentence Representations. In Proceedings of the NeurIPS Workshop on Human-Centric Machine Learning, Vancouver, Canada.
  119. 119.Rosina Lippi-Green. 2012. English with an Accent: Language, Ideology, and Discrimination in the United States. Routledge.
  120. 120.Bo Liu. 2019. Anonymized BERT: An Augmentation Approach to the Gendered Pronoun Resolution Challenge. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 120–125, Florence, Italy.
  121. 121.Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu, and Jiliang Tang. 2019. Does Gender Matter? Towards Fairness in Dialogue Systems. arXiv preprint arXiv:1910.10486.
  122. 122.Felipe Alfaro Lois, José A.R. Fonollosa, and Costa-jà. 2019. BERT Masked Language Modeling for Coreference Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 76–81, Florence, Italy.
  123. 123.Brandon C. Loudermilk. 2015. Implicit attitudes and the perception of sociolinguistic variation. In Alexei Prikhodkine and Dennis R. Preston, editors, Responses to Language Varieties: Variability, processes and outcomes, pages 137–156.
  124. 124.Anastassia Loukina, Nitin Madnani, and Klaus Zechner. 2019. The many dimensions of algorithmic fairness in educational applications. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, pages 1–10, Florence, Italy.
  125. 125.Kaiji Lu, Peter Mardziel, Fangjing Wu, Preetam Amancharla, and Anupam Datta. 2018. Gender bias in neural natural language processing. arXiv preprint arXiv:1807.11714.
  126. 126.Anne Maass. 1999. Linguistic intergroup bias: Stereotype perpetuation through language. Advances in Experimental Social Psychology, 31:79–121.
  127. 127.Nitin Madnani, Anastassia Loukina, Alina von Davier, Jill Burstein, and Aoife Cahill. 2017. Building Better Open-Source Tools to Support Fairness in Automated Scoring. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 41–52, Valencia, Spain.
  128. 128.Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W. Black. 2019. Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddings. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 801–809, Minneapolis, MN.
  129. 129.Ramón Antonio Martínez and Alexander Feliciano Mejía. 2019. Looking closely and listening carefully: A sociocultural approach to understanding the complexity of Latina/o/x students’ everyday language. Theory Into Practice.
  130. 130.Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, and Simone Teufel. 2019. It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 5270–5278, Hong Kong, China.
  131. 131.Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On Measuring Social Biases in Sentence Encoders. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 629–634, Minneapolis, MN.
  132. 132.Elijah Mayfeld, Michael Madaio, Shrimai Prabhumoye, David Gerritsen, Brittany McLaughlin, Ezekiel Dixon-Roman, and Alan W. Black. 2019. Equity Beyond Bias in Language Technologies for Education. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, Florence, Italy.
  133. 133.Katherine McCurdy and Oguz Serbetçi. 2017. Grammatical gender associations outweigh topical gender bias in crosslinguistic word embeddings. In Proceedings of the Workshop for Women & Underrepresented Minorities in Natural Language Processing, Vancouver, Canada.
  134. 134.Ninareh Mehrabi, Thamme Gowda, Fred Morstatter, Nanyun Peng, and Aram Galstyan. 2019. Man is to Person as Woman is to Location: Measuring Gender Bias in Named Entity Recognition. arXiv preprint arXiv:1910.10872.
  135. 135.Michela Menegatti and Monica Rubini. 2017. Gender bias and sexism in language. In Oxford Research Encyclopedia of Communication. Oxford University Press.
  136. 136.Inom Mirzaev, Anthony Schulte, Michael Conover, and Sam Shah. 2019. Considerations for the interpretation of bias measures of word embeddings. arXiv preprint arXiv:1906.08379.
  137. 137.Salikoko S. Mufwene, Guy Bailey, and John R. Rickford, editors. 1998. African-American English: Structure, History, and Use. Routledge.
  138. 138.Michael J. Muller. 2007. Participatory Design: The Third Space in HCI. In The Human-Computer Interaction Handbook, pages 1087–1108. CRC Press.
  139. 139.Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456.
  140. 140.Dong Nguyen, Rilana Gravel, Dolf Trieschnigg, and Theo Meder. 2013. “How Old Do You Think I Am?”: A Study of Language and Age in Twitter. In Proceedings of the Conference on Web and Social Media (ICWSM), pages 439–448, Boston, MA.
  141. 141.Malvina Nissim, Rik van Noord, and Rob van der Goot. 2020. Fair is better than sensational: Man is to doctor as woman is to doctor. Computational Linguistics.
  142. 142.Debora Nozza, Claudia Volpetti, and Elisabetta Fersini. 2019. Unintended Bias in Misogyny Detection. In Proceedings of the Conference on Web Intelligence, pages 149–155.
  143. 143.Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. 2019. Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries. Frontiers in Big Data, 2.
  144. 144.Alexandra Olteanu, Kartik Talamadupula, and Kush R. Varshney. 2017. The Limits of Abstract Evaluation Metrics: The Case of Hate Speech Detection. In Proceedings of the ACM Web Science Conference, Troy, NY.
  145. 145.Orestis Papakyriakopoulos, Simon Hegelich, Juan Carlos Medina Serrano, and Fabienne Marco. 2020. Bias in word embeddings. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 446–457, Barcelona, Spain.
  146. 146.Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing Gender Bias in Abusive Language Detection. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 2799–2804, Brussels, Belgium.
  147. 147.Ellie Pavlick and Tom Kwiatkowski. 2019. Inherent Disagreements in Human Textual Inferences. Transactions of the Association for Computational Linguistics, 7:677–694.
  148. 148.Xiangyu Peng, Siyan Li, Spencer Frazier, and Mark Riedl. 2020. Fine-Tuning a Transformer-Based Language Model to Avoid Generating Non-Normative Text. arXiv preprint arXiv:2001.08764.
  149. 149.Radomir Popovic,´ Florian Lemmerich, and Markus Strohmaier. 2020. Joint Multiclass Debiasing of Word Embeddings. In Proceedings of the International Symposium on Intelligent Systems, Graz, Austria.
  150. 150.Vinodkumar Prabhakaran, Ben Hutchinson, and Margaret Mitchell. 2019. Perturbation Sensitivity Analysis to Detect Unintended Model Biases. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 5744–5749, Hong Kong, China.
  151. 151.Shrimai Prabhumoye, Elijah Mayfeld, and Alan W. Black. 2019. Principled Frameworks for Evaluating Ethics in NLP Systems. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, Florence, Italy.
  152. 152.Marcelo Prates, Pedro Avelar, and Luis C. Lamb. 2019. Assessing gender bias in machine translation: A case study with google translate. Neural Computing and Applications.
  153. 153.Rasmus Précenth. 2019. Word embeddings and gender stereotypes in Swedish and English. Master’s thesis, Uppsala University.
  154. 154.Dennis R. Preston. 2009. Are you really smart (or stupid, or cute, or ugly, or cool)? Or do you just talk that way? Language attitudes, standardization and language change. Oslo: Novus forlag, pages 105–129.
  155. 155.Flavien Prost, Nithum Thain, and Tolga Bolukbasi. 2019. Debiasing Embeddings for Reduced Gender Bias in Text Classification. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 69–75, Florence, Italy.
  156. 156.Reid Pryzant, Richard Diehl Martinez, Nathan Dass, Sadao Kurohashi, Dan Jurafsky, and Diyi Yang. 2020. Automatically Neutralizing Subjective Bias in Text. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), New York, NY.
  157. 157.Arun K. Pujari, Ansh Mittal, Anshuman Padhi, Anshul Jain, Mukesh Jadon, and Vikas Kumar. 2019. Debiasing Gender biased Hindi Words with Word-embedding. In Proceedings of the International Conference on Algorithms, Computing and Artificial Intelligence, pages 450–456.
  158. 158.Yusu Qian, Urwa Muaz, Ben Zhang, and Jae Won Hyun. 2019. Reducing gender bias in word-level language models with a gender-equalizing loss function. In Proceedings of the ACL Student Research Workshop, pages 223–228, Florence, Italy.
  159. 159.Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. In Proceedings of the Association for Computational Linguistics (ACL).
  160. 160.John R. Rickford and Sharese King. 2016. Language and linguistics on trial: Hearing Rachel Jeantel (and other vernacular speakers) in the courtroom and beyond. Language, 92(4):948–988.
  161. 161.Anthony Rios. 2020. FuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifiers on African-American English. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), New York, NY.
  162. 162.Gerald Roche. 2019. Articulating language oppression: colonialism, coloniality and the erasure of TibetâA˘ Zs minority languages. Patterns of Prejudice. ´
  163. 163.Alexey Romanov, Maria De-Arteaga, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, Anna Rumshisky, and Adam Tauman Kalai. 2019. What’s in a Name? Reducing Bias in Bios without Access to Protected Attributes. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 4187–4195, Minneapolis, MN.
  164. 164.Jonathan Rosa. 2019. Contesting Representations of Migrant “Illegality” through the Drop the I-Word Campaign: Rethinking Language Change and Social Change. In Netta Avineri, Laura R. Graham, Eric J. Johnson, Robin Conley Riner, and Jonathan Rosa, editors, Language and Social Justice in Practice. Routledge.
  165. 165.Jonathan Rosa and Christa Burdick. 2017. Language Ideologies. In Ofelia García, Nelson Flores, and Massimiliano Spotti, editors, The Oxford Handbook of Language and Society. Oxford University Press.
  166. 166.Jonathan Rosa and Nelson Flores. 2017. Unsettling race and language: Toward a raciolinguistic perspective. Language in Society, 46:621–647.
  167. 167.Sara Rosenthal and Kathleen McKeown. 2011. Age Prediction in Blogs: A Study of Style, Content, and Online Behavior in Pre- and Post-Social Media Generations. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 763–772, Portland, OR.
  168. 168.Candace Ross, Boris Katz, and Andrei Barbu. 2020. Measuring Social Biases in Grounded Vision and Language Embeddings. arXiv preprint arXiv:2002.08911.
  169. 169.Richard Rothstein. 2017. The Color of Law: A Forgotten History of How Our Government Segregated America. Liveright Publishing.
  170. 170.David Rozado. 2020. Wide range screening of algorithmic bias in word embedding models using large sentiment lexicons reveals underreported bias types. PLOS One.
  171. 171.Elayne Ruane, Abeba Birhane, and Anthony Ventresque. 2019. Conversational AI: Social and Ethical Considerations. In Proceedings of the Irish Conference on Artificial Intelligence and Cognitive Science, Galway, Ireland.
  172. 172.Rachel Rudinger, Chandler May, and Benjamin Van Durme. 2017. Social bias in elicited natural language inferences. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 74–79, Valencia, Spain.
  173. 173.Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender Bias in Coreference Resolution. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 8–14, New Orleans, LA.
  174. 174.Elizabeth B.N. Sanders. 2002. From user-centered to participatory design approaches. In Jorge Frascara, editor, Design and the Social Sciences: Making Connections, pages 18–25. CRC Press.
  175. 175.Brenda Salenave Santana, Vinicius Woloszyn, and Leandro Krug Wives. 2018. Is there gender bias and stereotype in Portuguese word embeddings? In Proceedings of the International Conference on the Computational Processing of Portuguese Student Research Workshop, Canela, Brazil.
  176. 176.Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the Association for Computational Linguistics (ACL), pages 1668–1678, Florence, Italy.
  177. 177.Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. Social Bias Frames: Reasoning about Social and Power Implications of Language. In Proceedings of the Association for Computational Linguistics (ACL).
  178. 178.Hanna Sassaman, Jennifer Lee, Jenessa Irvine, and Shankar Narayan. 2020. Creating Community-Based Tech Policy: Case Studies, Lessons Learned, and What Technologists and Communities Can Do Together. In Proceedings of the Conference on Fairness, Accountability, and Transparency, Barcelona, Spain.
  179. 179.Danielle Saunders and Bill Byrne. 2020. Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem. In Proceedings of the Association for Computational Linguistics (ACL).
  180. 180.Tyler Schnoebelen. 2017. Goal-Oriented Design for Ethical Machine Learning and NLP. In Proceedings of the Workshop on Ethics in Natural Language Processing, pages 88–93, Valencia, Spain.
  181. 181.Sabine Sczesny, Magda Formanowicz, and Franziska Moser. 2016. Can gender-fair language reduce gender stereotyping and discrimination? Frontiers in Psychology, 7.
  182. 182.João Sedoc and Lyle Ungar. 2019. The Role of Protected Class Word Lists in Bias Identification of Contextualized Word Representations. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 55–61, Florence, Italy.
  183. 183.Procheta Sen and Debasis Ganguly. 2020. Towards Socially Responsible AI: Cognitive Bias-Aware Multi-Objective Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), New York, NY.
  184. 184.Deven Shah, H. Andrew Schwartz, and Dirk Hovy. 2020. Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview. In Proceedings of the Association for Computational Linguistics (ACL).
  185. 185.Judy Hanwen Shen, Lauren Fratamico, Iyad Rahwan, and Alexander M. Rush. 2018. Darling or Babygirl? Investigating Stylistic Bias in Sentiment Analysis. In Proceedings of the Workshop on Fairness, Accountability, and Transparency (FAT/ML), Stockholm, Sweden.
  186. 186.Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The Woman Worked as a Babysitter: On Biases in Language Generation. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 3398–3403, Hong Kong, China.
  187. 187.Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2020. Towards Controllable Biases in Language Generation. arXiv preprint arXiv:2005.00268.
  188. 188.Seungjae Shin, Kyungwoo Song, JoonHo Jang, Hyemi Kim, Weonyoung Joo, and Il-Chul Moon. 2020. Neutralizing Gender Bias in Word Embedding with Latent Disentanglement and Counterfactual Generation. arXiv preprint arXiv:2004.03133.
  189. 189.Jesper Simonsen and Toni Robertson, editors. 2013. Routledge International Handbook of Participatory Design. Routledge.
  190. 190.Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the Association for Computational Linguistics (ACL), pages 1679–1684, Florence, Italy.
  191. 191.Yolande Strengers, Lizhe Qu, Qiongkai Xu, and Jarrod Knibbe. 2020. Adhering, Steering, and Queering: Treatment of Gender in Natural Language Generation. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Honolulu, HI.
  192. 192.Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating Gender Bias in Natural Language Processing: Literature Review. In Proceedings of the Association for Computational Linguistics (ACL), pages 1630–1640, Florence, Italy.
  193. 193.Adam Sutton, Thomas Lansdall-Welfare, and Nello Cristianini. 2018. Biased embeddings from wild data: Measuring, understanding and removing. In Proceedings of the International Symposium on Intelligent Data Analysis, pages 328–339, ’s-Hertogenbosch, Netherlands.
  194. 194.Chris Sweeney and Maryam Najafan. 2019. A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings. In Proceedings of the Association for Computational Linguistics (ACL), pages 1662–1667, Florence, Italy.
  195. 195.Chris Sweeney and Maryam Najafan. 2020. Reducing sentiment polarity for demographic attributes in word embeddings using adversarial learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 359–368, Barcelona, Spain.
  196. 196.Nathaniel Swinger, Maria De-Arteaga, Neil Thomas Heffernan, Mark D.M. Leiserson, and Adam Tauman Kalai. 2019. What are the biases in my word embedding? In Proceedings of the Conference on Artificial Intelligence, Ethics, and Society (AIES), Honolulu, HI.
  197. 197.Samson Tan, Shafq Joty, Min-Yen Kan, and Richard Socher. 2020. It’s Morphin’ Time! Combating Linguistic Discrimination with Infectional Perturbations. In Proceedings of the Association for Computational Linguistics (ACL).
  198. 198.Yi Chern Tan and L. Elisa Celis. 2019. Assessing Social and Intersectional Biases in Contextualized Word Representations. In Proceedings of the Conference on Neural Information Processing Systems, Vancouver, Canada.
  199. 199.J. Michael Terry, Randall Hendrick, Evangelos Evangelou, and Richard L. Smith. 2010. Variable dialect switching among African American children: Inferences about working memory. Lingua, 120(10):2463–2475.
  200. 200.Joel Tetreault, Daniel Blanchard, and Aoife Cahill. 2013. A Report on the First Native Language Identification Shared Task. In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, pages 48–57, Atlanta, GA.
  201. 201.Mike Thelwall. 2018. Gender Bias in Sentiment Analysis. Online Information Review, 42(1):45–57.
  202. 202.Kristen Vaccaro, Karrie Karahalios, Deirdre K. Mulligan, Daniel Kluttz, and Tad Hirsch. 2019. Contestability in Algorithmic Systems. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing, pages 523–527, Austin, TX.
  203. 203.Ameya Vaidya, Feng Mai, and Yue Ning. 2019. Empirical Analysis of Multi-Task Learning for Reducing Model Bias in Toxic Comment Detection. arXiv preprint arXiv:1909.09758v2.
  204. 204.Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting Gender Right in Neural Machine Translation. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 3003–3008, Brussels, Belgium.
  205. 205.Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias. arXiv preprint arXiv:2004.12265.
  206. 206.Tianlu Wang, Xi Victoria Lin, Nazneen Fatema Rajani, Bryan McCann, Vicente Ordonez, and Caiming Xiong. 2020. Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation. In Proceedings of the Association for Computational Linguistics (ACL).
  207. 207.Zili Wang. 2019. MSnet: A BERT-based Network for Gendered Pronoun Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 89–95, Florence, Italy.
  208. 208.Kellie Webster, Marta R. Costa-jussà, Christian Hardmeier, and Will Radford. 2019. Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 1–7, Florence, Italy.
  209. 209.Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the GAP: A balanced corpus of gendered ambiguous pronouns. Transactions of the Association for Computational Linguistics, 6:605–618.
  210. 210.Walt Wolfram and Natalie Schilling. 2015. American English: Dialects and Variation, 3 edition. Wiley Blackwell.
  211. 211.Austin P. Wright, Omar Shaikh, Haekyu Park, Will Epperson, Muhammed Ahmed, Stephane Pinel, Diyi Yang, and Duen Horng (Polo) Chau. 2020. RECAST: Interactive Auditing of Automatic Toxicity Detection Models. In Proceedings of the Conference on Human Factors in Computing Systems (CHI), Honolulu, HI.
  212. 212.Yinchuan Xu and Junlin Yang. 2019. Look again at the syntax: Relational graph convolutional network for gendered ambiguous pronoun resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 96–101, Florence, Italy.
  213. 213.Kai-Chou Yang, Timothy Niven, Tzu-Hsuan Chou, and Hung-Yu Kao. 2019. Fill the GAP: Exploiting BERT for Pronoun Resolution. In Proceedings of the Workshop on Gender Bias in Natural Language Processing, pages 102–106, Florence, Italy.
  214. 214.Zekun Yang and Juan Feng. 2020. A Causal Inference Method for Reducing Gender Bias in Word Embedding Relations. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), New York, NY.
  215. 215.Daisy Yoo, Anya Ernest, Sofia Serholt, Eva Eriksson, and Peter Dalsgaard. 2019. Service Design in HCI Research: The Extended Value Co-creation Model. In Proceedings of the Halfway to the Future Symposium, Nottingham, United Kingdom.
  216. 216.Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the Conference on Artificial Intelligence, Ethics, and Society (AIES), New Orleans, LA.
  217. 217.Guanhua Zhang, Bing Bai, Junqi Zhang, Kun Bai, Conghui Zhu, and Tiejun Zhao. 2020a. Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting. In Proceedings of the Association for Computational Linguistics (ACL).
  218. 218.Haoran Zhang, Amy X. Lu, Mohamed Abdalla, Matthew McDermott, and Marzyeh Ghassemi. 2020b. Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings. In Proceedings of the ACM Conference on Health, Inference, and Learning.
  219. 219.Jieyu Zhao, Subhabrata Mukherjee, Saghar Hosseini, Kai-Wei Chang, and Ahmed Hassan Awadallah. 2020. Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer. In Proceedings of the Association for Computational Linguistics (ACL).
  220. 220.Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender Bias in Contextualized Word Embeddings. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 629–634, Minneapolis, MN.
  221. 221.Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 2979–2989, Copenhagen, Denmark.
  222. 222.Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018a. Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods. In Proceedings of the North American Association for Computational Linguistics (NAACL), pages 15–20, New Orleans, LA.
  223. 223.Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018b. Learning Gender-Neutral Word Embeddings. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 4847–4853, Brussels, Belgium.
  224. 224.Alina Zhiltsova, Simon Caton, and Catherine Mulwa. 2019. Mitigation of Unintended Biases against Non-Native English Texts in Sentiment Analysis. In Proceedings of the Irish Conference on Artificial Intelligence and Cognitive Science, Galway, Ireland.
  225. 225.Pei Zhou, Weijia Shi, Jieyu Zhao, Kuan-Hao Huang, Muhao Chen, and Kai-Wei Chang. 2019. Examining gender bias in languages with grammatical genders. In Proceedings of Empirical Methods in Natural Language Processing (EMNLP), pages 5279–5287, Hong Kong, China.
  226. 226.Ran Zmigrod, S. J. Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In Proceedings of the Association for Computational Linguistics (ACL), pages 1651–1661, Florence, Italy.

Citation

MLA
Blodgett, S. L., et al. “Language (Technology) Is Power: A Critical Survey of "Bias" in NLP”. arXiv, 2020, http://arxiv.org/abs/2005.14050v2.
APA
Blodgett, S. L., Barocas, S., Daumé, H., & Wallach, H. (2020). Language (Technology) is Power: A Critical Survey of "Bias" in NLP. arXiv. http://arxiv.org/abs/2005.14050v2
Chicago
Blodgett, S. L., S. Barocas, H. Daumé, and H. Wallach. 2020. “Language (Technology) Is Power: A Critical Survey of "Bias" in NLP”. arXiv. http://arxiv.org/abs/2005.14050v2.
Harvard
Blodgett, S.L. et al. (2020) “Language (Technology) is Power: A Critical Survey of "Bias" in NLP”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2005.14050v2.
Vancouver
1. Blodgett SL, Barocas S, Daumé H, Wallach H (2020) Language (Technology) is Power: A Critical Survey of "Bias" in NLP. arXiv

BibTeX

@article{blodgett2020language,
  title = {Language (Technology) is Power: A Critical Survey of "Bias" in NLP},
  author = {Blodgett, Su Lin and Barocas, Solon and Daumé, Hal and Wallach, Hanna},
  year = {2020},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2005.14050v2},
  eprint = {2005.14050}
}
Metadata:arXiv

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: https://creativecommons.org/licenses/by/4.0/