Back

Algorithmic Decision Making and the Cost of Fairness

Sam Corbett-DaviesEmma PiersonAvi FellerSharad GoelAziz Huq

article2017KDD1,423 citations

Demonstrates that standard algorithmic fairness constraints force decision-makers to apply race-specific risk thresholds, revealing a critical practical trade-off between public safety and statistical equality in criminal risk assessments.

Listen

Algorithmic decision tools are increasingly deployed across the criminal justice system to guide pretrial release and detention decisions. However, widespread concern exists that these tools exacerbate racial disparities, as evidenced by findings that black defendants are substantially more likely than white defendants to be incorrectly flagged as high risk. The article evaluates mathematical formulations of algorithmic fairness, demonstrating the fundamental trade-offs between maximizing public safety and satisfying popular criteria designed to eliminate demographic disparities.

To conduct this evaluation, the authors formulated algorithmic fairness as a constrained optimization problem, balancing public safety benefits against the costs of detention. They proved mathematical theorems regarding optimal decision rules under three common fairness definitions: statistical parity, conditional statistical parity, and predictive equality. They then empirically tested these rules on a dataset of 3,377 pretrial defendants from Broward County, Florida, comparing outcomes against an unconstrained public safety baseline.

Key findings show that the optimal unconstrained strategy requires a single, uniform risk threshold applied equally to all defendants, which inherently maximizes public safety and treats individuals equally regardless of race. In contrast, mathematically satisfying any of the three fairness criteria requires setting race-specific decision thresholds. Applying these race-specific constraints causes significant, measurable public safety losses. In the Broward County data, enforcing statistical parity requires detaining 17% low-risk defendants and results in an estimated 9% increase in violent recidivism among releasees relative to the safety-optimal benchmark. Enforcing predictive equality results in 14% of detainees being low risk and a 7% increase in violent crime, while conditional statistical parity yields 10% low-risk detainees and a 4% violent crime increase. Conversely, an unconstrained single threshold that detains 30% of defendants overall leads to stark racial disparities, detaining 40% of black defendants compared to 18% of white defendants and producing a 32% false positive rate for blacks versus 14% for whites.

These findings establish that tensions between public safety and algorithmic fairness are mathematically inevitable whenever underlying risk distributions differ across demographic groups. Furthermore, applying race-specific decision thresholds carries severe legal risks in the United States, as holding individuals of different races to different standards would likely trigger strict scrutiny under the Equal Protection Clause. The authors also show that testing for score calibration is insufficient to detect discriminatory modeling practices, because strategically adding noise can artificially depress scores for favored groups while preserving calibration.

Policymakers and system leaders should avoid treating fairness metrics as simple algorithmic fixes and instead address systemic disparities through broader policy levers. Practical next steps include improving risk model accuracy with higher-quality data to reduce overall error rates, adjusting detention thresholds upward to lower total incarceration, and investing in non-custodial interventions such as community supervision and supportive services to lower the social costs of classification errors. System designers must also subject predictive algorithms themselves to rigorous inspection rather than evaluating only output calibration.

These conclusions rest on the key assumption that two-year violent arrest data accurately reflects true recidivism risk, though systemic arrest patterns may introduce observational bias. The immediate utility framework also focuses on proximate safety and detention costs rather than long-term systemic effects. Despite these limitations, there is high confidence in the core finding that mathematical fairness constraints inherently force significant trade-offs against public safety in risk-based decision systems.

Stanford UniversityUniversity of California BerkeleyUniversity of Chicagoconstrained optimizationdecision tablesoptimizationalgorithmic decision makingalgorithmic fairnessfairness constraints
arXiv: 1701.082305harad/cost-of-fairness
  • Paper: Inherent Trade-Offs in the Fair Determination of Risk Scores, Jon Kleinberg et al. (2017). This paper establishes the mathematical impossibility of simultaneously satisfying calibration and error-rate balance across demographic groups, providing the core theoretical tension analyzed in the source.
  • Paper: Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments, Alexandra Chouldechova (2017). This study introduces the empirical analysis of calibration versus error rate trade-offs in recidivism risk scoring using Broward County data, directly motivating the constrained optimization formulation in the source.
  • Paper: Equality of Opportunity in Supervised Learning, Moritz Hardt et al. (2016). This work formalizes equalized odds and equal opportunity via group-specific threshold adjustments, which serve as the primary fairness definitions evaluated in the source.
  • Paper: Fairness through awareness, Cynthia Dwork et al. (2012). This foundational paper introduces formal definitions of individual and group fairness as optimization constraints, establishing the conceptual paradigm used by the source.
  • Paper: Certifying and Removing Disparate Impact, Michael Feldman et al. (2014). This paper establishes the statistical criteria and data repair methods for disparate impact, providing the foundation for demographic parity constraints analyzed in the source.
  • Paper: Big Data's Disparate Impact, Solon Barocas et al. (2016). This paper details the legal and structural mechanisms through which data-driven models produce disparate impact, providing necessary legal context for fairness constraints.
  • Paper: A Reductions Approach to Fair Classification, Alekh Agarwal et al. (2018). This paper generalizes the constrained optimization approach of fair classification by framing fairness constraints as a systematic reduction to cost-sensitive learning for general black-box predictors.
  • Paper: Delayed Impact of Fair Machine Learning, Lydia T. Liu et al. (2018). This work extends static threshold-optimization models by examining how fairness constraints and utility-maximizing policies dynamically impact group welfare and score distributions over time.
  • Paper: Fairness and Abstraction in Sociotechnical Systems, Andrew D. Selbst et al. (2019). This paper critiques the formalistic abstraction of fairness optimization in criminal justice by analyzing the sociotechnical traps of reducing justice to isolated mathematical thresholds.
  • Paper: AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias, Rachel Bellamy et al. (2019). This work operationalizes the suite of fairness metrics and post-processing threshold adjustments explored in the source into an open-source software library.
  • Paper: A Survey on Bias and Fairness in Machine Learning, Ninareh Mehrabi et al. (2019). This comprehensive survey contextualizes the trade-offs between accuracy, public safety, and mathematical fairness definitions across the broader machine learning lifecycle.
Cover for Algorithmic Decision Making and the Cost of Fairness

Abstract

Algorithms are now regularly used to decide whether defendants awaiting trial are too dangerous to be released back into the community. In some cases, black defendants are substantially more likely than white defendants to be incorrectly classified as high risk. To mitigate such disparities, several techniques recently have been proposed to achieve algorithmic fairness. Here we reformulate algorithmic fairness as constrained optimization: the objective is to maximize public safety while satisfying formal fairness constraints designed to reduce racial disparities. We show that for several past definitions of fairness, the optimal algorithms that result require detaining defendants above race-specific risk thresholds. We further show that the optimal unconstrained algorithm requires applying a single, uniform threshold to all defendants. The unconstrained algorithm thus maximizes public safety while also satisfying one important understanding of equality: that all individuals are held to the same standard, irrespective of race. Because the optimal constrained and unconstrained algorithms generally differ, there is tension between improving public safety and satisfying prevailing notions of algorithmic fairness. By examining data from Broward County, Florida, we show that this trade-off can be large in practice. We focus on algorithms for pretrial release decisions, but the principles we discuss apply to other domains, and also to human decision makers carrying out structured decision rules.

Topics

constrained optimizationdecision tablesoptimizationtradeclassifiersadaptive algorithmsproportional fairnessethnicitydesign of algorithmsbinary decision

Keywords

algorithmic decision makingalgorithmic fairnessfairness constraintsoptimal decision rulestaste-based discriminationpredictive equalitydisparate impactequal false positive ratesracial disparitiesthreshold rules

Table of Contents

  • 1 Introduction
  • 2 Background
  • 2.1 Defining algorithmic fairness
  • 2.2 Related work
  • 3 Optimal decision rules
  • 4 The cost of fairness
  • 5 The cost of public safety
  • 6 Detecting discrimination
  • 7 Discussion
  • References

Knowls

  1. Knowl 1 — Form and Optimality of Threshold Rules Under Fairness Constraints

    theoretical result

    Let X∈RpX \in \mathbb{R}^p denote visible features, g(X)∈{g1,…,gk}g(X) \in \{g_1, \dots, g_k\} denote protected group membership (such as race), Y∈{0,1}Y \in \{0, 1\} denote the binary outcome, and pY∣X=Pr⁡(Y=1∣X)p_{Y|X} = \Pr(Y = 1 \mid X) denote the probability of reoffending. Assume that the distribution D(pY∣X)\mathcal{D}(p_{Y|X}) has a strictly positive density on [0,1][0, 1].

    The decision rules d∗:Rp→[0,1]d^*: \mathbb{R}^p \to [0, 1] that maximize the immediate utility u(d,c)=E[d(X)(pY∣X−c)]u(d, c) = \mathbb{E}[d(X)(p_{Y|X} - c)] for a cost parameter c∈(0,1)c \in (0, 1) subject to various fairness criteria are unique (up to sets of probability zero) and have the following forms:

    1. Unconstrained Optimum: d∗(X)={1if pY∣X≥c0otherwised^*(X) = \begin{cases} 1 & \text{if } p_{Y|X} \ge c \\ 0 & \text{otherwise} \end{cases}

    2. Statistical Parity (satisfying E[d(X)∣g(X)]=E[d(X)]\mathbb{E}[d(X) \mid g(X)] = \mathbb{E}[d(X)]): d∗(X)={1if pY∣X≥tg(X)0otherwised^*(X) = \begin{cases} 1 & \text{if } p_{Y|X} \ge t_{g(X)} \\ 0 & \text{otherwise} \end{cases} where each tg(X)∈[0,1]t_{g(X)} \in [0, 1] is a constant threshold specific to group g(X)g(X).

    3. Predictive Equality (satisfying E[d(X)∣Y=0,g(X)]=E[d(X)∣Y=0]\mathbb{E}[d(X) \mid Y = 0, g(X)] = \mathbb{E}[d(X) \mid Y = 0], equating false positive rates across groups): d∗(X)={1if pY∣X≥tg(X)′0otherwised^*(X) = \begin{cases} 1 & \text{if } p_{Y|X} \ge t'_{g(X)} \\ 0 & \text{otherwise} \end{cases} where each tg(X)′∈[0,1]t'_{g(X)} \in [0, 1] is a group-specific constant threshold (with values generally different from the statistical parity thresholds).

    4. Conditional Statistical Parity (satisfying E[d(X)∣ℓ(X),g(X)]=E[d(X)∣ℓ(X)]\mathbb{E}[d(X) \mid \ell(X), g(X)] = \mathbb{E}[d(X) \mid \ell(X)], where ℓ:Rp→Rm\ell: \mathbb{R}^p \to \mathbb{R}^m is a projection onto legitimate risk factors, assuming D(pY∣X∣ℓ(X)=l)\mathcal{D}(p_{Y|X} \mid \ell(X) = l) has positive density on [0,1][0, 1] for all ll): d∗(X)={1if pY∣X≥tg(X),ℓ(X)0otherwised^*(X) = \begin{cases} 1 & \text{if } p_{Y|X} \ge t_{g(X), \ell(X)} \\ 0 & \text{otherwise} \end{cases} where the thresholds tg(X),ℓ(X)∈[0,1]t_{g(X), \ell(X)} \in [0, 1] depend on both group membership g(X)g(X) and legitimate attributes ℓ(X)\ell(X).

    Because optimal constrained rules require group-specific thresholds while the unconstrained safety-maximizing rule applies a single uniform threshold cc, satisfying these fairness criteria necessarily requires sacrificing public safety compared to the unconstrained optimum.

  2. Knowl 2 — Immediate Utility Formulation for Binary Decision Rules

    definition

    Define a binary decision rule d:Rp→[0,1]d: \mathbb{R}^p \to [0, 1], where d(x)d(x) represents the probability of taking action a1a_1 (e.g., detention) rather than action a0a_0 (e.g., release) for an individual with visible attributes x∈Rpx \in \mathbb{R}^p. Let Y∈{0,1}Y \in \{0, 1\} be the binary indicator of the benefit of taking action a1a_1 relative to a0a_0 (e.g., Y=1Y = 1 if the individual would commit a violent crime if released, and Y=0Y = 0 otherwise). Let X∈RpX \in \mathbb{R}^p and Y∈{0,1}Y \in \{0, 1\} be random variables for a randomly selected individual from the target population, and let pY∣X=Pr⁡(Y=1∣X)p_{Y|X} = \Pr(Y = 1 \mid X).

    For a detention cost parameter cc satisfying 0<c<10 < c < 1 (representing the cost of detention in units of violent crime prevented), the immediate utility u(d,c)u(d, c) of decision rule dd is defined as: u(d,c)=E[Yd(X)−cd(X)]=E[Yd(X)]−cE[d(X)]u(d, c) = \mathbb{E}[Y d(X) - c d(X)] = \mathbb{E}[Y d(X)] - c \mathbb{E}[d(X)]

    By conditioning on XX, the immediate utility can be rewritten as: u(d,c)=E[d(X)(pY∣X−c)]u(d, c) = \mathbb{E}[d(X)(p_{Y|X} - c)]

    This formulation indicates that detaining an individual increases immediate utility if and only if their recidivism probability exceeds the cost threshold, pY∣X>cp_{Y|X} > c. The metric reflects proximate benefits (crimes prevented, E[Yd(X)]\mathbb{E}[Y d(X)]) and costs (number of individuals detained, E[d(X)]\mathbb{E}[d(X)]) while assuming uniform detention costs cc across individuals and equal costs for all violent offenses.

  3. Knowl 3 — Duality Between Cost-Weighted Utility Maximization and Fixed Detention Rate Optimization

    theoretical result

    Let D\mathcal{D} be the family of decision rules satisfying statistical parity, conditional statistical parity, predictive equality, or the unconstrained set of all valid binary decision rules d:Rp→[0,1]d: \mathbb{R}^p \to [0, 1].

    There exists a bijection f:[0,1]→[0,1]f: [0, 1] \to [0, 1] such that for any detention cost parameter c∈(0,1)c \in (0, 1): arg⁡max⁡d∈DE[Yd(X)−cd(X)]=arg⁡max⁡d∈DE[d(X)]=f(c)E[Yd(X)]\arg\max_{d \in \mathcal{D}} \mathbb{E}[Y d(X) - c d(X)] = \arg\max_{\substack{d \in \mathcal{D} \\ \mathbb{E}[d(X)] = f(c)}} \mathbb{E}[Y d(X)] where the equivalence of maximizers is defined up to a set of probability zero.

    The bijection f(c)=E[d∗(X)]f(c) = \mathbb{E}[d^*(X)] represents the expected proportion of the population detained under the utility-maximizing rule d∗d^*. Because ff is strictly decreasing and continuous with f(0)=1f(0) = 1 and f(1)=0f(1) = 0, maximizing immediate utility with detention cost cc is formally dual to maximizing public safety (expected violent crimes prevented E[Yd(X)]\mathbb{E}[Y d(X)]) subject to detaining an exact target proportion α=f(c)\alpha = f(c) of defendants alongside the specified fairness constraint.

  4. Knowl 4 — Empirical Public Safety Costs of Fairness Constraints on Broward County Pretrial Data

    data/table

    The trade-offs between public safety and algorithmic fairness constraints were evaluated on a dataset of 3,377 Black and White pretrial defendants in Broward County, Florida. The binary outcome Y∈{0,1}Y \in \{0, 1\} indicates two-year violent recidivism. Risk scores pY∣X=Pr⁡(Y=1∣X)p_{Y|X} = \Pr(Y = 1 \mid X) were estimated using L1L_1-regularized logistic regression with Platt scaling on non-race features (achieving test set AUC 0.75, compared to 0.73 for COMPAS scores).

    Decision rules were constrained to detain an overall fraction of 30% of defendants, matching the proportion classified as medium or high risk (score ≥5\ge 5) by COMPAS. Thresholds were optimized to maximize public safety subject to detaining 30% under each fairness constraint. Performance metrics were averaged over 100 random 70/30 train-test splits.

    Fairness Constraint Percent of Detainees That Are Low Risk Estimated Increase in Violent Crime
    Statistical parity 17% 9%
    Predictive equality 14% 7%
    Conditional statistical parity 10% 4%

    In the table:

    • "Percent of detainees that are low risk" measures the fraction of detained defendants under the constrained rule who would have been released under an unconstrained safety-maximizing single-threshold rule.
    • "Estimated increase in violent crime" measures the percentage increase in violent recidivism among released defendants relative to the unconstrained safety-maximizing single-threshold rule.
    • Conditional statistical parity treated prior convictions as the legitimate factor ℓ(X)\ell(X), partitioned into four bins (00, 1–21\text{--}2, 3–43\text{--}4, and ≥5\ge 5).

    Enforcing fairness constraints requires releasing high-risk defendants and detaining lower-risk defendants to balance group rates, resulting in a 4%–9%4\%\text{--}9\% increase in violent crime committed by released individuals.

  5. Knowl 5 — Infra-Marginality and Disparities Induced by Single-Threshold Rules

    empirical result

    When an unconstrained safety-maximizing decision rule applies a single uniform threshold tt across all individuals (d(X)=1pY∣X≥td(X) = \mathbf{1}_{p_{Y|X} \ge t}), it typically violates statistical parity, predictive equality, and conditional statistical parity across demographic groups.

    This disparity arises from infra-marginality: group-level detention rates E[d(X)∣g(X)=gi]=Pr⁡(pY∣X≥t∣g(X)=gi)\mathbb{E}[d(X) \mid g(X) = g_i] = \Pr(p_{Y|X} \ge t \mid g(X) = g_i) and false positive rates E[d(X)∣Y=0,g(X)=gi]=Pr⁡(pY∣X≥t∣Y=0,g(X)=gi)\mathbb{E}[d(X) \mid Y = 0, g(X) = g_i] = \Pr(p_{Y|X} \ge t \mid Y = 0, g(X) = g_i) depend on the entire distribution of risk scores above threshold tt within each group. Even when two groups have identical average risk (E[pY∣X∣g(X)=g1]=E[pY∣X∣g(X)=g2]\mathbb{E}[p_{Y|X} \mid g(X) = g_1] = \mathbb{E}[p_{Y|X} \mid g(X) = g_2]), differences in higher-order moments (such as a heavier upper tail in one group) cause substantial differences in detention and false positive rates under identical thresholds.

    In the Broward County pretrial dataset:

    • Optimally detaining 30% of defendants using a single uniform threshold detains 40% of Black defendants compared to 18% of White defendants, violating statistical parity.
    • Among defendants who do not reoffend (Y=0Y = 0), the false positive detention rate is 32% for Black defendants compared to 14% for White defendants, violating predictive equality.

  6. Knowl 6 — Insufficiency of Score Calibration to Prevent or Detect Algorithmic Discrimination

    theoretical result

    A risk scoring function s:Rp→Rs: \mathbb{R}^p \to \mathbb{R} is calibrated across demographic groups g(X)g(X) if: Pr⁡(Y=1∣s(X),g(X))=Pr⁡(Y=1∣s(X))\Pr(Y = 1 \mid s(X), g(X)) = \Pr(Y = 1 \mid s(X)) While the true probabilities pY∣X=Pr⁡(Y=1∣X)p_{Y|X} = \Pr(Y = 1 \mid X) are inherently calibrated, verifying that a score s(X)s(X) is calibrated is insufficient to establish that the score is unbiased or non-discriminatory.

    A decision maker can construct calibrated scores that systematically discriminate against or favor a specific group:

    1. Mean-zero noise (e.g., N(0,σ2)\mathcal{N}(0, \sigma^2)) is added to the true risk predictors or scores of a favored demographic group.
    2. A predictive model is trained using these perturbed features to estimate the outcome YY.
    3. Because adding noise removes predictive information, the predicted scores for the favored group shrink toward their group mean.
    4. The resulting score distribution remains fully calibrated according to the definition above, but because extreme high scores are eliminated for the favored group, zero individuals in that group fall above a uniform decision threshold tt.

    Consequently, discrimination can occur through selective omission or degradation of informative features for one group even under a facially neutral, single-threshold policy applied to calibrated scores. Detecting such discrimination requires auditing the underlying feature inputs and model development process rather than inspecting score calibration alone.

  7. Knowl 7 — Limitations of Objective Modeling and Feature Bias in Pretrial Risk Assessment

    limitation

    Applying constrained optimization of immediate utility to algorithmic decision making involves several key limitations:

    1. Biased Risk Proxies: The target outcome YY reflects arrest for violent offenses rather than actual violent behavior. Differential policing intensities across neighborhoods can inflate arrest rates for minority defendants relative to White defendants with identical conduct, biasing risk estimates pY∣Xp_{Y|X}.
    2. Subgroup Validity and Information Omission: Feature predictive validity may differ across demographic subgroups. Excluding features to prevent disparate treatment or address subgroup invalidity can degrade model accuracy and inadvertently cause information loss analogous to redlining.
    3. Uniform Cost Assumptions: The model assumes a constant detention cost cc across all defendants, failing to account for idiosyncratic individual costs such as dependent child care obligations.
    4. Proximate vs. Long-Term Utility: Immediate utility u(d,c)=E[Yd(X)−cd(X)]u(d, c) = \mathbb{E}[Y d(X) - c d(X)] accounts only for proximate violent crimes prevented and individuals detained, ignoring the long-term economic, social, and intergenerational effects of detention.
    5. Group-Level Selection Objectives: The decision framework evaluates individuals independently and does not model group-level collective objectives (such as community representation or diversity in admissions settings).
    6. Constitutional Constraints: Implementing the optimal race-specific thresholds required by statistical parity or predictive equality involves explicit racial classifications, which trigger strict scrutiny review under the Equal Protection Clause of the Fourteenth Amendment in U.S. law.

Coverage note — No substantial contributed material from the paper was omitted. The knowls cover the immediate utility formulation, the form and optimality of threshold rules under fairness constraints (Theorem 3.2), the equivalence duality proposition (Proposition 3.3), the empirical evaluation and trade-off data on Broward County, the infra-marginality mechanism, the insufficiency of calibration for discrimination detection, and the stated limitations of the decision-making framework.

References

  1. 1.Lindsay C Ahlman and Ellen M Kurtz. 2009. Œe APPD randomized controlled trial in low risk supervision: Œe e‘ects on low risk supervision on rearrest. Philadelphia Adult Probation and Parole Department (2009).
  2. 2.Julia Angwin, Je‘ Larson, Surya MaŠu, and Lauren Kirchner. 2016. Machine bias: Œere’s so‰ware used across the country to predict future criminals. and it’s biased against blacks. ProPublica (5 2016).
  3. 3.Shamena Anwar and Hanming Fang. 2006. An Alternative Test of Racial Prejudice in Motor Vehicle Searches: Œeory and Evidence. Še American Economic Review (2006).
  4. 4.Ian Ayres. 2002. Outcome tests of racial disparities in police practices. Justice research and Policy 4, 1-2 (2002), 131–142.
  5. 5.Solon Barocas and Andrew D Selbst. 2016. Big data’s disparate impact. California Law Review 104 (2016).
  6. 6.Gary S Becker. 1957. Še economics of discrimination. University of Chicago Press.
  7. 7.Richard Berk. 2009. Œe role of race in forecasts of violent crime. Race and Social Problems (2009), 131–242.
  8. 8.Richard Berk. 2012. Criminal justice forecasts of risk: a machine learning approach. Springer Science & Business Media.
  9. 9.Richard Berk, Hoda Heidari, Shahin Jabbari, Michael Kearns, and Aaron Roth. 2017. Fairness in Criminal Justice Risk Assessments: Œe State of the Art. working paper (2017).
  10. 10.Richard Berk, Lawrence Sherman, Geo‘rey Barnes, Ellen Kurtz, and Lindsay Ahlman. 2009. Forecasting murder within a population of probationers and parolees: a high stakes application of statistical learning. Journal of the Royal Statistical Society: Series A (Statistics in Society) 172, 1 (2009), 191–211.
  11. 11.James A Berkovec, Glenn B Canner, Stuart A Gabriel, and Timothy H Hannan. 1994. Race, redlining, and residential mortgage loan performance. Še Journal of Real Estate Finance and Economics 9, 3 (1994), 263–294.
  12. 12.Alexandra Chouldechova. 2016. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. arXiv preprint arXiv:1610.07524 (2016).
  13. 13.Angele Christin, Alex Rosenblat, and danah boyd. 2015. Courts and Predictive ` Algorithms. Data & Civil Rights: Criminal Justice and Civil Rights Primer (2015).
  14. 14.Mona Danner, Marie VanNostrand, and Lisa Spruance. August, 2015. Risk-Based Pretrial Release Recommendation and Supervision Guidelines. (August, 2015).
  15. 15.Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd Innovations in Šeoretical Computer Science Conference. ACM, 214–226.
  16. 16.Michael Feldman, Sorelle A Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 259–268.
  17. 17.Benjamin Fish, Jeremy Kun, and Adam D aniel Lelkes. 2016. A Con€dence-Based  Approach for Balancing Fairness and Accuracy. CoRR abs/1601.05764 (2016).
  18. 18.Fisher v. University of Texas at Austin. 2016. 136 S. Ct. 2198, 2208 (2016).
  19. 19.Sharad Goel, Justin Rao, and Ravi Shro‘. 2016. Personalized Risk Assessments in the Criminal Justice System. Še American Economic Review 106, 5 (2016), 119–123.
  20. 20.Sharad Goel, Justin M Rao, and Ravi Shro‘. 2016. Precinct or Prejudice? Understanding Racial Disparities in New York City’s Stop-and-Frisk Policy. Annals of Applied Statistics 10, 1 (2016), 365–394.
  21. 21.Sara Hajian, Josep Domingo-Ferrer, and Antoni Martinez-Balleste. 2011. Discrimination prevention in data mining for intrusion and crime detection. In Computational Intelligence in Cyber Security (CICS), 2011 IEEE Symposium on. IEEE, 47–54.
  22. 22.Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. In Advances In Neural Information Processing Systems. 3315–3323.
  23. 23.Eric Holder. 2014. (2014). Remarks at the National Association of Criminal Defense Lawyers 57th Annual Meeting.
  24. 24.Shahin Jabbari, MaŠhew Joseph, Michael Kearns, Jamie Morgenstern, and Aaron Roth. 2016. Fair Learning in Markovian Environments. arXiv preprint arXiv:1611.03071 (2016).
  25. 25.MaŠhew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. 2016. Fairness in Learning: Classic and Contextual Bandits. In Advances in Neural Information Processing Systems 29. Curran Associates, Inc., 325–333.
  26. 26.Jongbin Jung, Connor Concannon, Ravi Shro‘, Sharad Goel, and Daniel G. Goldstein. 2017. Simple rules for complex decisions. (2017). Working paper.
  27. 27.Faisal Kamiran, Indre˙ Zliobait ˇ e, and Toon Calders. 2013. ‹antifying explainable ˙ discrimination and removing illegal discrimination in automated decision making. Knowledge and information systems 35, 3 (2013), 613–644.
  28. 28.Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2012. Fairness-aware classi€er with prejudice remover regularizer. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 35–50.
  29. 29.Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent trade-o‘s in the fair determination of risk scores. Proceedings of Innovations in Šeoretical Computer Science (ITCS) (2017).
  30. 30.Je‘ Larson, Surya MaŠu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica (5 2016).
  31. 31.Kristian Lum and William Isaac. 2016. To predict and serve? Signi€cance 13, 5 (2016), 14–19.
  32. 32.J Monahan and JL Skeem. 2016. Risk Assessment in Criminal Sentencing. Annual review of clinical psychology 12 (2016), 489.
  33. 33.ScoŠ E Page. 2008. Še di‘erence: How the power of diversity creates beˆer groups, €rms, schools, and societies. Princeton University Press.
  34. 34.Emma Pierson, Sam CorbeŠ-Davies, and Sharad Goel. 2017. Fast threshold tests for detecting discrimination. (2017). Working paper available at hŠps: //arxiv.org/abs/1702.08536.
  35. 35.John C. PlaŠ. 1999. Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. In Advances in Large Margin Classi€ers. 61–74.
  36. 36.Andrea Romei and Salvatore Ruggieri. 2014. A multidisciplinary survey on discrimination analysis. Še Knowledge Engineering Review 29, 05 (2014), 582–638.
  37. 37.Camelia Simoiu, Sam CorbeŠ-Davies, and Sharad Goel. 2017. Œe problem of infra-marginality in outcome tests for discrimination. Annals of Applied Statistics (2017). Forthcoming.
  38. 38.Jennifer L. Skeem and Christopher T. Lowencamp. 2016. Risk, Race, and Recidivism: Predictive Bias and Disparate Impact. Criminology 54, 4 (2016), 680–712.
  39. 39.Sonja B Starr. 2014. Evidence-based sentencing and the scienti€c rationalization of discrimination. Stan. L. Rev. 66 (2014), 803.
  40. 40.Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2017. Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classi€cation without Disparate Mistreatment. In Proceedings of the 26th International World Wide Web Conference.
  41. 41.Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning Fair Representations. In Proceedings of Še 30th International Conference on Machine Learning. 325–333.
  42. 42.Indre Zliobaite. 2017. Measuring discrimination in algorithmic decision making. Data Mining and Knowledge Discovery (2017).

Citation

MLA
Corbett-Davies, S., et al. “Algorithmic Decision Making and the Cost of Fairness”. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 797–806, https://doi.org/10.1145/3097983.3098095.
APA
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017). Algorithmic Decision Making and the Cost of Fairness. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 797–806. https://doi.org/10.1145/3097983.3098095
Chicago
Corbett-Davies, S., E. Pierson, A. Feller, S. Goel, and A. Huq. 2017. “Algorithmic Decision Making and the Cost of Fairness”. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 797–806. https://doi.org/10.1145/3097983.3098095.
Harvard
Corbett-Davies, S. et al. (2017) “Algorithmic Decision Making and the Cost of Fairness”, Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, pp. 797–806. Available at: https://doi.org/10.1145/3097983.3098095.
Vancouver
1. Corbett-Davies S, Pierson E, Feller A, Goel S, Huq A (2017) Algorithmic Decision Making and the Cost of Fairness. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, pp 797–806

BibTeX

@inproceedings{Corbett_Davies_2017, series={KDD ’17}, title={Algorithmic Decision Making and the Cost of Fairness}, url={http://dx.doi.org/10.1145/3097983.3098095}, DOI={10.1145/3097983.3098095}, booktitle={Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining}, publisher={ACM}, author={Corbett-Davies, Sam and Pierson, Emma and Feller, Avi and Goel, Sharad and Huq, Aziz}, year={2017}, month=Aug, pages={797–806}, collection={KDD ’17} }
Metadata:Crossref

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF