ITD140 · Fairness & Bias in ML. Two families of fairness definitions, how each is measured, and why they can pull against each other. Use the sliders, then test yourself after each section.
A model learns patterns from data. When that data reflects historical or structural inequities — or is measured unevenly across groups — the model can reproduce or amplify those patterns, even when the protected attribute (race, sex, age, etc.) is never used as an input.
Because "fair" has no single mathematical definition, researchers formalized several. They fall into two broad families: group fairness (compare outcomes across groups) and individual fairness (compare outcomes between similar people). The rest of this page contrasts them.
1. A hiring model is trained on 10 years of a company's hires, which historically skewed heavily toward one demographic. The model is accurate on this data but recommends mostly that same demographic. This is best described as:
2. Removing the protected attribute (e.g., race) from the model's inputs guarantees the model is fair.
3. Using "number of prior arrests" as a feature to predict "criminality," when arrest rates differ across groups for reasons unrelated to underlying behavior, is an example of:
Group fairness asks: does the model produce comparable outcomes across groups defined by a protected attribute? It is measured with statistics computed per group. Common criteria:
P(Ŷ=1 | A=a) is the same for every group a.Below, two groups apply for a loan. Group B's scores sit lower on average — imagine measurement bias, not a difference in true creditworthiness. Move each group's decision threshold and watch the acceptance rates and the demographic-parity gap.
1. Demographic parity is satisfied when:
2. In the widget above, if Group B scores lower on average, how can you reach demographic parity?
3. Which criterion requires equal true-positive AND false-positive rates across groups?
Demographic parity only looks at who gets accepted. It ignores whether those decisions are correct. A model could hit perfect parity while accepting the wrong people in one group. Equalized odds fixes attention on accuracy per group: it asks for equal true-positive rates (qualified people accepted) and equal false-positive rates (unqualified people wrongly accepted) across groups.
For groups A and B, equalized odds holds when both of these match:
TPRA = TPRB and FPRA = FPRB
Equal opportunity is the weaker cousin: it requires only the true-positive rates to match. Requiring both rates is what makes equalized odds strict.
Below, each group has 100 applicants — half genuinely qualified (positives), half not (negatives) — but Group B's scores are shifted lower, and shifted unevenly for its qualified vs. unqualified applicants. Move each threshold and try to match both rates at once.
1. Equalized odds requires that, across groups, the model has equal:
2. "Equal opportunity" differs from equalized odds because it requires only:
3. In the widget, matching the TPR gap to ~0 often leaves the FPR gap non-zero. The lesson is that:
Individual fairness (Dwork et al., 2012) takes a different view: similar individuals should be treated similarly. Rather than comparing group averages, it constrains the model at the level of pairs of people.
Formally, for a similarity metric d on individuals and a model f, individual fairness asks that outputs not change faster than inputs (a Lipschitz condition):
| f(x) − f(y) | ≤ L · d(x, y)
Two people who are close on the legitimate features (d(x,y) small) must receive close outputs. The hard part: choosing d — a bad similarity metric can smuggle bias back in.
Below, Individual A is fixed. Slide how different Individual B is on the legitimate features (d), and set the tolerance L. The green region is everything the rule permits. Turn on the biased model with B in a different protected group and watch the point jump above the line — the violation is worst when the two people are most alike.
1. The core principle of individual fairness is:
2. In the widget, the biased model's point sits above the allowed line only when d is small. Why does bias violate individual fairness most for similar people?
3. The hardest practical part of applying individual fairness is:
The two families answer different questions, so satisfying one need not satisfy the other — and sometimes they directly conflict.
The slider below morphs from a single group-blind threshold (left) toward per-group thresholds chosen to equalize acceptance rates (right). Watch the group gap shrink while a pair of "twin" applicants — same score, different groups — eventually gets split apart.
1. Using different decision thresholds per group to achieve demographic parity can violate individual fairness because:
2. A single group-blind threshold tends to:
3. When groups have different base rates, the impossibility results tell us that:
4. Which statement best summarizes the relationship between the two families?