Writing · 2024
How Machines Learn to Discriminate
Embedding fairness into machine-learning algorithms — how models trained on real-world data automate existing bias along race and sex, even absent any ill intent. And what it takes to push back.
The Thesis
Nobody has to design a discriminatory algorithm. Inspired by Professor Lise Getoor's Ethics & Algorithms course and Ruha Benjamin's The New Jim Code, this talk shows how models trained on past real-world data tend to automate — and amplify — existing discrimination on the basis of race and sex. Bias doesn't have to be written in; it's inherited from the data we feed the machine.
The Feedback Loop
Data collected from the real world carries the real world's bias — and algorithms develop by training on that data. When the data is biased, the algorithm inherits the bias. Then its decisions flow back into the world, generating the next round of biased data.
Note: this is how biased algorithms don't just reflect systems of oppression — they reinforce them.
Data Bias in Action
The same photo of a hand holding a thermometer, run through Google Vision — once as-is, once with the skin tone edited light. The dark-skinned hand is labeled “Gun” at 61% confidence; the light-skinned hand, “Monocular” at 60%. Nobody programmed that. The training data did [1].
The Lack-of-Transparency Problem
The model can't articulate why its predictions are what they are — and when it encodes bias, that opacity hides it. Transparency about who designs these systems is what makes accountability possible.
What Is Fairness, Anyway?
Fairness can be defined many ways. The talk focuses on two families — and they pull in different directions:
Individual Fairness
People are treated equally to one another despite varying protected traits. Prioritizes equal treatment — the same rule for everybody.
Group Fairness
People are treated equitably to one another despite varying protected traits. Prioritizes equal outcome — the results even out across groups.
The Fairness Playground
Two groups of candidates with identical true talent. But history depressed the recorded scores of most of group B — the world was biased, so the data is biased. Now you run the hiring model. Pick a policy and watch who gets in, who gets wrongly rejected, and what each definition of fairness trades away.
An illustrative simulation — 160 synthetic candidates, same talent distribution in both groups. Fairness definitions from Fu, Aseri, Singh & Srinivasan [3].
Equality, Equity, Justice
The classic picture, and the reason “treat everyone the same” isn't the end of the conversation [2]:
Correcting Bias in the Process
Pre-Processing
Detect and remove bias in the data where possible, add corrective sampling, and embed features carefully before training ever starts.
Processing
Evaluate the model's rankings for bias while it works — and search for inference: is it reconstructing protected attributes from proxies?
Post-Processing
Examine causal relations in the outputs and verify results for the accuracy-versus-fairness trade-off before anything ships.
Question Your Own Bias
The people building the pipeline are part of the pipeline. Fairness constraints in the algorithm can't compensate for never asking the question.
The Takeaway
There is no one correct way to define fairness. Equal treatment and equal outcome are both principled — and, as the playground above shows, they can be mutually exclusive on biased data. The choice between them is not a technical decision; it's an ethical one. The danger isn't choosing wrong — it's letting the data choose by default.
References
- Kayser-Bril, N. (2020). Google apologizes after its Vision AI produced racist results. AlgorithmWatch.
- MobilizeGreen (2021). Environmental equity vs. environmental justice.
- Fu, R., Aseri, M., Singh, P. V., & Srinivasan, K. (2021). “Un”Fair Machine Learning Algorithms. Management Science.