4. Bias in AI¶
An AI system learns whatever pattern actually exists in its training data — including patterns that reflect real-world unfairness. This isn't a bug that shows up occasionally; it's a direct, built-in consequence of how training (Lesson 3) works.
Where bias comes from¶
| Source | Example |
|---|---|
| Unrepresentative data | A face-recognition system trained mostly on one group performs worse on everyone else |
| Historical bias baked into past decisions | A hiring tool trained on a company's past hiring data can learn to favor whatever group was historically hired most, regardless of actual qualification |
| Missing data | A medical AI trained mostly on one population can perform worse diagnosing people outside it |
The common thread: the model doesn't invent bias — it reproduces and can even amplify bias that was already present in its training data, often invisibly, since nothing about the training process flags "this pattern is unfair" versus "this pattern is useful."
Whose data, and with what consent?¶
A related question matters just as much: where did the training data actually come from, and did the people in it agree to it being used that way? A dataset of people's faces or voices, collected without real consent, raises serious ethical problems even before bias enters the picture at all. Unit 2 puts this into practice directly — when you choose what to train your own model on.
Try it
A school attendance system uses face recognition trained mostly on photos of adults. Predict one way it might perform worse for students, and explain why, using the "unrepresentative data" row above.
Tip
"Whose data was this trained on, and who might be underrepresented in it?" is a genuinely useful question to ask about almost any AI system you encounter — including ones you build yourself.