3. Testing and Iterating¶
A model that performs badly almost always has a specific, fixable cause — rarely "AI just doesn't work."
Common causes of a poorly-performing model¶
| Symptom | Likely cause |
|---|---|
| Confuses two classes constantly | The classes are too visually similar, or samples for each overlap too much |
| Works great in testing, fails on anything slightly different | Not enough variety in training samples (same angle, same background, same lighting every time) |
| One class is predicted almost always, regardless of input | That class has far more samples than the others — an unbalanced dataset |
| Performs worse on certain subjects within a class | Those subjects were underrepresented in your training samples — this is Unit 1, Lesson 4's bias lesson, now something you can actually observe in a model you built yourself |
The fix is always the same shape¶
- Identify which specific case above you're seeing.
- Add, remove, or rebalance samples to address it directly.
- Retrain.
- Test again, with new examples.
Try it
Take your Lesson 2 model and deliberately make one class's training samples much less varied than the other (e.g. all the same angle and background). Retrain, test with varied new images, and see the accuracy actually drop — then fix it and confirm it improves.
Tip
More samples alone doesn't fix every problem — more varied samples usually matters more than more total samples.