Skip to content

3. Testing and Iterating

A model that performs badly almost always has a specific, fixable cause — rarely "AI just doesn't work."

Common causes of a poorly-performing model

Symptom Likely cause
Confuses two classes constantly The classes are too visually similar, or samples for each overlap too much
Works great in testing, fails on anything slightly different Not enough variety in training samples (same angle, same background, same lighting every time)
One class is predicted almost always, regardless of input That class has far more samples than the others — an unbalanced dataset
Performs worse on certain subjects within a class Those subjects were underrepresented in your training samples — this is Unit 1, Lesson 4's bias lesson, now something you can actually observe in a model you built yourself

The fix is always the same shape

  1. Identify which specific case above you're seeing.
  2. Add, remove, or rebalance samples to address it directly.
  3. Retrain.
  4. Test again, with new examples.

Try it

Take your Lesson 2 model and deliberately make one class's training samples much less varied than the other (e.g. all the same angle and background). Retrain, test with varied new images, and see the accuracy actually drop — then fix it and confirm it improves.

Tip

More samples alone doesn't fix every problem — more varied samples usually matters more than more total samples.

Next: Confidence Scores and Uncertainty →