4. Overfitting¶
Unit 2, Lesson 3 described a model that "works great in testing, fails on anything slightly different" as a symptom. This lesson shows you the actual mechanism behind it.
Training loss vs. test loss¶
Playground tracks two separate loss numbers: how wrong the network is on the data it's training on, and how wrong it is on held-back data it hasn't trained on. A healthy training run brings both down together. Overfitting is when training loss keeps dropping while test loss stops improving or gets worse — the network is memorizing the specific training examples instead of learning the general pattern.
Making it happen on purpose¶
- Pick a dataset and turn the noise slider up, so the data is messier and less cleanly separable.
- Add several extra hidden layers and neurons — more than the problem actually needs.
- Train for a long time.
- Watch the decision boundary twist itself into an oddly specific shape that closely wraps individual noisy points, rather than a clean general boundary.
That overly-twisted boundary is overfitting made visible — a simpler boundary would actually generalize better to new points, even though it fits the noisy training data slightly worse.
Try it
Deliberately cause overfitting using the steps above, and take note of what happens to the gap between training loss and test loss as it happens.
Tip
An overly complex decision boundary that bends around every noisy data point isn't a sign of a "smarter" network — it's usually a sign of a network that's stopped generalizing.