5. Hidden Layers and Complexity¶
Some patterns genuinely need more network to solve — but "more" always comes with trade-offs from Lesson 4.
Try a hard dataset with too little network¶
Switch to the Spiral dataset — a shape no single straight line can cleanly separate — and leave the network at just one hidden layer with one neuron. Train it. It will struggle or fail outright: too little capacity to represent a pattern this complex, a problem called underfitting.
Add capacity until it works¶
Add more neurons to the hidden layer, and add more hidden layers, retraining as you go. At some point, the network gains enough capacity to actually trace the spiral's shape in its decision boundary.
The trade-off¶
| Too little capacity | Too much capacity |
|---|---|
| Underfitting — can't represent the real pattern at all | Overfitting risk (Lesson 4) — can represent noise as if it were signal |
| Fast to train | Slower to train |
The goal is always "enough capacity to solve the actual problem" — not the maximum possible. A simpler network that solves the problem well is generally better than a needlessly complex one that barely does better.
Try it
Solve the Spiral dataset by adding hidden layers/neurons one step at a time, noting the smallest network size that actually succeeds.
Tip
When two networks perform about the same, prefer the smaller one — it trains faster, and (per Lesson 4) is usually less prone to overfitting.