Skip to content

5. Hidden Layers and Complexity

Some patterns genuinely need more network to solve — but "more" always comes with trade-offs from Lesson 4.

Try a hard dataset with too little network

Switch to the Spiral dataset — a shape no single straight line can cleanly separate — and leave the network at just one hidden layer with one neuron. Train it. It will struggle or fail outright: too little capacity to represent a pattern this complex, a problem called underfitting.

Add capacity until it works

Add more neurons to the hidden layer, and add more hidden layers, retraining as you go. At some point, the network gains enough capacity to actually trace the spiral's shape in its decision boundary.

The trade-off

Too little capacity Too much capacity
Underfitting — can't represent the real pattern at all Overfitting risk (Lesson 4) — can represent noise as if it were signal
Fast to train Slower to train

The goal is always "enough capacity to solve the actual problem" — not the maximum possible. A simpler network that solves the problem well is generally better than a needlessly complex one that barely does better.

Try it

Solve the Spiral dataset by adding hidden layers/neurons one step at a time, noting the smallest network size that actually succeeds.

Tip

When two networks perform about the same, prefer the smaller one — it trains faster, and (per Lesson 4) is usually less prone to overfitting.

Next: Connecting Back to Teachable Machine →