Question 1

What is backpropagation, in simple terms?

Accepted Answer

Backpropagation is how a neural network figures out which of its weights caused a mistake. After the network makes a prediction (the forward pass), it compares the prediction to the correct answer to get an error. Backpropagation then runs that error backward through the network, layer by layer, using the chain rule from calculus to compute exactly how much each individual weight contributed to the mistake. In this visualizer, that is the right-to-left orange animation: each neuron receives its share of the blame (its "delta"), and that determines how its incoming weights will be adjusted.

Question 2

What do the blue and red connections mean?

Accepted Answer

Each line is a weight — a single number the network learns. Blue means the weight is positive (it excites the next neuron: more input produces more output), and red means it is negative (it inhibits the next neuron). The thicker the line, the larger the weight. At the start of training the weights are small random values, so all the lines look thin and faint. As the network learns, watch some connections grow strong and others shrink toward zero — that pattern of strengths IS the network's knowledge.

Question 3

What is a decision boundary?

Accepted Answer

The decision boundary is the line (or curve) where the network switches from predicting one class to predicting the other. In the data plot, the colored background shows the network's prediction for every possible point: blue regions mean "class blue", orange regions mean "class orange", and the boundary is where they meet. An untrained network has a meaningless boundary; as training progresses you can watch it bend and curve until it separates the two groups of points. Deeper or wider networks can produce more complex boundaries.

Question 4

Why does the spiral dataset need ReLU and two hidden layers?

Accepted Answer

The spiral is the hardest dataset in this tool because the two classes wrap around each other — separating them requires a very intricate decision boundary. A single small hidden layer simply cannot bend space enough to trace the spiral arms. Two layers of six ReLU neurons can: the first layer carves space into pieces, and the second layer combines those pieces into spiral-shaped regions. This is exactly why "deep" learning matters — stacking layers lets networks build complex shapes out of simple ones. Note it also needs a smaller learning rate (0.03) and a few thousand epochs, so use Fast mode.

Question 5

What is the learning rate, and what happens if I set it too high?

Accepted Answer

The learning rate controls how big a step each weight takes during every update. Too small (0.01) and learning is painfully slow — the loss creeps down over thousands of epochs. Too large (try 3.0!) and the weights overshoot the correct values and bounce around or even diverge: you will see the loss jump up and down instead of decreasing, and the decision boundary flicker chaotically. Finding a good learning rate is one of the most important practical skills in deep learning. Try it yourself: train XOR with 0.5, then reset and try 3.0.

Question 6

Is this how real models like ChatGPT are trained?

Accepted Answer

Yes — the core algorithm is identical: forward pass, loss, backpropagation, weight update, repeated billions of times. The differences are scale and architecture. This visualizer trains a network with around 20-60 weights on 120 data points; GPT-class models have hundreds of billions of weights trained on trillions of tokens, use the transformer architecture instead of simple dense layers, and train on thousands of GPUs for months. But every one of those billions of weights is updated by exactly the same mechanism you are watching here: gradient descent driven by backpropagation.

Deep Learning Visualizer — Watch a Neural Network Train, Live

Interactive Calculator

Frequently Asked Questions

Turn your GPU into an OpenAI-compatible API endpoint