The Perceptron

You Already Built One

Here is a secret: you already built a neuron. Back in logistic regression you took some inputs, weighted them, added them up, and squashed the result through a sigmoid. That exact pipeline is a single neuron. We are just going to rename it, and then learn to stack it.

Watch one neuron do its thing:

The recipe has three moves:

• Weight every input by how much it matters.
• Sum the weighted inputs, plus a bias.
• Squash that total through an activation to get one output.

In symbols, the sum is

$z = w_1 x_1 + w_2 x_2 + \ldots + w_n x_n + b$

and the output is $\hat{y} = \sigma(z)$: the activation applied to $z$.

A neuron is nothing more than a weighted sum followed by a squash. Every model in deep learning is built from this one part.

It Is Modeled on a Brain Cell

The name is not an accident. A real brain cell collects signals from thousands of others. Some incoming connections are strong, some weak, and it adds up all that signal. If the total crosses a threshold, the cell fires.

The artificial version copies the shape almost part for part:

The very first version (Rosenblatt, 1958) used a hard step: fire (output $1$) if $z$ passed a threshold, otherwise stay silent (output $0$). Modern neurons swap that hard step for a smooth activation like the sigmoid, so the output changes gently and we can follow gradients, the very trick that made logistic regression trainable.

Hard step or smooth sigmoid, the idea is the same: the activation decides how strongly the neuron fires.

One Neuron Is One Straight Line

Here is the geometric truth that matters most. The neuron fires when $z \ge 0$ and stays quiet when $z < 0$. The surface where it flips, where $z = 0$, is a straight line (a flat plane in higher dimensions).

So a single neuron is just a straight boundary cutting space in two:

• The weights set which way the line tilts. They point in the direction perpendicular to it, the little arrow in the animation.
• The bias sets where the line sits, sliding it back and forth.

Everything on one side fires, everything on the other stays quiet. That is the entire job.

One neuron draws one straight cut. That is its power, and also its limit.

It Can Find the Line Itself

We do not set the weights by hand. The neuron learns them from examples, and the rule is beautifully simple: find a point it got wrong, and nudge the line a little toward fixing that point. Then repeat.

Each mistake tugs the boundary a bit. On data a line can separate, this keeps going until no point is left on the wrong side, and the neuron has found a dividing line entirely on its own. It is the same learn from your error idea as gradient descent, in its simplest possible form.

The Catch

One neuron, one straight line. Elegant, but also a wall.

Some problems cannot be split by any straight line, no matter how you tilt or slide it. The classic example is XOR (true when exactly one input is on), and a single neuron is helpless against it.

To bend the boundary, we need more than one neuron. Stack them into layers, and the straight cuts start combining into curves.

That stacking is where real neural networks begin, and it is exactly where we go next.