A Whole New Setting: No Labels
Every model so far learned from answers, this email is spam, this patient is sick. Unsupervised learning has none of that. You get a pile of data points and one job: find the structure in them yourself.
The most common version is clustering: find the natural clumps. Group your customers into segments, group news articles by topic, group pixels by colour, all without anyone telling you what the groups are.
Supervised learning fills in a known answer. Unsupervised learning discovers a structure nobody labelled.
k-means: the Idea
k-means is the simplest clustering method. You tell it how many groups you want, call it k, and it finds k centres so that every point belongs to its nearest one.
That’s the goal. The clever bit is how it finds those centres, and it’s a beautifully simple two-step dance.
The Dance: Assign, Then Move
- Scatter k centres at random.
- Assign every point to whichever centre is closest. Now you have k groups.
- Move each centre to the average position of its own group.
- The centres shifted, so some points are now closer to a different centre. Go back to step 2. Assign, move, assign, move…
Each pass, the centres drift into place, and after a few rounds they stop moving. Done.
The whole algorithm is two steps on repeat: assign points to the nearest centre, then move each centre to the middle of its points.
Quietly, every step shrinks the total distance from points to their centre, which is why it always settles down (to a good-enough answer).
The Catch: You Must Choose k
k-means can’t tell you how many clusters exist, you have to pick k up front. And the choice matters:
- Too few, and real groups get merged.
- Too many, and real groups get chopped into pieces.
A common trick is the elbow method: try several values of k, plot how tight the clusters get, and take the k where the improvement suddenly flattens out, the bend in the curve.
The Other Catch: It Loves Round Blobs
Because k-means measures straight-line distance to a centre, it does great on roundish clumps and badly on long or curvy shapes:
Two obvious crescents, and k-means slices a straight line clean through both of them. Distance-to-a-centre simply can’t wrap around a curve.
A couple of smaller cautions come along too:
- Random starts can go wrong. A bad initial scatter can settle into a poor answer, so in practice you run it several times (or use a smart start called k-means++) and keep the best.
- Scale matters. Scale your features first, exactly as with k-NN.
Why It’s Everywhere Anyway
Despite the caveats, k-means is a workhorse: customer segmentation, document grouping, image compression (shrink millions of colours down to k), and quick preprocessing. When someone wants to find groups in data, it’s almost always the first thing they try.
Next we tackle the opposite problem in unsupervised learning: not grouping the rows, but squeezing the columns, taking data with hundreds of features and compressing it down to the few that matter. That’s dimensionality reduction.