Don’t Pick k, Build a Tree
k-means and GMM both made you commit to a number of clusters, k, before you even looked. That’s often the hardest choice, and you’re just guessing.
Hierarchical clustering refuses to guess. Instead of carving the data into k pieces, it builds a whole tree of clusters, bottom-up. The recipe (called agglomerative) is almost embarrassingly simple:
- Start with every point as its own tiny cluster.
- Merge the two closest clusters into one.
- Repeat until everything is a single cluster, recording every merge along the way.
No k, no random starts, no guessing. You just keep merging the nearest pair and write down what happened.
The Dendrogram
That merge history is a tree, and it has a name: the dendrogram. It’s the most useful thing hierarchical clustering gives you.
- Each leaf at the bottom is a single point.
- Each join is a merge.
- The height of a join = how far apart those two clusters were when they merged. Low joins = very similar; tall joins = only loosely related.
And here’s the payoff. You don’t pick k up front, you cut the tree at whatever height you like, afterward:
- Cut high → a few big clusters.
- Cut low → many small ones.
One tree hands you every value of k at once. You look at the structure first and then decide, which is exactly backwards from k-means, in the good way.
Linkage: What “Closest” Means
“Merge the two closest clusters” hides one decision: how do you measure distance between two whole clusters? That choice, called linkage, changes the shapes you get:
| Linkage | Cluster-to-cluster distance | Tends to produce |
|---|---|---|
| Single | the closest pair of points | long, straggly chains |
| Complete | the farthest pair | tight, compact blobs |
| Average | the average of all pairs | a balance of the two |
| Ward | whichever merge keeps clusters tightest | even, round clusters (popular default) |
When to Reach for It
Strengths:
- No k needed up front, you see the whole hierarchy.
- The dendrogram is interpretable, great for nested structure (think biology’s tree of life, or org charts).
- Deterministic, no random initialisation to get unlucky with.
Weaknesses:
- Slow, comparing all pairs costs roughly or worse, so it struggles on big datasets.
- Greedy and permanent, a merge, once made, is never undone, so an early mistake sticks.
Reach for hierarchical clustering when the data is modest in size and you care about the nested structure, not just a flat set of groups.
That rounds out clustering. Next in unsupervised learning we stop grouping points and start compressing features, which is where dimensionality reduction comes in.