The last linkage is the AgglomerativeClustering default - the ward linkage. The ward linkage (or Ward's method) is more involved than the previous linkages.

If you're familiar with the k-means clustering algorithm (e.g., from my Cluster Analysis with Python online course), then ward linkage will be familiar to you.

The ward linkage uses what are known as prototypes. Think of prototypes as being data points the algorithm "makes up" to be the center of each cluster.

The most common type of prototype is a centroid which is defined as the average of all the data points in the cluster.

Graphically, the orange and green dots are the centroids of the two clusters in our running example:

The ward linkage first evaluates each cluster by finding all the distances (i.e., the dotted lines) between the data points in a cluster and their centroid:

Consider the orange cluster above. There are eight data points in the cluster and eight distances between each data point and the centroid.

The ward linkage takes each one of these distance values and squares them (i.e., the distance multiplied by itself). It then adds all these squared distances together.

It then repeats this process for the green cluster.

Lastly, it then adds the sum of the squared distances for the orange cluster to the sum of the squared distances for the green cluster.

Think of this last calculation (i.e., adding the two sums together) as being the baseline.

The ward linkage then combines the two clusters and calculates a new centroid (represented in blue):

The ward linkage then sums up all the squared distances for the new, larger cluster:

Think of the sum of the squared distances for the new, larger cluster as being the new hotness.

Ward linkage calculates the difference between the baseline and the new hotness and treats this as a penalty for combining clusters. Ward linkage looks to make this penalty as small as possible when considering which clusters to combine.

By minimizing the penalty, ward linkage prioritizes combining clusters that are most similar.

Even though Ward's method is the default, here's the code for reference:

It shouldn't come as a surprise that using different linkages can produce different clusterings.

So, you may be asking yourself, "which linkage should I use?"

When it comes to cluster analysis, I'm a big fan of experimenting to see what works the best. For example, using different linkages and see which produces the most useful clusters.

That being said, here are some rules of thumb:

• Single link is good at handling non-spherical cluster shapes, but is sensitive to outliers.

• Complete link is less susceptible to outliers, but can break large clusters and it tends to produce spherical clusters.

• Think of average link as being a compromise between single and complete linkages.

• Think of Ward's method as being a bit of an improvement over average link.