By default, the AgglomerativeClustering class' n_clusters hyperparameter is set to 2. The way to think about this is that the algorithm defaults to the simplest clustering scenario - only two clusters.

However, to use clustering to drive real-world insights, you need to find the optimal number of clusters. The default is almost always too low and letting the algorithm find as many clusters as it can usually produces unusable results.

Clustering, like so much of machine learning, needs to be tuned if you want to be successful.

At a high level, here is a process for tuning agglomerative hierarchical clustering:

1. Start with n_clusters = 2 and evaluate the quality of the clusters.

2. Move to n_clusters = 3 and evaluate the quality.

3. Repeat this process with higher values of n_clusters.

4. Pick the n_clusters value with the highest quality score.

The above tuning process is conceptually simple, but there's a catch.

How do you define "cluster quality."

In this tutorial, we will use a cluster quality calculation known as the silhouette coefficient​. This calculation gives each data point (i.e., row in the dataset) a score ranging from -1 to 1.

Generally speaking, scores close to 1 indicate high quality, and scores close -1 indicate a poor quality.

The score is dependent on the value used for n_clusters. The goal of the tuning process is to find a value of n_clusters that accomplishes the following simultaneously:

• The highest average silhouette score across all the data points.

• A collection of clusters that provide business insights.

As we will see later, sometimes you will give up some of the first bullet to get more of the second bullet.

The first step in the tuning process is to define the values of n_clusters to be evaluated. The following code creates a Python range object to hold these values: