The confusion matrix provides you additional visibility into the model's predictions beyond what overall accuracy can tell you.

Each row of the confusion matrix shows the model's predictive performance for each label (i.e., <=50K vs. >50K) in the test dataset.

Correct predictions run along the diagonal, starting from the top left and going to the bottom right. Using the confusion matrix, you can easily calculate the following:

• The accuracy for the <=50K labels is:

  • 2910 / (2910 + 790) = 78.65%

• The accuracy for the >50K labels is:

  • 3115 / (3115 + 585) = 84.19%

The model is about 5.5% more accurate in predicting incomes greater than $50,000/year than incomes less than that.

Notice how the confusion matrix already provides an interpretation that business stakeholders can easily understand?

NOTE - I typically don't show confusion matrices to my business stakeholders. I've found it's usually more confusing than helpful. I normally summarize confusion matrix findings as bullets in a PowerPoint slide.

It's scarce in practice to have a group of stakeholders that will accept only the following as the model interpretation:

• Estimated average model accuracy is 81.3%.

• However, the model's accuracy will range from 79.2% to 83.4%.

• The model scored 81.4% accuracy on the test dataset.

• The model is 5.5% accurate for high-income earners on the test dataset.

In my experience, business stakeholders want their explanation in terms of the features that drive the model's predictions.

Enter permutation feature importance.

Permutation feature importance is a powerful tool for interpreting ML models because it works with any algorithm: logistic regression, decision trees, random forests, etc.

Here's the base intuition of how it works:

1. ML models find features with the most information to make the best predictions possible given the training data.

2. If you remove these features, the model's predictive performance declines.

Since we have a trained model, we can't really remove the features, but we can do something else that comes close.

We can randomize (i.e., permute) the feature values and then see how much the predictive performance declines as a result. The features with the most information (i.e., the most important) will result in the largest declines in predictive performance.

The intuitive way to think about this is in the extreme worst case scenario. If a feature is permuted and the predictive performance doesn't decline at all, that feature has no information useful for making predictions!

Just in case you're wondering, permutation feature importance works on copies of the original data.

Permutation feature performance takes a trained model, a dataset, and then permutes each feature one at a time (the other features remain unchanged) and records the resulting decline in predictive performance.

For example, the DecisionTreeClassifier was trained using eight features. Here's how permutation feature importance works with this model:

1. The Age feature is permuted and then predictions are made using the model with the other test dataset (un-permuted) features.

2. The accuracy in the predictions with permuted Age values is compared to the accuracy with the original Age values (the decline is calculated and stored).

3. This process is then repeated with the one-hot encoded Education features.

4. Then with the one-hot encoded MaritalStatus features...

So on and so forth - until all the features have been permuted and all the accuracy declines are calculated and stored.

The main downside of permutation performance is that it is compute-intensive (i.e., it can take a long time to run). However, the wait is worth the model insights provided.

Here's the code using scikit-learn in Python: