#Explaining machine learning models

Sophisticated machine learning algorithms can produce accurate predictions, but their notorious "black box" nature may limit their adoption.

The SHAP value is a great tool among others (like LIME, InterpretML, or ELI5) to improve the transparency of ML models.

What is the Shapley Value?

Assume Ann, Bob, and Cindy together were hammering an "error" wood log, 38 inches, to the ground.

After work, they went to a local bar for a drink and a mathematician came to join them asking: "What is everyone’s contribution (in inches)?"

How to answer this question? I listed all the permutations and came up with the data in the following Table.

When the ordering is A, B, C, the marginal contributions of the three are 2, 32, and 4 inches, respectively.

The table shows the coalition of (A,B) or (B,A) is 34 inches, so the marginal contribution of C to this coalition is 4 inches.

Let's take the average of all the permutations for each person to get each individual's contribution:

• Ann is 2 inches
• Bob is 32 inches
• Cindy is 4 inches.

The Shapley value is the average of the marginal contributions across all permutations.