“Feature Importance” will not be enough. You furthermore mght need to take a look at “Error Contribution” if you should know which features are helpful in your model.
10 hours ago
The concept of “feature importance” is widely utilized in machine learning as essentially the most basic form of model explainability. For instance, it’s utilized in Recursive Feature Elimination (RFE), to iteratively drop the least essential feature of the model.
Nevertheless, there may be a misconception about it.
The incontrovertible fact that a feature is very important doesn’t imply that it is useful for the model!
Indeed, once we say that a feature is very important, this simply signifies that the feature brings a high contribution to the predictions made by the model. But we must always consider that such contribution could also be mistaken.
Take an easy example: a knowledge scientist by chance forgets the Customer ID between its model’s features. The model uses Customer ID as a highly predictive feature. As a consequence, this feature can have a high feature importance even when it is definitely worsening the model, since it cannot work well on unseen data.
To make things clearer, we are going to must make a distinction between two concepts:
- Prediction Contribution: what a part of the predictions is as a consequence of the feature; that is comparable to feature importance.
- Error Contribution: what a part of the prediction errors is as a consequence of the presence of the feature within the model.
In this text, we are going to see how you can calculate these quantities and how you can use them to get priceless insights a couple of predictive model (and to enhance it).
Suppose we built a model to predict the income of individuals based on their job, age, and nationality. Now we use the model to make predictions on three people.
Thus, we’ve got the bottom truth, the model prediction, and the resulting error: