A tech data scientist’s stack to enhance stubborn ML models

This text is considered one of a two-part piece documenting my learnings from my Machine Learning Thesis at Spotify. You should definitely also try the second article on how I implemented Feature Importance on this research.

In 2021, I spent 8 months constructing a predictive model to measure user satisfaction as a part of my Thesis at Spotify.

My goal was to grasp what made users satisfied with their music experience. To achieve this, I built a LightGBM classifier whose output was a binary response:
y = 1 → the user is seemingly satisfied
y = 0 → not a lot

Predicting human satisfaction is a challenge because humans are by definition unsatisfied. Even a machine isn’t so fit to decipher the mysteries of the human psyche. So naturally my model was as confused as one might be.

From Human Predictor to Fortune Teller

My accuracy rating was around 0.5, which is the worst possible consequence you’ll be able to get on a classifier. It means the algorithm has a 50% likelihood of predicting yes or no, and that’s as random as a human guess.

So I spent 2 months trying and mixing different techniques to enhance the prediction of my model. In the long run, I used to be finally capable of improve my ROC rating from 0.5 to 0.73, which was a giant success!

On this post, I’ll share with you the techniques I used to significantly enhance the accuracy of my model. This text might come in useful each time you’re coping with models that just won’t cooperate.

Resulting from the confidentiality of this research, I cannot share sensitive information, but I’ll do my best possible for it to not sound confusing.