A guide to Neptune.ai for tracking your experiments in Python

Every data scientist is conversant in experimentation.

You already know the drill. You get a dataset, load it right into a Jupyter notebook, explore it, preprocess the information, fit a baseline model or two, after which train an initial final model, reminiscent of XGBoost. The primary time around, perhaps you don’t tune the hyperparameters and include 20 features. Then, you check your error metrics.

They appear okay, but perhaps your model is overfitting a bit. So you select to tune some regularization parameters (eg max depth) to scale back the complexity of the model and run it again.

You see somewhat improvement from the last run, but perhaps you need to also:

• Add more features
• Perform feature selection and take away some features
• Try a special scaler to your features
• Tune different/more hyperparameters

As the various sorts of tests you need to run increases, the tougher it’s to recollect which combos of your “experiments” actually yielded the very best results. You may only run a notebook so over and over, print out the outcomes, and replica/paste them to a Google doc before you get frustrated.