Learn the right way to construct neural networks for direct causal inference

Constructing machine learning models is fairly easy nowadays, but often, making good predictions shouldn’t be enough. On top, we need to make causal statements about interventions. Knowing with high accuracy that a customer will leave our company is sweet, but knowing what to do about it — for instance sending a coupon — is a lot better. This can be a bit more involved, and I explained the fundamentals in my other article.

I like to recommend reading this text before you proceed. I showed you the way you’ll be able to easily come to causal statements every time your features form a sufficient adjustment set, which I may even assume for the remainder of the article.

The estimation works using so-called meta-learners. Amongst them, there are the S- and the T-learners, each with their very own set of disadvantages. In this text, I’ll show you one other approach that might be seen as a tradeoff between these two meta-learners that may provide you with higher results.

Allow us to assume that you may have a dataset (X, t, y), where X denotes some features, t is a definite binary treatment, and y is the consequence. Allow us to briefly recap how the S- and T-learners work and after they don’t perform well.

S-learner

Should you use an S-learner, you fix a model M and train it on the dataset such that M(X, t) ≈ y. Then, you compute

Treatment Effects = M(X, 1) – M(X, 0)

and that’s it.

The issue with this approach is that the mode could select to disregard the feature t completely. This typically happens if you happen to have already got tons of of features in X, and t drowns on this noise. If this…