A deep-dive into the newest technology used to debias rating models

Recommender systems are amongst probably the most ubiquitous Machine Learning applications on the planet today. Nonetheless, the underlying rating models are tormented by quite a few biases that may severely limit the standard of the resulting recommendations. The issue of constructing unbiased rankers — also referred to as unbiased learning to rank, ULTR — stays one of the crucial essential research problems inside ML and remains to be removed from being solved.

On this post, we’ll take a deep-dive into one particular modeling approach that has relatively recently enabled the industry to manage biases very effectively and thus construct vastly superior recommender systems: the two-tower model, where one tower learns relevance and one other (shallow) tower learns biases.

While two-tower models have probably been utilized in the industry for several years, the primary paper to formally introduce them to the broader ML community was Huawei’s 2019 PAL paper.

PAL (Huawei, 2019) — the OG two-tower model

Huawei’s paper PAL (“position-aware learning to rank”) considers the issue of position bias throughout the context of the Huawei app store.

Position bias has been observed over and once more in rating models across the industry. It simply signifies that users usually tend to click on items which might be shown first. This will likely be because they’re in a rush, because they blindly trust the rating algorithm, or other reasons. Here’s a plot demonstrating position bias in Huawei’s data:

Position bias is an issue because we simply can’t know whether users clicked on the primary item since it was indeed probably the most relevant for them or since it was shown first — and in recommender systems we aim to unravel the previous learning objective, not the latter.

The answer proposed within the PAL paper is to factorize the educational problem as

p(click|x,position) = p(click|seen,x) x p(seen|position),