Feature engineering, unsupervised classification, and anomaly detection with the flexibility of the GMM algorithm
12 hours ago
Gaussian Mixture Model (GMM) is a straightforward, yet powerful unsupervised classification algorithm which builds upon K-means instructions as a way to predict the probability of classification for every instance. This property of GMM makes it versatile for a lot of applications. In this text, I’ll discuss how GMM may be utilized in feature engineering, unsupervised classification, and anomaly detection.
Model Description
While the Gaussian distribution of a single or multiple variables of a dataset attempts to represent all the population probabilistically, GMM makes an assumption that there exist subpopulations within the dataset and every follows its own normal distribution. In an unsupervised fashion, GMM attempts to learn the subpopulations inside the data and its probabilistic representation of every data point [1]. This property of GMM allows us to make use of the model to search out points which have low probability of belonging to any subpopulation and, due to this fact, categorize such points as outliers.
GMM essentially extends the multivariate Gaussian distribution to suit the subpopulation case by utilizing components to represent these subpopulations and alters the multivariate probability distribution function to suit the components. As a mild reminder, the probability density function of the multivariate Gaussian looks like this:
In GMM, the probability of every instance is modified to be the sum of probabilities across all components and component weights are parameterized as 𝜙. GMM requires that the sum of all components weights is 1 so it will probably treat each component as a ratio of the entire. GMM also incorporates feature means and variances for every component. The model looks like this:
Notice the parallels between multivariate distribution and GMM. In essence, the GMM algorithm finds the…