What is a mixture model?

You may have learned about mixture models in a machine learning or statistics course. A mixture model with K component densities is defined by

• a set of K nonnegative mixture weights summing to one, and
• a corresponding set of K nonnegative component densities, each of which integrates to one.

The sum of the product of the mixture weights and component densities is guaranteed to be nonnegative and integrates to one, meaning it is a valid probability density.

What is a subtractive mixture model?

There are other ways to construct functions which are are nonnegative and sum to one (valid probability densities). One such way involves subtractive mixture models. It is possible to construct

• a set of K mixture weights, which are not necessarily nonnegative and do not necessarily sum to one, and
• a corresponding set of K nonnegative component densities, each of which integrates to one

such that the sum of the product of the mixture weights and component densities is nonnegative and integrates to one.

Sampling

Assuming we can sample from the component densities, it is easy to sample from a mixture model. First, sample a component index from a categorical distribution defined by the mixture weights. Then, sample from the component densities.

The same sampling technique does not apply to subtractive mixture models.