Home

Papers and Preprints

CV

Courses

Interesting links

Bayesian Methods for Mixtures of Normal Distributions

Abstract

Mixture distributions are typically used to model data in which each observation is assumed to have arisen from one of a number of different groups. They also provide a convenient and flexible class of models for density estimation.

While a Bayesian analysis of mixture models has certain advantages over a classical approach, it is not without its problems. In theory quantities of interest may be written down as integrals, but in practice these integrals cannot be done analytically. When the number of groups in the data is assumed known, the Gibbs sampler can be used to perform this integration numerically, but the non-identifiability of the mixture model parameters causes label-switching in the Gibbs sampler output and makes inference for the individual components of the mixture meaningless. We show that the usual method of dealing with this problem (imposing simple identifiability constraints on the mixture model parameters) is sometimes inadequate, and present a more flexible approach to solving this problem, which allows sensible clustering to be performed in a Bayesian context and allows interpretations for groups to be discovered rather than imposed. We illustrate the success of our approach on several examples.

When the number of groups in the data is considered unknown more sophisticated methods are required to perform the integration necessary for a Bayesian analysis. One method is described by Richardson and Green (1997), which they apply successfully to univariate data. We describe an alternative method which views the parameters of the model as a (marked) point process, extending methods suggested by Ripley (1977) to create a Markov birth-death process with an appropriate stationary distribution. We apply this method successfully to both univariate and bivariate data.

Finally we examine ``on-line'' methods for mixture models, in which the posterior distribution of the parameters is updated as observations arrive sequentially, and are then discarded. We show that the computationally trivial Quasi-Bayes method of Makov and Smith (1977) can be improved upon at the expense of small additional computational complexity.

Download gzipped postscript version of complete thesis.