Birth-death MCMC methods for mixtures with an unknown number of 
components,  with application to hidden Markov models


By SHI, J. Q. 
Department of Computing Science and Department of Statistics, University of Glasgow, UK
Murray-Smith, R. 
Department of Computing Science, University of Glasgow, UK 
and Titterington, D. M.  
Department of Statistics, University of Glasgow, UK


Summary. Mixture models have very wide application. However, the problem 
of determining the number of components, $K$ say, is one of the 
intractable problems in this area. There are several approaches in the 
literature for testing $K$ for different values. Recently, an alternative 
approach is to treat $K$ as unknown, and to model $K$ and the mixture 
component parameters jointly, see e.g. Richardson and Green (1997).  In 
this paper, we propose an approach based on a birth-death process. In 
contrast with the approach in Stephens(2000), we make use of the latent 
indicators so that the approach can be used to solve the problems with 
missing data when calculation of the likelihood requires knowledge of the 
unobservable latent variables.  Specifically, we use the method to analyse 
hidden Markov models with unknown numbers of states. The model and the 
algorithm are illustrated by some real examples.  


Keywords: Bayesian inference; Birth-death process; Hidden Markov model; 
Markov chain Monte Carlo method; Mixtures with an unknown number of 
components.