Title:  Fitting sequence data with mixtures of self-reproducing kernels.


Abstract:
Self-reproducing (SR) kernels make an attractive building block for fitting 
high-dimensional sequence data, as they provide highly flexible mixture 
models as well as simple and effective model selection mechanisms.  By 
sequence data, we mean strings of correlated data points whose 
dimensionality is challenging, such as sequences of binary digits of length 
64, for which the sample space is enormous. An example of a 
self-reproducing kernel is the normal, which satisfies the rule "a normal 
mixture of normals is also normal." We will introduce two other such 
kernels, the mutation kernel for binary sequences and the Poisson kernel 
for spherical data.


For each SR kernel there is a location parameter mu that describes the 
central location of the sequence and a sieve parameter sigma that makes the 
density more and more concentrated about mu as it goes to zero. If we fit a 
mixture of SR kernels with a fixed sigma using nonparametric maximum 
likelihood, we get a unique solution. (We could also fit a fixed number of 
components K, with sigma a free parameter; the likelihood solutions are no 
longer unique.)  As sigma gets smaller, the fits become closer and closer 
to the empirical distribution of the data, while as sigma gets larger one 
fits fewer and fewer components. The result of the estimation procedure is 
a "mixture tree" of related estimators.


This leaves the problem of selecting good values of the sieve parameter, 
for which we will use an auxiliary goodness-of-fit criterion. One can turn 
the SR kernel into a quadratic distance function for evaluating the fit of 
estimators. The SR property enables one to compute the needed 
high-dimensional integrals without numerical integration. The distance can 
be used as a guide to minimum risk model building. As part of this 
procedure one must also choose one or more reasonable smoothing parameters 
h for the distance. This can be done using a simple 
pseudo-degrees-of-freedom calculation.


This work is being carried out with students Shu-Chuan Chen, Surajit Ray, 
and Ke Yang.