Harvard University - Department of Statistics
Texture is a powerful cue in visual perception, and texture analysis and synthesis has been an active research area in computer vision. We present a statistical theory for texture modeling and random field approximation, which combines multi-channel filtering and random field modeling via the maximum entropy principle. Our theory characterizes a texture by a random field, the modeling of which consists of two steps. (I) Feature extraction: a set of finite-support filters (usually linear) is applied to the random field and the marginal distributions of the filter responses are considered as features concerning textural appearances, which is supported by physiological observations on visual cells. (II) Feature fusion: the model is then derived by maximizing the entropy over all distributions with the same marginal distributions as in (I), and the resulting model is called Filter, Random field, And Maximum Entropy (FRAME) model. Sieve-MLE is used for non-parametric fitting, where a histogram matching algorithm is proposed for computation and simulation. We think of FRAME as low-dimensional approximation to the "true" distribution that generates texture images, and prove that any MRF can be approximated arbitrarily close by FRAME. Motivated by a mini-max entropy principle, we propose a filter pursuit procedure for selecting a parsimonious set of meaningful filter "words" from a well designed filter "vocabulary" when modeling a certain texture image. A variety of texture synthesis experiments are described to illustrate our theory (intersting pictures will be displayed). Many previous methods and concepts are interpreted and clarified in a unified point of view.
This talk is based on the joint work with S.C. Zhu and D.B. Mumford. Support from Prof. D.B. Rubin and many helpful discussions with Prof. A.P. Dempster are warmly acknowledged.