For signal and image classification problems, such as the ones in medical or geophysical diagnostics and military applications, extracting relevant features is one of the most important tasks. As an attempt to automate the feature extraction procedure and to understand what the critical features for classification are, we developed the so-called local discriminant basis (LDB) method which rapidly selects an orthonormal basis suitable for signal/image classification problems from a large collection of orthonormal bases (e.g., wavelet packets and local trigonometric bases). Here, the basis functions well-localized in the time-frequency plane and carrying discriminant information are used as feature extractors. Once the LDB is selected, a small number of most significant coordinates are fed into a traditional classifier such as Linear Discriminant Analysis (LDA) or Classification and Regression Tree (CART). The performance of these statistical methods is enhanced since the LDB method reduces the dimensionality of the problems without losing important information for classification. Moreover, since the basis functions well-localized in the time-frequency plane are used as feature extractors, interpretation of the classification results becomes easier and more intuitive than using the conventional methods.
In this talk, we describe the original LDB (which maximizes relative entropy of time-frequency energy distributions among classes) as well as its recent improvements; i.e., stabilization by the "spin cycle" procedure (which compensates the lack of translation invariance property of these orthonormal bases) and maximization of relative entropy of empirical probability densities of classes in the basis coordinates instead of the time-frequency energy distributions.
Finally, we show applications of these methods to geophysical signal classification and image texture classification problems.