University of Washington - Department of Statistics & Computer Science and Engineering
Effective discrimination is essential in many tasks including classification, anomaly detection, learning from partially labeled data. Statistical approaches used in this contexts generally fall into two major categories - generative and discriminative - depending on the criterion used for estimating the model's structure and parameters.
We present a general framework for discriminative estimation based on the maximum entropy principle and its extensions. Our framework preserves the advantages of using generative models while optimizing a discriminative criterion, allows the use of prior information and extends to a wide variety of discrimination problems including the ones mentioned above. In particular, Support Vector Machines are naturally subsumed by our framework and we provide several extensions.
Joint work with Tommi Jaakkola and Tony Jebara.