Boeing Research - Department of Autonomous Systems
We consider the problem of â€œunderstandingâ€, on a massively automated scale, the behavior of people. To a first approximation, people are rational agents in the sense of optimizing a utility function. If their utility function can be estimated, one can define â€œunderstandingâ€ as identifying the likely underlying incentives for the observed actions and â€œintentâ€ as the best course of action available to an agent optimizing such a function. Inverse reinforcement learning (IRL) techniques provide the necessary foundation for estimating an agentâ€™s utility function and predicting agent intent. While applied IRL algorithms do suffer from the large dimensionality of the reward function space, many applications that can benefit from an IRL-based approach to assessing agent intent involve a domain expert or analyst. We describe a procedure for scaling up the estimation by eliciting good IRL basis functions from the domain expert and discuss issues and some experiments in modeling the perceptual and decision space for the analysis of vehicle movements and detection of anomalous driving behavior.