Carnegie Mellon University - Department of Statistics
Neural networks are a useful statistical tool for nonparametric regression. In my talk I will describe a methodology for nonparametric regression within the Bayesian framework. I focus on the problem of model selection and model averaging, including estimation of normalizing constants and searching of the model space in terms of both the optimal number of hidden nodes in the network as well as the best subset of explanatory variables. Estimating the normalizing constant is necessary for estimating the posterior probabilities of different models, and it is a difficult problem because of the multi-modality and irregularity of the contours of the posterior. Searching the model space for models of high posterior probability is difficult because of the large size of the space, and because of the need to explore both different numbers of hidden nodes in the network, as well as different subsets of explanatory variables. Another aspect of my neural network methodology is work on noninformative priors for neural networks, which are useful because of the difficulty in interpreting the parameters. I also have several asymptotic consistency results for the posterior of a neural network. Assuming a true regression function, Hellinger neighborhoods of the true function have posterior probability tending to one asymptotically. Also, the mean square error of the predictive regression function goes to zero in probability.