Adrian Raftery: Research on Deterministic Simulation Models

There are two main cultures of quantitative research: statistical modeling and deterministic simulation models. Deterministic simulation models attempt to represent the underlying mechanism explicitly, and typically consist of interlocking systems of differential or difference equations. They are much used in several environmental, engineering and policy-making disciplines. There has been little interaction between statistical and simulation modelers, although that has started to change recently. I believe that there is a natural complementarity between the two: simulation modelers take great pains to represent the signal in the system and do so in a sophisticated way, while largely ignoring measurement error and noise, and statisticians model the noise very carefully while often using very simple (often linear) approximations to the signal.

Deterministic simulation models require the user to specify inputs such as initial conditions or parameters of the differential equations, and yield outputs. Our approach has been to represent information about both inputs and outputs by probability distributions, and combine them using Bayesian methods and methods for combining prior distributions. Our first effort was the Bayesian synthesis method (Raftery, Givens and Zeh 1995). There was a technical problem with this, and its solution led to the Bayesian melding approach (Poole and Raftery 2000). This was used, for example, as the basis for the International Whaling Commission's policy on bowhead whales. The Bayesian uncertainty assessment of Bates et al (2003) is a special case of this, applied to multicompartment models.

More recently, in the context of atmospheric science (weather forecasting and air pollution), we have been developing methods to deal with problems that often arise. One is that one run of a model can take a long time time to run, making Monte-Carlo based methods such as Bayesian melding impracticable. We have been addressing this by using methods for statistical postprocessing of model ensembles (Raftery et al 2005, Gneiting et al 2005).

Another problem is that the outputs from the model may be on a different scale from measurements of the outputs; for example the outputs are on a grid and the data are at irregularly spaced points. We have developed an extension of Bayesian melding to address this problem (Fuentes and Raftery 2005). A related issue is that the quantity of interest map be of very high dimension, with complex correlations, for example a weather map. To address this, we have developed the Geostatistical Output Pertubation (GOP) method, which produces a joint predictive distribution based on model output (Gel et al 2004).

Papers

Berrocal, V.J., Raftery, A.E. and Gneiting, T. (2008). ``Probabilistic Quantitative Precipitation Forecasting using a Two-Stage Spatial Model.'' Technical Report no. 532, Department of Statistics, University of Washington.

Alkema, L., Raftery, A.E. and Brown, T. (2008). ``Bayesian melding for estimating uncertainty in national HIV prevalence estimates.'' Working Paper no. 82, Center for Statistics and the Social Sciences, University of Washington.

Alkema, L., Raftery, A.E. and Clark, S.J. (2007). Probabilistic projections of HIV prevalence using Bayesian melding. Annals of Applied Statistics, 1, 229-248.

Berrocal, V., Raftery, A.E. and Gneiting, T. (2007). Combining Spatial Statistical and Ensemble Information in Probabilistic Weather Forecasts. Monthly Weather Review, 135, 1386-1402.

Wilson, L.J., Beauregard, S., Raftery, A.E. and Verret, R. (2007). Calibrated Surface Temperature Forecasts from the Canadian Ensemble Prediction System Using Bayesian Model Averaging (with Discussion). Monthly Weather Review, 135, 1364-1385. Discussion pages 4226-4236.

Sloughter, J.M., Raftery, A.E. and Gneiting, T. (2007). Probabilistic Quantitative Precipitation Forecasting Using Bayesian Model Averaging. Monthly Weather Review, 135, 3209-3220.

Sevcikova, H., Raftery, A.E. and Waddell, P. (2007). Assessing Uncertainty in Urban Simulations Using Bayesian Melding. Transportation Research B, 41, 652-669.

Tewson, P. and Raftery, A.E. (2006). Real-Time Calibrated Probabilistic Forecasting Website. Bulletin of the American Meteorological Society, 7, 880-882.

Gneiting, T. and Raftery, A.E. (2005). Weather forecasting with ensemble methods. Science, 310, 248-249.

Raftery, A.E., Gneiting, T., Balabdaoui, F. and Polakowski, M. (2005). Using Bayesian Model Averaging to Calibrate Forecast Ensembles. Monthly Weather Review, 133, 1155-1174.

Gneiting, T., Raftery, A.E., Westveld, A. and Goldman, T. (2005). Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation. Monthly Weather Review, 133, 1098-1118.

Fuentes, M. and Raftery, A.E. (2005). Model evaluation and spatial interpolation by Bayesian combination of observations with outputs from numerical models. Biometrics, 66, 36--45.

Gel, Y., Raftery, A.E. and Gneiting, T. (2004). Calibrated probabilistic mesoscale weather field forecasting: The Geostatistical Output Perturbation (GOP) method (with Discussion). Journal of the American Statistical Association, 99, 575-590.
Earlier technical report version with color figures.

Bates, S., Raftery, A.E. and Cullen, A.C. (2003). Bayesian Uncertainty Assessment in Deterministic Models for Environmental Risk Assessment. Environmetrics, 14, 355-371.

Poole, D.J. and Raftery, A.E. (2000). Inference for deterministic simulation models: The Bayesian melding approach. Journal of the American Statistical Association, 95, 1244-1255. Earlier, more complete technical report version (Postscript).

Poole, D., Givens, G.H. and Raftery, A.E. (1999). A proposed stock assessment method and its application to bowhead whales, Balaena mysticetus. Fishery Bulletin, 97, 144-152. Earlier technical report version.

Givens, G.H., Zeh, J.E. and Raftery, A.E. (1996). Implementing the current management regime for aboriginal subsistence whaling to establish a catch limit for the Bering--Chukchi--Beaufort Seas stock of bowhead whales. Report of the International Whaling Commission, 46, 493--501.

Givens, G.H. and Raftery, A.E. (1996). Local adaptive importance sampling for multivariate densities with strong nonlinear relationships. Journal of the American Statistical Association, 91, 132-141.

Givens, G.H., Zeh, J.E. and Raftery, A.E. (1995). Assessment of the Bering-Chukchi-Beaufort Seas stock of bowhead whales using the BALEEN II model in a Bayesian synthesis framework. Report of the International Whaling Commission, 45, 345-364.

Givens, G.H., Raftery, A.E. and Zeh, J.E. (1995). Response to comments by Butterworth and Punt in SC/46/AS2 on the Bayesian synthesis approach. Report of the International Whaling Commission, 45, 325-330.

Raftery, A.E., Givens, G.H. and Zeh, J.E. (1995). Inference from a deterministic population dynamics model for bowhead whales (with Discussion). Journal of the American Statistical Association, 90, 402-430. Rejoinder. [The 1995 JASA-Applications and Case Studies Invited Paper.]

Givens, G.H., Raftery, A.E. and Zeh, J.E. (1994). A reweighting approach for sensitivity analysis within the Bayesian synthesis framework for population assessment modeling. Report of the International Whaling Commission, 44, 377-384.

Givens, G.H., Raftery, A.E. and Zeh, J.E. (1993). Benefits of a Bayesian approach for synthesizing multiple sources of evidence and uncertainty linked by a deterministic model. Report of the International Whaling Commission, 43, 495-500.

Raftery, A.E., Shier, P. and Obilade, T. (1980). Domestic space heating and solar energy in Ireland. International Journal of Energy Research, 4, 31-39.