University of Washington - Department of Biostatistics
League tables of performance provide a simple tool for comparison of hospitals, schools, and various other institutions. In typical applications, institutions are assessed by their performance relative to some population average: any institution whose performance is significantly different from this average may be liable to investigation, in order to uncover exceptional behavior.
The assumption of a single acceptable performance rate here is very conservative, and in some cases, up to 70% of institutions are â€œflagged upâ€ as outliers, even allowing for multiple comparisons. This is plainly ridiculous, and motivates us to study acceptable variation in performance. The approach suggested is to assume a normal distribution for institutions that perform acceptably, and attempt to estimate it allowing for the presence of contaminants. Connections are made between our formal model-based approach and well-known existing robust estimation procedures with more ad hoc derivations. The model-based approach also leads directly to measures of â€œoutlyingnessâ€ for each institution; we argue that these can and should be combined using methods from the False Discovery Rate literature. Connections between such methods and Bayesian decision-theoretic approaches are also discussed.