Anchor Regression: Heterogeneous Data Meets Causality
Many traditional statistical prediction methods mainly deal with the problem of overfitting to the given data set. On the other hand, there is a vast literature on the estimation of causal parameters for prediction under interventions. However, both types of estimators can perform poorly when used for prediction on heterogeneous data. We show that the change in loss under certain perturbations (interventions) can be written as a convex penalty. This motivates anchor regression, a “causal” regularization scheme that encourages the estimator to generalize well to perturbed data. The novel methodology has connections to instrumental variable regression and robust optimisation.