One possible Bayesian approach to estimation for multivariate categorical data is to combine inferences over a set of possible models, weighted by posterior model probabilities. In many examples, it is appropriate to use a standard class of models, for example log-linear interaction models, or graphical models. However, for some examples, such as a square contingency table where cross-classification is by variables with the same levels, the choice of model class is less obvious. I will describe an approach which uses permutation invariance, as a criterion for constructing a model class. The prior distributions for the model parameters satisfy the same invariance. As an example, I will present the analysis of a square contingency table.