THE LACK OF CONSISTENCY FOR STATISTICAL DECISION PROCEDURES

Simpson's paradox exhibits seemingly deviant behavior where the data generated in independent experiments support a common decision, but the aggregated data support a different outcome. It is shown that this kind of inconsistent behavior occurs with many, if not most, statistical decision processes. Examples are given for the Kruskal-Wallis test and a Bayesian decision problem. A simple theory is given that permits one to determine whether a given decision process admits such inconsistencies, to construct examples, and to find data restrictions that avoid such outcomes.