Optimal outgroup analysis

We present and critically examine a statistical criterion for the selection of outgroup taxa for rooting evolutionary trees. The criterion is the amount of phylogenetic signal for the ingroup when the states of the candidate outgroup taxa are assumed to be plesiomorphic relative to the ingroup for the purpose of measuring plesiomorphy content of the outgroup taxon. A statistical measure of rooted, ingroup signal was subjected to a suite of critical tests which indicate that it provides a proxy measure of plesiomorphy content. As the evolutionary distance between the ingroup ancestral node and outgroup taxa increases, the tree-independent measure of signal decreases, tracking the decay in plesiomorphy content and the increase in convergence to the ingroup states. We show that a priori generalizations about optimal outgroup taxon sampling strategies are likely to be misleading, and that testing far the suitability of available outgroup taxon sampling in specific instances is warranted. Software for optimal outgroup analysis is available. (C) 1998 The Linnean Society of London.