Type ISTS markers are more informative than cytochrome b in phylogenetic reconstruction of the Mustelidae (Mammalia: Carnivora)

We compared the utility of five nuclear gene segments amplified with type I sequence-tagged site (STS) primers versus the complete mitochondrial cytochrome b (cyt b) gene in resolving phylogenetic relationships within the Mustelidae, a large and ecomorphologically diverse family of mammalian carnivores. Maximum parsimony and likelihood analyses of separate and combined data sets were used to address questions regarding the levels of homoplasy, incongruence, and information content within and among loci. All loci showed limited resolution in the separate analyses because of either a low amount of informative variation ( nuclear genes) or high levels of homoplasy (cyt b). Individually or combined, the nuclear gene sequences had less homoplasy, retained more signal, and were more decisive, even though cyt b contained more potentially informative variation than all the nuclear sequences combined. We obtained a well-resolved and supported phylogeny when the nuclear sequences were combined. Maximum likelihood and Bayesian phylogenetic analyses of the total combined data (nuclear and mitochondrial DNA sequences) were able to better accommodate the high levels of homoplasy in the cyt b data than was an equally weighted maximum parsimony analysis. Furthermore, partition Bremer support analyses of the total combined tree showed that the relative support of the nuclear and mitochondrial genes differed according to whether or not the homoplasy in the cyt b gene was downweighted. Although the cyt b gene contributed phylogenetic signal for most major groupings, the nuclear gene sequences were more effective in reconstructing the deeper nodes of the combined tree in the equally weighted parsimony analysis, as judged by the variable-length bootstrap method. The total combined data supported the monophyly of the Lutrinae (otters), whereas the Melinae (badgers) and Mustelinae (weasels, martens) were both paraphyletic. The American badger, Taxidea taxus (Taxidiinae), was the most basal taxon. Because hundreds of type I STS primer sets spanning the complete genomes of the human and mouse have been published and thus represent many independently segregating loci, the potential utility of these markers for molecular systematics of mammals and other groups is enormous.