Bayesian-Weighted Triplet and Quartet Methods for Species Tree Inference

Inference of the evolutionary histories of species, commonly represented by a species tree, is complicated by the divergent evolutionary history of different parts of the genome. Different loci on the genome can have different histories from the underlying species tree (and each other) due to processes such as incomplete lineage sorting (ILS), gene duplication and loss, and horizontal gene transfer. The multispecies coalescent is a commonly used model for performing inference on species and gene trees in the presence of ILS. This paper introduces Lily-T and Lily-Q, two new methods for species tree inference under the multispecies coalescent. We then compare them to two frequently used methods, SVDQuartets and ASTRAL, using simulated and empirical data. Both methods generally showed improvement over SVDQuartets, and Lily-Qwas superior to Lily-T for most simulation settings. The comparison to ASTRAL was more mixed-Lily-Q tended to be better than ASTRAL when the length of recombination-free loci was short, when the coalescent population parameter. was small, or when the internal branch lengths were longer.

Automated summary

This paper introduces two new methods for species tree inference, Lily-T and Lily-Q, and compares them with two frequently used methods, SVDQuartets and ASTRAL. The results show that Lily-Q generally outperforms the other methods in most simulation settings.