  As costs of genome sequencing have dropped precipitously, development of
efficient bioinformatic methods to analyze genome structure and evolution have
become ever more urgent. For example, most published phylogenomic studies
involve either massive concatenation of sequences, or informal comparisons of
phylogenies inferred on a small subset of orthologous genes, neither of which
provides a comprehensive overview of evolution or systematic identification of
genes with unusual and interesting evolution (e.g. horizontal gene transfers,
gene duplication and subsequent neofunctionalization). We are interested in
identifying such "outlying" gene trees from the set of gene trees and
estimating the distribution of the tree over the "tree space".
  This paper describes an improvement to the kdetrees algorithm, an adaptation
of classical kernel density estimation to the metric space of phylogenetic
trees (Billera-Holmes-Vogtman treespace), whereby the kernel normalizing
constants, are estimated through the use of the novel holonomic gradient
methods. As the original kdetrees paper, we have applied kdetrees to a set of
Apicomplexa genes and it identified several unreliable sequence alignments
which had escaped previous detection, as well as a gene independently reported
as a possible case of horizontal gene transfer.

Populations and Evolution