  This is an extended version of the paper Lee and McLachlan (2014b) with
simulations and applications added. This paper introduces a finite mixture of
canonical fundamental skew t (CFUST) distributions for a model-based approach
to clustering where the clusters are asymmetric and possibly long-tailed (Lee
and McLachlan, 2014b). The family of CFUST distributions includes the
restricted multivariate skew t (rMST) and unrestricted multivariate skew t
(uMST) distributions as special cases. In recent years, a few versions of the
multivariate skew t (MST) model have been put forward, together with various
EM-type algorithms for parameter estimation. These formulations adopted either
a restricted or unrestricted characterization for their MST densities. In this
paper, we examine a natural generalization of these developments, employing the
CFUST distribution as the parametric family for the component distributions,
and point out that the restricted and unrestricted characterizations can be
unified under this general formulation. We show that an exact implementation of
the EM algorithm can be achieved for the CFUST distribution and mixtures of
this distribution, and present some new analytical results for a conditional
expectation involved in the E-step.

Methodology