  We propose a novel technique for training deep networks with the objective of
obtaining feature representations that exist in a Euclidean space and exhibit
strong clustering behavior. Our desired features representations have three
traits: they can be compared using a standard Euclidian distance metric,
samples from the same class are tightly clustered, and samples from different
classes are well separated. However, most deep networks do not enforce such
feature representations. The DEFRAG training technique consists of two steps:
first good feature clustering behavior is encouraged though an auxiliary loss
function based on the Silhouette clustering metric. Then the feature space is
retracted onto a Grassmann manifold to ensure that the L_2 Norm forms a
similarity metric. The DEFRAG technique achieves state of the art results on
standard classification datasets using a relatively small network architecture
with significantly fewer parameters than many standard networks.

Computer Vision and Pattern Recognition