  Convolutional Neural Networks (Convnets) have achieved good results in a
range of computer vision tasks the recent years. Though given a lot of
attention, visualizing the learned representations to interpret Convnets, still
remains a challenging task. The high dimensionality of internal representations
and the high abstractions of deep layers are the main challenges when
visualizing Convnet functionality. We present in this paper a technique based
on clustering internal Convnet representations with a Dirichlet Process
Gaussian Mixture Model, for visualization of learned representations in
Convnets. Our method copes with the high dimensionality of a Convnet by
clustering representations across all nodes of each layer. We will discuss how
this application is useful when considering transfer learning, i.e.\
transferring a model trained on one dataset to solve a task on a different one.

Computer Vision and Pattern Recognition