  Radiomic representations can quantify properties of regions of interest in
medical image data. Classically, they account for pre-defined statistics of
shape, texture, and other low-level image features. Alternatively, deep
learning-based representations are derived from supervised learning but require
expensive annotations from experts and often suffer from overfitting and data
imbalance issues. In this work, we address the challenge of learning
representations of 3D medical images for an effective quantification under data
imbalance. We propose a \emph{self-supervised} representation learning
framework to learn high-level features of 3D volumes as a complement to
existing radiomics features. Specifically, we demonstrate how to learn image
representations in a self-supervised fashion using a 3D Siamese network. More
importantly, we deal with data imbalance by exploiting two unsupervised
strategies: a) sample re-weighting, and b) balancing the composition of
training batches. When combining our learned self-supervised feature with
traditional radiomics, we show significant improvement in brain tumor
classification and lung cancer staging tasks covering MRI and CT imaging
modalities.

Computer Vision and Pattern Recognition;Artificial Intelligence