  Annotated medical images are typically rarer than labeled natural images
since they are limited by domain knowledge and privacy constraints. Recent
advances in transfer and contrastive learning have provided effective solutions
to tackle such issues from different perspectives. The state-of-the-art
transfer learning (e.g., Big Transfer (BiT)) and contrastive learning (e.g.,
Simple Siamese Contrastive Learning (SimSiam)) approaches have been
investigated independently, without considering the complementary nature of
such techniques. It would be appealing to accelerate contrastive learning with
transfer learning, given that slow convergence speed is a critical limitation
of modern contrastive learning approaches. In this paper, we investigate the
feasibility of aligning BiT with SimSiam. From empirical analyses, different
normalization techniques (Group Norm in BiT vs. Batch Norm in SimSiam) are the
key hurdle of adapting BiT to SimSiam. When combining BiT with SimSiam, we
evaluated the performance of using BiT, SimSiam, and BiT+SimSiam on CIFAR-10
and HAM10000 datasets. The results suggest that the BiT models accelerate the
convergence speed of SimSiam. When used together, the model gives superior
performance over both of its counterparts. We hope this study will motivate
researchers to revisit the task of aggregating big pre-trained models with
contrastive learning models for image analysis.

Computer Vision and Pattern Recognition