AdaFilter: Adaptive Filter Fine tuning for Deep Transfer Learning
There is an increasing number of pre trained deep neural network models. However, it is still unclear how to effectively use these models for a new task. Transfer learning, which aims to transfer knowledge from source tasks to a target task, is an effective solution to this problem. Fine tuning is a popular transfer learning technique for deep neural networks where a few rounds of training are applied to the parameters of a pre trained model to adapt them to a new task. Despite its popularity, in this paper, we show that fine tuning suffers from several drawbacks. We propose an adaptive fine tuning approach, called AdaFilter, which selects only a part of the convolutional filters in the pre trained model to optimize on a per example basis. We use a recurrent gated network to selectively fine tune convolutional filters based on the activations of the previous layer. We experiment with 7 public image classification datasets and the results show that AdaFilter can reduce the average classification error of the standard fine tuning by 2.54%
AAAI