Transfer learning, in which a network is trained on one task and re-purposed on another, is often used to produce neural network classifiers when data is scarce or full-scale training is too costly. When the goal is to produce a model that is not only accurate but also adversarially robust, data scarcity and computational limitations become even more cumbersome. We consider robust transfer learning, in which we transfer not only performance but also robustness from a source model to a target domain. We start by observing that robust networks contain robust feature extractors. By training classifiers on top of these feature extractors, we produce new models that inherit the robustness of their parent networks. We then consider the case of "fine tuning" a network by re-training end-to-end in the target domain. When using lifelong learning strategies, this process preserves the robustness of the source network while achieving high accuracy. By using such strategies, it is possible to produce accurate and robust models with little data, and without the cost of adversarial training. Additionally, we can improve the generalization of adversarially trained models, while maintaining their robustness.
Paper summary: This paper explores the problem of robustly transfer learning using only standard training (as opposed to adversarial training (AT)) on the target domain. The authors start by highlighting that intermediate representations learned by adversarially trained networks are themselves fairly robust. Then they propose two strategies for robust transfer from a robust model trained on the source domain: (1) naturally fine-tuning the final linear layer on the target domain and (2) naturally fine-tuning all the layers using lifelong learning strategies. They study transfer between CIFAR10 and CIFAR100, as well as, from ImageNet to CIFAR10/100.
High-level comments: Overall, I find the paper interesting and well-written. Prior work from Hendrycks et al. showed that AT on the source domain followed by *adversarial* fine-tuning on the target domain attains better performance as compared to just AT on the target domain. The main contribution of this paper is to show that using instead careful *natural* fine-tuning on the target domain is sufficient to recover a reasonable amount of this robustness. 
Even though the clean/robust accuracy of the proposed approach is lower than just doing adversarial training/prior work from Hendrycks et al., I feel this paper could be useful to the community for two main reasons:
1. The authors perform a nice exploration of the thesis that robust models have robust representations, and how this connects to transfer learning. In particular, the experiments in Figure 1 (the effect of naturally re-training later layers of a robust network on its robustness) and Figure 5 (where the authors show that their naturally fine-tuned models have some of the unexpected benefits from Tsipras et al.) seem particularly interesting.
2. Despite its lower accuracy, this approach could be useful in settings where data is scarce or compute is expensive, and hence adversarial training on the target domain is not successful. 
Specific comments/questions:
i. Could the authors clarify what they mean by point 3 (re: validation accuracy drop) below Table 3? As far as I can tell, the drop in clean validation accuracy between Table 1 and Table 2 are similar for both the naturally and adversarially pre-trained models.
ii. In Table 2, it would also be interesting to see the performance when the source domain is CIFAR10 and the target domain is CIFAR100.
iii. For Table 2 is the eps=8? This should be mentioned in the caption as it is important to highlight that Tables 2 and 3 are not directly comparable.
iv. Why isn’t experiment in Figure 3 should be repeated for CIFAR10 as well? The authors should add this result to the paper, even if in the appendix. It is important to verify that this trend is not specific to CIFAR100 and holds across datasets (even though CIFAR10/100 are not too different).
v. The comment at the end of page 6 re: natural model is confusing (“Note, this seems to...perfectly”)---as far as I can tell, Figure 4 does not include the results of fine-tuning a naturally trained model.
vi. General comment motivated by the comment ("Note...perfectly") mentioned 5 above: For all the adversarial evaluation in the paper, the authors should also try CW attacks/black-box attacks to get a more confident estimate of their model’s robustness.
vii. The authors reference prior work on the tradeoff between robustness and accuracy and motivate Section 6 as an avenue to alleviate this trade-off for their model. However, I don’t see the lower performance of their model as an instance of this trade-off---the model in the paper performs worse in terms of both clean and adversarial accuracy. The approach proposed in Section 6 seems interesting, but more as an approach to improve the *overall* performance of the model. The authors mention this in retrospect, but I think the narrative of this section should be modified to make this clearer.
viii. In Table 5, in the experiments corresponding to CIFAR100+ -> CIFAR100, is the dataset split into two halves (for the source and target domains) or is the fine-tuning performed on the same data. In general, I find it odd that natural fine-tuning on the *same data* can improve both the clean and adversarial accuracy of the model (compared to the CIFAR100+ robust baseline). Is the robust model trained long enough/with enough hyperparameter search?
Overall, the exploration in the paper seems novel and could be useful to the community. Thus, I recommend acceptance.