Deep neural networks are highly effective tools for human and animal pose estimation. However, robustness to out-of-domain data remains a challenge. Here, we probe the transfer and generalization ability for pose estimation with two architecture classes (MobileNetV2s and ResNets) pretrained on ImageNet. We generated a novel dataset of 30 horses that allowed for both within-domain and outof-domain (unseen horse) testing. We find that pretraining on ImageNet strongly improves out-of-domain performance. Moreover, we show that for both pretrained and networks trained from scratch, better ImageNet-performing architectures perform better for pose estimation, with a substantial improvement on out-of-domain data when pretrained. Collectively, our results demonstrate that transfer learning is particularly beneficial for out-of-domain robustness.
= Summary 
This paper analyzes the effect of ImageNet pretraining on out-of-domain visual recognition. Specifically, in this paper, the recognition problem is narrowed to pose estimation on a horse profile image dataset where the out-of-domain indicates horse IDs unseen during training. The paper presents a new horse pose estimation dataset and extensive experimental analysis to demonstrate the benefit of ImageNet pretraining. 
= Decision
I would recommend to reject this submission mainly due to the shortcomings of the proposed dataset, which make the analysis and conclusion of the paper unconvincing.
First, the dataset is very limited in terms of diversity. It only contains 8K horse profiles images, each of which contains only a single horse, and only 30 horses of the same species appear in the images. Furthermore, since the dataset are sampled from video sequences, images of the same horse ID could be too similar in terms of appearance. Thus, all the images in the dataset seems within a single and very specific domain, "profiles of a small number of Thoroughbred horses", and not appropriate to evaluate "out-of-domain" robustness of pose estimation networks in consequence.
Second, the dataset split strategy is weird. Images of 10 horse IDs are considered as the "within domain" dataset while the others as "out-of-domain" dataset, and a subset of the within domain dataset is used for training or finetuning the pose estimation networks. This means that the networks could seriously overfitted to the small image set of 10 horse IDs. Thus, it is obvious that the networks will work very well with and without ImageNet pretraining on the within domain test set in which exactly the same 10 horses appear, and that they will not work well on the "out-of-domain" dataset due to the overfitting issue. 
Also, focusing only on the "horse" class looks not proper. Note that ImageNet already contains many horse images, and ImageNet pretrained networks would have a capability to extract horse-related features. Thus, the advantage of ImageNet pretraining on the horse pose estimation task is not surprising but a result that many in this field can easily expect. If this is not a big issue, I rather would like to recommend to exploit existing human pose datasets (e.g., MPII) since they are larger enough in size, and guarantee a larger variety of poses and person appearances than the proposed horse dataset.
= Other comments
The manuscript is overall written clearly, but it is hard to understand the curves in the figures: colors are not clearly distinguishable (e.g., red vs. magenta), the roles of "MobileNets" and "ResNets" placed on top of upper horizontal bars are not unknown too (if they indicate that the top-2 accuracy scores are given by Resnets variants, why are the curves of MobileNets and ResNets connected?)