GoogLeNet &quot;won&quot; ImageNet 2014[1], but VGG is often found to be more flexible.<p>I&#x27;m not entirely sure, but I think that the multiple softmax layers in GoogLeNet might make it easier to modify for this purpose than the VGG architecture.<p>[1] <a href="http:&#x2F;&#x2F;image-net.org&#x2F;challenges&#x2F;LSVRC&#x2F;2014&#x2F;results" rel="nofollow">http:&#x2F;&#x2F;image-net.org&#x2F;challenges&#x2F;LSVRC&#x2F;2014&#x2F;results</a> (look for &quot;Classification+localization with provided training data: Ordered by classification error&quot;)