FTravi:

Maybe you could shed some light on why this would make a difference, if it makes a difference at all.


I’m not sure what you mean.
My argument is, that the feature extractor should yield an empty activation in both frameworks, if the model is created correctly and all parameters are equal.
I guess Caffe/Lua might use another rounding approach in calculating the output shape, which could yield a 1x1 activation in the end, and you could check it manually in both frameworks.
I’m unfamiliar with the paper, but find it a bit strange, that the last pooling layer is removed from the stimuli model, which gets the larger inputs, and not from the target model. Feeding the small input to the stimuli model works fine:
model_stimuli = nn.Sequential(*list(model.features.children())[:-1])
model_target = nn.Sequential(*list(model.features.children()))

out = model_stimuli(torch.randn(1, 3, 28, 28))
print(out.shape)
> torch.Size([1, 512, 1, 1])