See :
DL4J Resnet50: https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-zoo/src/main/java/org/deeplearning4j/zoo/model/ResNet50.java 2
Torch Resnet50 :
https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py
I got the layers information and printed them on console. As you can see in the image Zoo_Models_Resnet50702×688 14.7 KB -  the stride mismatch in second convolution operation (Torch uses 1 while **dl4j uses 2! ) . This results in decreasing the output volume rapidly in dl4j.