Thank you @ptrblck and @vgsprasad
I got it to work and I did remove the softmax (thank you ptrbick for the explanation for that)
This is how my forward feed is now
  def forward(self, x):
        x = self.model.conv1(x)
        x = self.model.bn1(x)
        x = self.model.relu(x)
        x = self.model.maxpool(x)

        x = self.model.layer1(x)
        x = self.model.layer2(x)
        x = self.model.layer3(x)
        x = self.model.layer4(x)
        x = self.model.avgpool(x)

        x = x.view(x.size(0), -1)
        x = nn.functional.relu(self.fc1(x))
        x = self.dropout(x)
        #x = nn.functional.softmax(self.fc2(x), dim=1)
        x = self.fc2(x)
        return x

And it even works better than the one I did in keras.
I really appreciate your advice.