Could you post the model definition in Keras as well as PyTorch?
I guess you might use different conv settings, e.g. using padding='same' in Keras while not using any padding in PyTorch, which would yield different output activation shapes.