Ok, from looking at the kaggle kernel quite a lot is different:

Keras seems to use batch size of 1, your PyTorch code uses 32
Keras is not normalizing, PyTorch is using normalization
PyTorch is using RandomResizedCrop…

And here might be the issue, you don’t specify any parameters for RandomResizedCrop.
By default you end up using the following:
torchvision.transforms.RandomResizedCrop(size, scale=(0.08, 1.0), ratio=(0.75, 1.3333333333333333), interpolation=2)

From doc:
A crop of random size (default: of 0.08 to 1.0) of the original size and a random aspect ratio (default: of 3/4 to 4/3) of the original aspect ratio is made. This crop is finally resized to given size. This is popularly used to train the Inception networks.
Using default parameters is too strong. (0.08 of the image can be a part of not too much information for your CNN). I would just use the same resize method and drop normalization as in Keras.
transform = transforms.Compose([
    transforms.Resize((150, 150)),
    transforms.ToTensor(),
    ])