Thank you @ptrblck. It took a little work and trial and error but I got it working.
train_transform = transforms.Compose([
    transforms.Resize(size),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(10),
    transforms.ToTensor(),
    transforms.Normalize(mean, std)
])

From this to this
train_transform = transforms.Compose([
    transforms.Resize(size),
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(10),
    transforms.ToTensor()
    transforms.RandomErasing(p=0.3)
])

I had to remove the normalization for it to work I think, but as you mention putting it on the end after the tensor seems to do the trick thank you