Right, something else that I have overlooked!
In that case, you need to have two different composed transforms, that you select accordingly when you create the datasets:
transform_valid = transforms.Compose([transforms.ToTensor()])
transform_train = transforms.Compose([transforms.ToTensor(), transforms.Normalize(...)])

In case you might want the images to stay images, and not tensors, you can also set transform=None when you call your dataset, but then you need something like this:
class class DatDataSet(Dataset):
....
def __getitem__(self,idx):
    img=pd.read_table(self.datpath[idx],header=None,sep='\s+')
    img=img.values;
    img_pil = Image.fromarray(img)
    if self.transform is None:
        img = img_pil
    else:
        img = self.transform(img_pil)
    sample={'data':img,'label':int(self.labellist[idx])}
    return sample

However, this will output PIL.Image objects.