Thanks for the code.
It is indeed an out of memory issue.
You are currently using a batch size of the complete validation dataset via:
val_data = ValDataset()
valloader = DataLoader(dataset=val_data, batch_size=len(val_data), num_workers=0, shuffle=False, drop_last=False)

Based on the previous outputs of your notebook, the length of the validation dataset should be 130000 - 117000 = 13000, which will use too much memory.
I’ve tested different shapes and it seems a batch size of ~5000 would be the limit using approx. 44GB of your GPU.