After preprocessing (I did resampling and converted to SUV)my dataset ,the image size was [284, 143, 143].
Then I extracted patches(applied padding and used unfold function) to get  patches of size torch.Size([7, 4, 4, 64, 64, 64]) which are a total of 112 patches of 64 * 64 * 64.  for patch_A and patch_B both.
A and B refer to two different modality
Then I created a custom dataloader and splitted my images for training .