goenka04:

#plt.imshow(patches_orig) ==> THrows an error ypeError: Invalid shape (1, 64, 128, 96)


You can’t visualize more than 3 channels by matplotlib imshow here channel is 64. But you can visualize 2D slices to check the image.
plt.imshow(patches_orig[0, 32,...], cmap='gray') 
plt.show()

Or if you are using notebook IDE, follow this tutorial 3 to visualize 3D images.
After that, you have to create Dataset and DataLoader to make the images iterable, convert to tensor, and other augmentations/transformations to make it as following structure:
Batch_size, modality, H, W, D
If you are using only MR or T1 the modality should be 1.
Check out PyTorch and other tutorials for these basic things.
Sending the Dataloader sample patches to 3D CNN depends on what kind of CNN or network architecture you will be using and also what your application is.
For example, in the case of 3D UNet, the image sizes have to be divisible by 2 and have to be 4/5 magnitude multiples of 2 considering maxpool operation.
For example, in 2D UNet 224, 224 is a preferred dimension since both dimensions are divisible by 2: 224>112>64>32>16 …
Once you have the network chosen, you can easily feed the images as following:
input, target = next(iter(Dataloader)) # taking one sample
model = Chosen_CNN(**args)
output = model(input)

Also, I didn’t mention loss criteria, optimizers, and metric stuff that comes into the broad context.