I have a few questions:

From what I can observe, your dataset is basically an image, stored in the form of a csv(correct me if I am wrong), why don’t you store it as an image or nifti instead?
Have you tried shuffling your dataset?
Can you show your network?