Hi @Tupakula_Mallikarjun,
I fixed two things in my code. In my constructor on Data class I had forgotten to call super. So i added this in the code:
class Data(Dataset):
 def __init__(self,d,transforms=None):
  super().__init__()
  self.transforms = transforms
  self.data = mat73.loadmat(d)

Then I used dataloader to load images in batch of 2 and trained them. I saw some significant change with that but I would also advice increasing the number of epochs.
Another thing that I changed was instead of converting image using np.rot90 I did this:
img = cv2.cvtColor(i,cv2.COLOR_BGR2RGB).astype(np.float32)
img /= 255.0

I am not sure why the image thing helped but it did. Hope this helps!