hey thanks for responding. Yes I can show in code but I believe I can explain better here first, if thats okay.
So its a dataset of multiview images. I am feeding in 4 images at a time of the same scene from 3 different views. The network gives out 4 tensors for each of those 4 images. Right, in the loss I want to minimize two things. One is the simple MSE with the ground truth tensors. Alongside that I want to add one more loss to it. So I have the mapping where pixel 1 of image 1 lies in image 2, image 3 and image 4. So I want to go make sure the network predicts the same thing over the 3 images. More specifically if pixel(0, 0) in image 1 is  (10, 10) in image 2, (20,20) in image 3 and (45, 45) in image 4 then for the 4 out put tensors values should be equal over those 4 indices. Otherwise some loss value is gonna get added alongside the MSE.
Also I would like to mention I am doing regression not classification. Each of those tensors are of size 500x500x13. So for each pixel you have 13 values, each ranging from [0-1]. The input is RGB image sized 500x500x3.