Hello Frank!
Thank you very much for your detailed reply. Your answer has cleared all my doubts!
On another note, could you please elaborate a bit more on your first statement " Batch is a single ‘holistic’ tensor of shape [3, 3, 256, 256]. It is not a ‘collection’. " The mental picture I create of ‘Batch’ is a collection of 3 images each of shape [3,256,256]. I also understand that this is just really a single 4D tensor. Is this what you meant by a ‘holistic’ tensor ?
Thank you again!