Hi Rohit!



 Rohit_R:

The mental picture I create of ‘Batch’ is a collection of 3 images each of shape [3,256,256]. However you point out that this is not true.


Well, first, you could consider this a semantic distinction about
what “collection” ought to mean.  But leaving that aside …
Your mental picture is not unreasonable.  It does have aspects,
however, that could be misleading.
From a low-level technical perspective, in the simplest case pytorch
stores the data for a tensor of shape [3, 3, 256, 256] as a
contiguous array of 3 * 3 * 256 * 256 (in your case) bytes.
Ignoring strides and various kinds of views, you can’t build a
[3, 3, 256, 256]-tensor from 3 [3, 256, 256]-tensors without
copying that data from the three separate tensors into the contiguous
data array for the new [3, 3, 256, 256]-tensor.
From the perspective of functionality, if the batch tensor were a
(general-purpose) collection of three image tensors, you might
imagine building a batch that consisted of a [3, 256, 256]-image,
a [1, 256, 256]-image, and a [3, 128, 128]-image.  (Some
languages refer to such constructs as “ragged arrays.”)  But you
can’t.  The three images that make up the batch tensor all have to
have the same shapes.  They are “slices” of a higher-dimensional
tensor (rather than items in a general-purpose collection), and, as
such, their shapes are constrained to be the same.
Best.
K. Frank