N1h1l1sT:

On the 2nd point, for this list-like batch, I don’t think there will be any fundamental change since predicting with different size images on this network already works, so long as you do it 1 picture at a time. Just, instead of iterating on the 1st dimension of N x C x H x W tensor, you iterate over the length of the list with C x H_i x W_i tensors as elements of the list


The issue is that the underlying implementations on CPU and GPU are usually written to take advantage of batching for data reuse. Of course, you can always write your own function that takes in a list of tensors, one per each image. The performance will likely be much lower than batching (even if it means zero-padding is used).
I am referring to changes in the underlying kernel implementations. These kernels are highly shape dependent (e.g., the best axis for data reuse changes depending on the relative sizes) of which batch size is an important parameter.