Thanks @arnowaczynski for the accurate description of the case. Indeed you can end up having channels last inputs and not optimized for it hardware.
The only thing I can add here is that images_tensor = images_tensor.to(device="cuda", memory_format=torch.contiguous_format) will work faster than images_tensor.contiguous()