Solved by alfonsomhc in post #2 
              
              
                I got an answer in github (R-CNN predictions change with different batch sizes · Issue #4257 · pytorch/vision · GitHub): 

This is expected and is due to padding the differently-sized input images with zeros so that all images have the same size (for batching). 
If you crop all images to have the sa…