I believe the standard approach is to use a Center Crop although different datasets may use different “rules” sometimes. Note that you may want to resize the images first (e.g., examples/main.py at 01539f9eada34aef67ae7b3d674f30634c35f468 · pytorch/examples · GitHub 1) before taking a center crop due to the variation in resolution of images.