Nvidia GPUs are only able to launch a limited number of threads (ex. 1024 for 1080ti) in parallel. I was wondering how pytorch adjusts grid and block size to deal with this limitation when input size is beyond the maximum parallel capability. For example, when input size is (128, 3, 270, 360) (in NCHW), even if it can fit into GPU memory, it cannot be processed in parallel altogether. I can think of splitting the large input into multiple chunks so that each chunk can be executed in parallel in GPU, and execute all of input chunks iteratively. But i believe doing this optimally maximizing GPU utilization is not that simple. How is pytorch dealing with this issue? It would be very much appreciated if any of you guys can point out source codes that i can refer to.