From what I’ve seen, in pytorch “elementwise” kernels just use hardcoded 512 threads per block (and data is processed like a flat 1d view). And kernels with special dimensions (“vector kernels”) are mostly dispatched to libraries like cudnn, cublas, so these select block sizes somehow…