@tom told me the following, which makes sense:

Usually the PyTorch API follows CuDNN. This is the case here, too, where CuDNN only offers symmetric 0 padding (as far as I know). I’ve not looked at the fallback cuda implementation, but unless it’s a lot more efficient (relative to “what’s possible”) than the CPU one, there just isn’t much point not split out the padding.