Thank you very much for your answer. I was 99% sure that they yield the same result, I just could not find the error on my code. But I think have just found it.
I was permuting the output channels after separating them. Like this:
classification = out[:, 0]  # (N, 32, 32)
regression = out[:, 1:]     # (N, 2, 32, 32)
regression = regression.permute(0, 2, 3, 1).contiguous()  # (N, 32, 32, 2)

Now I am permuting before separating the classifications regression outputs, and the regression results are something meaningful.
By the way, the kernel_size=2 on my question was a typo, it should be 1 on both layers. Is there any reason why you used kernel_size=3 in your examples?
Thank you again!