I think the difference in the output comes form your manual padding which is only applied to one side.
You could split the padding and add half of the shape to both sides. F.pad might be easier to use than the manual torch.cat approach.