Thanks for getting back!
I guess the pattern is created by the last nn.ConvTranspose2d layer, which uses a kernel height of 1 and a stride of 2 in both spatial dimensions.
This will skip every second row. Sorry for missing this.
You could of course change the overall architecture and try to use overlapping or neighboring kernels.
This is a quick and dirty fix using the current architecture and increasing the last kernel dimension:
model = nn.Sequential(
    nn.ConvTranspose2d(5, 2, (2, 2), 2),
    nn.ConvTranspose2d(2, 2, (2, 4), 2),
    nn.ConvTranspose2d(2, 2, (2, 3), 2),
    nn.ConvTranspose2d(2, 2, (2, 3), stride=2, padding=(1, 0), output_padding=(1, 0)),
)

Let me know, if that works better.