Hello! Recently I ran several experiments on OCR with CTC for my own research, and I would suggest you to try the following:
Instead of compressing some of the cnn output dimensions to one, in your case H is squeezed to 1, I would recommend to change CNN’s kernel_size to get wider dimension - thus you can use some pooling/attention operation over it:
# let input image size is (b, c, h, w) = (16, 3, 64, 1024)
# let cnn output size is (16, 2048, 128, 512)
# then

pooling =  nn.AdaptiveAvgPool2d((512, 512))

out = out.view(16, 2048, 128*512)
out = pooling(out)

Of course you can use classical maxPooling, or some custom self-attention, but in my experiments with IAM/Bentham I found the averagePooling best.
You can also try to use different dimension to multiply, for example (16, 2048*128, 512), etc.
Try to find your best.
Speaking about some background behind this, well, I think that we getting much wider feature map by keeping dimension not-equal-to-one, and by aggregation we can reduce the overall size, but also extract valuable feats.