You may just need some padding at the right place. What I usually do in these situations is the goode olde print() debugging, printing the sizes of the tensors after each convolution to better track what’s going on.
You can calculate the “same” padding for convolution via
(w - k + 2*p)/s + 1 = o
=> p = (s(o-1) - w + k)/2
where p is the padding amount, w is the input width, k is the kernel size, and o is the desired output size. In your case you need to do that for both height and width separately.