An input of [batch_size, 1024, 1, 1] won’t work for a standard resnet, as the conv layers use kernels with more than a single pixel and you are also using pooling layers, which won’t be able to decrease the spatial dimensions.
You could try to

reshape the encoder output, and create “fake” spatial dims (not sure, how well this would work),

expand the encoder output (and basically repeat the pixels),
use another model with e.g. linear layers
add transposed convolutions after the encoder to upsample the activation and pass it then to the resnet

I’m sure there are more options.