Solved by ptrblck in post #2 
              
              
                Your last conv layer in your encoder throws this error, since it receives an input of [1, 30, 1017, 1] and tries to use kernel_size=3. 
You have to lower the kernel size to 1 or modify your model somewhere before this layer.