Hello, I just realized that the expected output size of each layer on the table does not reflect the output sizes I observed.
I overlooked the padding and assumed it was 0 because the table did not specify it. Setting the padding = 1 resolves the main problem.
I then got the problem on the batch_size that @AbdulsalamBande mentioned. However, this is only a problem for training and setting the model to eval mode allows my model to test the dummy tensor.
Thank you for the help!