Solved by ptrblck in post #2 
              
              
                You won’t be able to calculate the input values of the mean calculation. 
However, if you are more concerned about the shapes (not the exact values) you could use e.g. nn.ConvTranspose2d.