Solved by Federico_Pala in post #2 
              
              
                Hey, you want to go from 41x41 to 321x321? If you upsample 8 times with conv transpose 2d you get 328. Then a conv filter of size 8x8, you should get 321x321