Solved by R_B in post #3 
              
              
                Finally I changed my method : every time I call net(image), I make sure to interpolate the image to the desired size with torch.nn.functionnal.interpolate. This way I only have one dataset. 
Thank you ! 
`