Hi Max!



 max_mz:

1 : torch.nn.Upsample + torch.nn.Conv2d
2 : torch.nn.ConvTranspose2d


Upsample plus Conv2d and ConvTranspose2d would do similar
things, but they differ distinctly in detail.
Use Upsample (without Conv2d) if you want cheaper upsampling,
but without trainable parameters, and use ConvTranspose2d if you
want the trainable parameters.
Although you could add Conv2d to Upsample to get trainable
parameters and similar functionality to ConvTranspose2d, you
wouldn’t get the full benefit.
Consider ConvTranspose2d with a 2x2 kernel.  It, in effect, duplicates
each pixel to form a 2x2 block of identical pixels, and then applies
each of the 4 in_channels x out_channels slices in its 2x2
height x width spatial array as single-pixel “depthwise” convolutions
to each of the 4 pixels in the 2x2 block.
If, instead, you use Upsample to explicitly create that 2x2 block
of identical (or non-identical, but similar interpolated pixels), and
then apply a Conv2d with a 2x2 kernel, its 4 in_channels x
out_channels depthwise convolutions get summed together,
because they are summed across the 4 identical pixels in the 2x2
block (or blended together if Upsample used interpolation).
So you have the same number of parameters and cost of training
as using ConvTranspose2d, but you’ve thrown away (summed
together) three-fourths of that expressiveness and trainability.
Best.
K. Frank