Thanks for the toy code!
I tried using it, but here I also get an error I could not solve.
The line loss = criterion(output, mask) leads to:

cuda() takes from 1 to 2 positional arguments but 3 were given

With the original code I tried increasing the batch size as well as the spartial size of die initial input. Both did not change anything. It is still the same error. Shouldn’t it work without changing the pooling operation of the model?
Thanks