@ptrblck
Sorry for coming back, I faced an error on your last answer for old version of pytorch  (0.3.0)
so it works when i test, but when my input is [torch.cuda.FloatTensor of size 1x1024x50x89 (GPU 0)] it trowed me an error that *** AssertionError: MaskedFill can't differentiate the mask and i had to change the lst two lines to:
       z = (x > x.view(B, C, -1).mean(2)[:, :, None, None]).float()
       z = (x <= x.view(B, C, -1).mean(2)[:, :, None, None]).float()