Solved by JuanFMontesinos in post #4 
              
              
                So then use a masking 
If I got it you has a tensor (N,M,P) 
mask = x!=0
y_mean = (y*mask).sum(dim=0)/mask.sum(dim=0)

I think this would fit what you want right?