To get the non zero indices you could use .nonzero(as_tuple=True)
x[x.nonzero(as_tuple=True)].view(-1, x.shape[1])

will give you back the original tensor with all the zero tensors removed. Then you could just use .mean(dim=1) as usual