Oh right, second axis makes sense. 
Well, this should work:
avg = A.sum(axis=1) / (A.sum(axis=2) > 0).sum(axis=1).view(-1,1).float()

In the denominator, we are counting the number of non-zero arrays on the second axis. We can then rely on broadcasting 3 for the division.