What is the problem you’re trying to solve? Are you trying to solve a segmentation/classification problem?
And what is outputs_mean? Is it a probability matrix with dimensions of [batch size, label] or is it a mask matrix with dimensions of [H, W] ? I think it would be easier to answer if you could elaborate on the question a bit more.