I am not sure that using Cross Entropy does what I want it to; I suspect it trains it to maximise the class index, however in a single image I can have multiple classes present at various pixel locations. So far the network seems to learn to detect only the pixels of the class with the highest index?
Pixel wise accuracy is done by the following snippet (I had assumed softmax was needed), as such the code below won’t work without it, that is because each channel-per-class had a binary value.
           input = images.cuda(non_blocking=True).float()
            labels = labels.cuda(non_blocking=True).long()
            outputs = model(input)

            b_size = labels.shape[0]

            # iterate labels in dim 0, e.g., batch size
            for i in range(b_size):
                #
                #   squeeze and transform the [7,224,224] to
                #   a single dim [351232] of pixel values 
                #   all concat across one dim for all channels
                #
                label   = labels[i].view(1,-1).squeeze()
                actual  = f(outputs[i].view(1, -1).squeeze())
                l_size  = labels[i].shape[0]
                #
                #   pixel-wise comparison (note this requires softmaxed/min-maxed values)
                #
                actual  = (actual > 0.9).float() * 1
                same    = (actual == label).sum()
                total   = l_size * 224 * 224
                match   = same.item() / total
                #
                # 50176 is the 224 x 224 so we need to 
                # divide and normalise the correct values now
                #
                correct.append(match)
                total   += 1

I now suspect that what you had originally suggested (a target with single dim) would make more sense as it would allow me to argmax and calculate accuracy based on that.