Suppose you train a standard CNN classifier, where each image in the training set either shows a cat or a dog. Such a model would ideally evaluate an image with neither animal as 50-50, and the same for an image with both a cat and a dog in it.
Can these cases be reliably distinguished by inspecting the pre-softmax outputs? You might expect two low activations in the former case, and two high ones in the latter?
Sorry if this is well-known. I’m sure it must has been studied, but a quick google search didn’t help.