I agree. Still, there may be something to the intuition that the raw output of the “cat node” node should correlate with the presence of cats, even if you have K-1 other animals categories. It may of course be that the “cat node” really represents the signal “lack of non-cat animal”.
I’m not hoping for a mathematical theorem (although that would be nice), but wonder if it has been studied empirically. Since it’s such an obvious idea, I guess something would have been published on this.