You would have to check the model performance by switching the last non-linearity.
Note that zero outputs don’t necessarily mean a “bad” behavior. If your model is able to successfully predict the targets, this activation (with zeros) seems to be a good representation such that the last linear layer can act as the classifier and compute the outputs.

Performance is good. And yes, having a 0 doesn’t mean its bad but if I were to extract penultimate layer activations, I could simply drop the 0 valued columns and thus have even reduced dimensions.

Are these 500 features the input features or are you creating them in the model as an intermediate activation?
Unfortunately, I don’t know what the current method would be to perform feature selection in a neural network. If I remember it correctly, I’ve read some papers which claim that e.g. the parameter or gradient magnitude does not correlate to the “importance” of these units. However, I cannot find the reference at the moment.

Yes, those are the input features. I see. The parameter reference seems interesting.
Thank you!