MichaelMMeskhi:

since I am using a ReLU as my non-linear activation, in the penultimate layer, obviously I end up with some 0 values. Would using a LeakyReLU be a better approach?


You would have to check the model performance by switching the last non-linearity.
Note that zero outputs don’t necessarily mean a “bad” behavior. If your model is able to successfully predict the targets, this activation (with zeros) seems to be a good representation such that the last linear layer can act as the classifier and compute the outputs.



 MichaelMMeskhi:

What I meant in regards to your second part, was that I have around ~500 features and would like to do some sort of feature engineering. I know a NN technically does that by learning a new representation and such. But what I meant was is there a specific way to achieve that or it just depends on what my problem is and what I want.


Are these 500 features the input features or are you creating them in the model as an intermediate activation?
Unfortunately, I don’t know what the current method would be to perform feature selection in a neural network. If I remember it correctly, I’ve read some papers which claim that e.g. the parameter or gradient magnitude does not correlate to the “importance” of these units. However, I cannot find the reference at the moment.