Hey @ptrblck I appreciate your feedback! Yes, that’s what I ended up using. My question is since I am using a ReLU as my non-linear activation, in the penultimate layer, obviously I end up with some 0 values. Would using a LeakyReLU be a better approach?
What I meant in regards to your second part, was that I have around ~500 features and would like to do some sort of feature engineering. I know a NN technically does that by learning a new representation and such. But what I meant was is there a specific way to achieve that or it just depends on what my problem is and what I want.