I am trying to use a simple 3 layer neural net to predict a scaler output given an input of dimension 430. For my network, I use 2 layers of dimensions 600 and 80 and I use leakyReLU non-linearities. I also tried to do the regression using ridge regression. My solution for regression is better than my pytorch implementation. I was wondering what. I have been playing around with layers, dimensions, non-linearity, learning rate but non improved the results. I was wondering if this is something to be expected and if not, what else would you suggest analyzing.