the normalization that I used is z_score but I also tried min max scaling, both gave the same results! the project is a research project so there is not much resources about it, if any. the last thing that I can think of is using LSTM but somehow I lost hope using NNs since random forest gave better results, however I lost the whole day trying to reduce the overfit of the random forest regressor but I didn’t achieve that. As you suggested I already tried fitting with one feature but that didn’t worked, in fact sometimes it gave worse performance.
Yes my Net learn in the first epochs and then stops working somehow, I’d like to think about it that it is struggling to get out or rather break that loss value. I tried cyclic LR, LR Scheduler but it also didn’t work. I’m run out of Ideas