Solved by googlebot in post #4 
              
              
                That function is just not easy to learn, starting from noise. It also doesn’t help that inputs and outputs (target bias=25) are far from zero. 

amount of training may be too low (lr_rate and number of epochs). lr_scheduler may also be needed to fine-tune curves
you’re not using quadratic loss (MSE)…