Generally, a neural net fits a piecewise function of x, and there are no x*x terms in a network. Thus, adding x^2 input is beneficial.



 Martijn:

Why does providing x^3 to the model result in such a poor result?


Because cubic polynomial root finding is more difficult that quadratic, indeed it becomes a nonconvex problem, I believe.
If you mean a problem with quadratic target equation, perhaps it is just a numerical issue, as your x^3 column range is -1e9…1e9, while it should have zero effect after training.