Why do neural networks need a lot of data compared to other ML algorithms? I'm specifically referring to graphs like this. It's clearly shows how for small amount of data traditional machine learning algorithms or simple statistical learning perform better than neural networks.


You're asking so much of a neural network.
In a simple linear regression, you're giving considerable constraints on the model.
$$
\mathbb E[y_i]=\beta_0 +\beta_1x_i
$$
As $x$ increases by $1$ unit, $y$ changes by $\beta_1$. That's all there is to the relationship.
In a large neural network, you're asking the model to figure out everything. If there is supposed to be an interaction between input features, the model has to figure it out; you didn't tell the model to use an interaction feature. If there is supposed to be curvature, the model has to figure it out; you didn't tell the model to consider curvature.
In some sense, the neural network determines the regression parameters and also has to figure out the features to which those parameters correspond.
Meanwhile, there is noise, where the observed value is not exactly the conditional expected value. The model has to sort out what it signal and what is noise. When there is a stray point, is that because there is a large residual or because the model should curve towards that point? Without having a lot of data, you can't know, but the model wants to fit the given points as well as it can, so it is inclined to try to get near that point and have a small residual. If that is the wrong choice, then it has overfit to the noise and will struggle to give generalizable results.
