Why do not more data points reduce the error of Gaussian Process Regression? I am exploring using GP to approximate the value functions in some dynamic programming problems. Since all my data points are generated by myself, I can always train the GP with a larger dataset. However, I found that after reaching a certain threshold, adding more data points is not helpful for me to reduce the RMSE of GP predictions on the test set. That is, switching from 50 to 500 data points can improve my GP's performance a lot. But if I switch from 500 to 2000 data points, the RMSE is effectively the same. Does that suggest that there is something wrong with my codes? Or is it a feature of GP regression?

Your data-generating process might have some inherited variability/noise. Ultimately that variability is irreducible. Putting this in the context of simple linear regression: where $y \sim N(X\beta, \sigma_{\text{error}}^2 I)$. Even if we have infinite training and testing points, our $\sigma_\text{error}^2$ estimate will not get smaller in terms of magnitude, just more accurate in terms of the estimated value. In that regard, for classification, there is the concept of Bayes error rate, which again is our lowest possible error rate for a given dataset, it's the same idea just with class densities instead of "measurement noise".
