Is it correct to replace the oldest data when using Gaussian Process in blackbox hyper-parameter optimization? I try to apply GPR in a blackbox HPO question. My input will have 6 dimensions like X=[x1,...x6]. The implementation is quite straightforward with sklearn with a Matern core. currently, I collect data from online result, and keep adding into training data and fit on training data. Based on the fitted one, get max point with acquisition function and replace current point used online. This will be different from most situation in most tutorials. Is this process reasonable? Do I need to remove the oldest data point from training data after a period of time or keep it?(could I assume the distribution will keep same day by day or I need to consider it change with time and remove some old data points)

There's no need to remove the old data points. You can just append the data and grow the list of queried points as you proceed. The reason that you would not want to remove the oldest data is that this will increase the posterior uncertainty in the region of the removed points, and then that will influence how the acquisition function will select the next query point. You might even find that removing the old data causes the optimization to re-visit the location from which you removed the old data (because of the high uncertainty in that region).
