Is my xgboost model overfitting? (timeseries) I am very skeptical looking at this, I haven't used the xgboost library that much before, maybe someone can help me out. The errors and the predictions on my test set are too good to be true. I mean, I normally don't see this type of performance. Is there any thing I am overlooking here?
To reproduce:
library(xgboost)
library(caret)

m <- as.matrix(data)
k <- round(nrow(data)*0.95)
train <- m[1:k,]
test <- m[(k+1):nrow(data),]

m1_xgb <- xgboost(
  data = train[,2:ncol(train)],
  label = train[,1],
  nthread = 16,
  nrounds = 2000,
  objective = "reg:squarederror",
  early_stopping_rounds = 20,
  max_depth = 3, 
  eta = .25, verbose = 2)

pred_xgb <- predict(m1_xgb, test[,2:ncol(test)])
yhat <- pred_xgb
y <- test[,1]

plot(y,yhat, xlab = "Actual",ylab = "Predicted",main = "Actual",col="steelblue");box();grid()
abline(lm(yhat ~ y),col="red")
postResample(yhat, y)


plot(y,type='l',col="red");box();grid()
lines(yhat,col="green")
legend(200, 95.8, legend=c("Actual", "Predicted"),
       col=c("red", "green"), lty=1:2, cex=0.8)


     RMSE  Rsquared       MAE 
0.07043080 0.99728270 0.05669385  


Whenever performance is too good to be true, the big two suspects are usually

*

*an improper training/validation/test-split (the typical problem is not splitting time-series as past vs. future: when you randomly select validation records ignoring time, the model only needs to interpolate rather than to learn to predict into the future), and

*target leakage, where something in the predictors gives away the answer, but cannot be use in real-life for prediction (e.g. predicting death and using whether the date of death field is missing (or not) as a feature, calculating some feature using future values e.g. when interpolating missing values etc.).

It's hard to tell whether (2) could be occuring in your case, but assuming that your time series is ordered by time in the original data matrix m item (1) may not be the problem.
Why do I sound like I expect that there is a problem? Well, I find that when a model performance is too good to be true, it is very often one of these two things (rather than a surprisingly good model performance).
