Which metric to use to evaluate and compare meta models? I'm developing meta models based on Zero Inflated Model (sklego.meta.ZeroInflatedRegressor)
That is, they are models that have two tasks to perform:

*

*Train a classifier to tell us whether the target is zero, or not. If the classification is not zero, then...

*Train a regressor on all samples with a non-zero target.

Code example:
    #build the model
    model = ZeroInflatedRegressor(
    classifier=ExtraTreesClassifier(random_state=0),
    regressor=ExtraTreesRegressor(random_state=0))

    r2 = cross_val_score(model, scaled_train, y_train.values.ravel(), scoring = 'r2') 

Initially, I thought about using R², but I'm not confident in my choice.
Does anyone have any suggestions?
NOTE: my data are time series. Maybe this makes some difference.

Given that your outcome is a predictive density (i.e., for each future point in time, you predict the probability for an outcome of $0$, the probability for an outcome of $1$ and so forth), the standard evaluation metric is a proper scoring rule. We have a scoring-rules tag, and the tag wiki contains more information and pointers to literature. Much of the scoring rules literature indeed happens in a time series forecasting context, see e.g., the papers by Gneiting and coauthors.
