You can use the RMSE metric, like so:
from sklearn.metrics import mean_squared_error

def compute_metrics(eval_pred):
    predictions, labels = eval_pred
    rmse = mean_squared_error(labels, predictions, squared=False)
    return {"rmse": rmse}

Sources:

scikit learn - Is there a library function for Root mean square error (RMSE) in python? - Stack Overflow 1