When I add a custom compute_metrics function to the Trainer, I get the warning “Not all data has been set. Are you sure you passed all values?” at each evaluation step.
This warning is defined in the finalize function of the class trainer_pt_utils.DistributedTensorGatherer:
if self._offsets[0] != self.process_length:
logger.warn(“Not all data has been set. Are you sure you passed all values?”)
This is my compute_metrics function:
def compute_metrics(eval_pred):
preds, labels = eval_pred
preds = np.argmax(preds, axis=1)
accuracy = round(accuracy_score(labels, preds),3)
micro_f1 = round(f1_score(labels, preds, average = "micro"),3)
macro_f1 = round(f1_score(labels, preds, average = "macro"),3)

return {"Accuracy": accuracy, "Micro F1": micro_f1, "Macro F1": macro_f1}

The additional metrics are successfully returned. So what does this warning mean? Any help would be much appreciated.