Hey @NDugar, the simplest way to do this would be with the accuracy metric in the datasets library, e.g.
from datasets import load_metric

# You might need to install scikit-learn
metric = load_metric("accuracy")
results = metric.compute(references=[0, 1], predictions=[0, 1])
results
# {'accuracy': 1.0}

As you can see in this example, the main bit of work you need to do is create the arrays for the references (i.e. ground truth labels) and predictions (predicted labels). If you’re using the Trainer you can get these arrays easily by calling Trainer.predict(test_dataset)