yeah that makes sense that the engine should only be implemented once and I get that use case. The class you’re proposing might also make training GANs a lot easier as well.
Tests are obviously much needed… The metrics are sort of their own thing and only interact with the TQDM class - not History . Here’s how to use them:
from torchsample.metrics import CategoricalAccuracy

# get a prediction -> shape = (samples, classes)
y_pred = model.predict(x_train)
# y_train is ground truth  -> shape = (samples,)

acc = CategoricalAccuracy(top_k=2)
score = acc(y_pred, y_train)

So, they act ust like a loss function… testing them should be straight-forward then.
Right now, I’m focusing on finishing the predict and evaluate functions, as well as what to do with the *_loader() functions because it gets a lot more complex to handle multiple inputs/targets and especially to handle an optional target.
Another reason I like your idea is because it might make hyper-param optimization (something like a MetaModelTrainer) much easier … some way to train a model over multiple datasets/splits, or train multiple models over a single dataset, and keep track of those experiments through a CSV-type logger would be great.