I’ll review how other python projects structure their test code and infrastructure, there may be a few ideas there that could be useful to this project.
Maybe non *_loader() functions are not needed? To simplify usage there could be some method helpers like:

def loader_from_tensor(tensor, batch_size=64, shuffle=False, pin_memory=False):
    y_unused = torch.Tensor(tensor.size(0))
    loader = torch.utils.data.DataLoader(
        torch.utils.data.TensorDataset(tensor, y_unused), batch_size=batch_size, shuffle=shuffle, pin_memory=pin_memory)

loader = loader_from_tensor(tensor)
loss = trainer.evaluate_loader(loader)
y = trainer.predict_loader(loader)

I like your MetaModelTrainer idea. Combined with KFold, sounds like a sweet thing to have! The CSV-like logger sounds pretty cool too.
Unrelated question: do you have any plans to merge your image transforms into pytorch/vision? I think this would avoid the need to have 2 implementations of ImageFolder and TensorDataset? And would make you code more accessible to PyTorch users.