Llm, Model Selection, Model Evaluation, Genai.
APIs Open AI Evals — This is an open source framework for comparing responses from multiple LLMs. It can be represented as yaml file, give an input data set, select a class that performs eval (e.g. Matching), metrics (e.g. accuracy) and start the evaluation. Hugging face evaluate: