Llm, Model Selection, Model Evaluation, Genai.
summary), use semantic comparison metrics like BERTScore. It compares the semantic similarity between tokens of Ground Truth and LLM response. For open-ended tasks (e.g. Generative tasks) and tasks without Ground Truth use another LLM for evaluation. Studies show that LLMs have positional bias,