FACTS is a great paper from @GoogleDeepMind on measuring the factuality of LLM outputs. You can now download their prompt templates from @huggingface to improve LLM-based fact-checking yourself!
๐ The paper introduces the FACTS Grounding benchmark for evaluating the factuality of LLM outputs.
๐ค Fact-checking is automated by an ensemble of LLM judges that verify if a response is fully grounded in a factual reference document.
๐งช The authors tested different prompt templates on held-out data to ensure their generalization.
๐ It's highly educational to read these templates to learn how frontier labs design prompts and understand their limitations.
๐พ You can now download and reuse these prompt templates via the prompt-templates library!
๐ The library simplifies sharing prompt templates on the HF hub or locally via standardized YAML files. Letโs make LLM work more transparent and reproducible by sharing more templates like this!