Teach your LLM to answer with facts, not fiction
'Facts' aren't as black and white as people think.   "What does Charmander evolve into?"
   "What does the spell 'avada kedavra' do?"
   "What is the Sindarin word for 'friend'?"
   "What are the names of Santa's reindeer?"
   "Where did Robin Hood live?"
   "Where did Achilles die?"

These are all 'factual questions' you can find answers to from reputable sources like Wikipedia. Google displays 'fact boxes' for several of them. Wolfram Alpha provides answers for three of them. The answers to some of these questions are part of what passes for 'general knowledge' in some societies.It's no surprise that LLMs trained on human writings produce text which claims things that aren't true are facts. Humans do that all the time.There are well attested reputable sources that will tell you Abraham Lincoln was a vampire hunter, others that say he was a Lego Master Builder, and others still will tell you that among his notable quotes is "Party on dudes - be excellent to each other". So what's an LLM to do when it's trying to extend a paragraph of information about Abraham Lincoln?When an LLM is suggesting what might come next in a piece of text... it doesn't know if it's supposed to guess a probable word from a Wikipedia article, an Onion article, a Project Gutenberg manuscript, or an Archive Of Our Own fanfic. So you get a bit of all that.