One of the most commonly recommended methods of reducing hallucination in LLMs is called "grounding" in which the LLM is asked not to answer from its own knowledgestore, but rather is given the answer (such as through ANN search) and asked merely to summarize or otherwise distill it down for the user. The (incorrect) underlying assumption is that hallucination derives from mismatches between reality and what is encoded in the LLM's knowledgestore. By providing it the correct answer and switching it to a summarization, rather than reference, task, the idea is that hallucination will be reduced or eliminated because the LLM need only reword the provided information, rather than utilize its own encoded information. The problem with this is that all distillation tasks like summarization still entirely rely upon that knowledgestore to generate their outputs, meaning that they will still freely hallucinate even in a grounded scenario. As the results below demonstrate, neither grounding nor setting temperature to 0.0, both widely-touted strategies that are commonly promoted by poorly-trained AI consultants as quick methods of eliminating hallucination, fail.
Take for example this trivial toy example of a news summarization task:
Divide the television news transcript below into a set of discrete stories, make them into a bulleted list with a title and short description and summary of each. Each story should include a Title, Description and Summary.\n\nTELEVISION NEWSTRANSCRIPT:\nWelcome to tonights broadcast. Our first story is that Donald Trump announced he was running for president again. Joe Biden is traveling to China next week to discuss Ukraine.
Let's run it through Bison-32K with a temperature of 0.2 and leave all other parameters at their defaults. Note how it correctly identifies the two stories, but adds a variety of detail not present in the original text, hallucinating details about the year, location, slogans, leaders and topics:
**Donald Trump Announces Presidential Run**
Description: Former President Donald Trump has announced that he will be running for president again in 2024.
Summary: Trump made the announcement at his Mar-a-Lago estate in Florida. He said that he was running to "save America" and to "make America great again."
**Joe Biden to Visit China**
Description: President Joe Biden will be traveling to China next week to discuss the ongoing conflict in Ukraine.
Summary: Biden will meet with Chinese President Xi Jinping to discuss ways to end the war in Ukraine. The two leaders will also discuss other issues, such as trade and climate change.
Let's try to constrain the LLM by instructing it not to use information beyond what is in the transcript:
Divide the television news transcript below into a set of discrete stories, make them into a bulleted list with a title and short description and summary of each. Each story should include a Title, Description and Summary. Do not use any information beyond what is in the transcript below.\n\nTELEVISION NEWSTRANSCRIPT:\nWelcome to tonights broadcast. Our first story is that Donald Trump announced he was running for president again. Joe Biden is traveling to China next week to discuss Ukraine.
Unfortunately the result is identical:
**Donald Trump Announces Presidential Run**
Description: Former President Donald Trump has announced that he will be running for president again in 2024.
Summary: Trump made the announcement at his Mar-a-Lago estate in Florida. He said that he was running to "save our country" and to "make America great again."
**Joe Biden to Visit China**
Description: President Joe Biden will be traveling to China next week to discuss the ongoing conflict in Ukraine.
Summary: Biden will meet with Chinese President Xi Jinping to discuss ways to end the war in Ukraine. The two leaders will also discuss other issues, such as trade and climate change.
What if we set the temperature to 0.0, as is also commonly recommended to eliminate hallucination? The results are nearly identical, with only the first Trump slogan removed:
**Donald Trump Announces Presidential Run**
Description: Former President Donald Trump has announced that he will be running for president again in 2024.
Summary: Trump made the announcement at his Mar-a-Lago estate in Florida. He said that he is running to "Make America Great Again" again.
**Joe Biden to Visit China**
Description: President Joe Biden will be traveling to China next week to discuss the ongoing conflict in Ukraine.
Summary: Biden will meet with Chinese President Xi Jinping to discuss ways to end the war in Ukraine. The two leaders will also discuss other issues, such as trade and climate change.
What if we set temperature all the way to 0.99 to maximize randomness?
Here are tonight's top three stories:
**Donald Trump Announces 2024 Presidential Run**
_Description:_ Former President Donald Trump announces his candidacy for president in 2024.
_Summary:_ Trump made the announcement at his Mar-a-Lago estate in Florida, surrounded by family and supporters. He criticized the Biden administration and promised to "Make America Great Again" again.
**Joe Biden to Visit China Next Week**
_Description:_ President Joe Biden will travel to China next week to discuss the war in Ukraine
_Summary:_ Biden will meet with Chinese President Xi Jinping in Beijing. The two leaders are expected to discuss the war in Ukraine, Taiwan, and other regional security issues. The White House said the meeting will be "an opportunity to continue our ongoing dialogue with China and build on the progress we have made in recent months."
**Winter Storm Hits East Coast**
_Description:_ A massive winter storm is hitting the East Coast, bringing heavy snow, ice, and wind.
_Summary:_ The storm is expected to impact millions of people from Virginia to Maine.Blizzard warnings are in effect for parts of the region.
Interestingly, if given the same prompt, a temperature of 0.2, but no story at all, we get the following results, showing just how vivid hallucination can be:
**STORY 1**
Title: Florida Man Arrested for Stealing 100 Golf Balls from Driving Range
Description: A Florida man was arrested after he was caught on camera stealing 100 golf balls from a driving range.
Summary: The man, who has not been identified, was seen on surveillance video taking the golf balls from the range and putting them in his car. He was later arrested and charged with theft.
**STORY 2**
Title: Woman Gives Birth to Sextuplets in Texas
Description: A Texas woman gave birth to sextuplets on Monday, becoming the first woman in the state to do so.
Summary: The woman, who has not been identified, gave birth to the sextuplets at a hospital in Houston. The babies, who are all boys, are all healthy and doing well.
**STORY 3**
Title: New Study Finds Link Between Cell Phone Use and Cancer
Description: A new study has found a link between cell phone use and cancer.
Summary: The study, which was published in the journal JAMA Internal Medicine, found that people who use cell phones for more than 10 years have a slightly increased risk of developing brain cancer. The study's authors say that more research is needed to confirm the findings.
**STORY 4**
Title: President Signs Executive Order on Climate Change
Description: President Obama signed an executive order on climate change on Tuesday.
Summary: The executive order directs the Environmental Protection Agency to develop new regulations to reduce greenhouse gas emissions. The order also creates a new task force to coordinate the government's response to climate change.