LLMs can generate both creative, coherent text and factually incorrect "hallucinations," posing a significant challenge for real-world applications.
The combination of the scalable Transformer architecture and massive text datasets enables models like GPT to develop "parametric knowledge" as they grow in size.
The quality of web-scraped datasets like Common Crawl, even after filtering, directly contributes to model hallucinations by embedding misinformation.
Hallucinations are categorized as either faithfulness errors, which contradict a given source text, or factuality errors, which stem from incorrect learned knowledge.
The TruthfulQA dataset provides a benchmark for measuring an LLM's tendency to repeat common misconceptions and conspiracy theories across various categories.
A step-by-step demonstration shows how to use Python, LangChain, and Hugging Face Datasets to run the TruthfulQA benchmark on a model like GPT-3.5 Turbo.
Key techniques to mitigate hallucinations include careful prompt crafting, domain-specific fine-tuning, output evaluation, and retrieval-augmented generation (RAG).
RAG reduces hallucinations by augmenting prompts with relevant, up-to-date information retrieved from a vector database of document embeddings.
Naive RAG can fail due to poor retrieval or generation, but advanced methods like Rowan selectively apply retrieval to significantly improve factuality.
58:00Vijay Krishan Gupta & Gauravdeep Singh Lotey
47:28Tanmay Bakshi
23:50Aditi Godbole
42:26Meta Atamel & Guillaume Laforge
29:11Michael Hunger
24:42Martin O'Hanlon
56:46Jodie Burchell
27:11Emanuele Fabbiani
.png?w=240&auto=compress,format)
.png?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
Universidad De Valladolid
Municipality of Valladolid, Spain