Three months ago, Air Canada found out the hard way that RAG hallucinations aren’t just an engineering problem: they’re a legal liability. Their customer service chatbot confidently told a grieving passenger about a bereavement fare discount that didn’t exist. The airline argued they weren’t responsible for what their AI said. The court disagreed. Air Canada paid. Air Canada paid This isn’t an isolated incident. NYC’s MyCity chatbot is currently feeding business owners fabricated information about local laws. ChatGPT helped lawyers draft a legal brief filled with completely invented case citations, resulting in sanctions. fabricated information about local laws Here’s the uncomfortable truth: while 55% of organizations are experimenting with generative AI, only 10% have shipped it to production. The gap isn’t technical capability. It’s trust. while 55% of organizations are experimenting with generative AI, only 10% have shipped it to production. And if you’re relying on popular tools like RAGAS, DeepEval, or basic LLM self-evaluation to catch hallucinations before they reach users, I have bad news. Most RAG Hallucination Detectors Barely Beat Coin Flips Cleanlab just published comprehensive benchmarks testing every major hallucination detection method across six different RAG applications: financial Q&A, medical diagnosis, complex reasoning tasks, and more. The results are worse than most teams realize. On FinanceBench (a dataset of financial document questions requiring numerical reasoning): On FinanceBench RAGAS Faithfulness failed to produce scores for 83.5% of examplesG-Eval performed barely better than random guessingDeepEval’s hallucination metric showed inconsistent effectivenessOnly TLM (the Trustworthy Language Model) consistently caught errors with high precision RAGAS Faithfulness failed to produce scores for 83.5% of examples G-Eval performed barely better than random guessing DeepEval’s hallucination metric showed inconsistent effectiveness Only TLM (the Trustworthy Language Model) consistently caught errors with high precision On DROP (questions requiring multi-step reasoning over Wikipedia articles): On DROP RAGAS failed on 58.9% of examplesMost specialized detection models couldn’t reliably identify incorrect responsesThe performance gap between tools widened as complexity increased RAGAS failed on 58.9% of examples Most specialized detection models couldn’t reliably identify incorrect responses The performance gap between tools widened as complexity increased Let that sink in: the most popular open-source RAG evaluation framework couldn’t even generate scores for most examples in real-world financial data. Let that sink in: the most popular open-source RAG evaluation framework couldn’t even generate scores for most examples in real-world financial data. the most popular open-source RAG evaluation framework couldn’t even generate scores for most examples in real-world financial data. If you’re shipping RAG in production with these tools as your safety net, you’re flying blind. The Problem: Everyone’s Measuring the Wrong Type of Uncertainty Traditional approaches only measure aleatoric uncertainty: the known unknowns. This is uncertainty the model is aware of because the prompt is vague, incomplete, or genuinely ambiguous. aleatoric uncertainty Methods like LLM self-evaluation (“rate your confidence 1–5”) and token probability scores (logprobs) fall into this category. They work when the model knows it doesn’t know something. Production hallucinations don’t happen because models admit uncertainty. They happen because the model confidently generates plausible-sounding b*llshit based on incomplete or misinterpreted context. They happen because the model confidently generates plausible-sounding b*llshit based on incomplete or misinterpreted context. This is epistemic uncertainty: the unknown unknowns. The model doesn’t know what it doesn’t know. epistemic uncertainty Think about Air Canada’s chatbot: it didn’t respond with “I’m uncertain about bereavement fares.” It confidently hallucinated specific policy details because it failed to properly ground its response in the retrieved documentation. Think about Air Canada’s chatbot: Most detection tools can’t catch this because they’re only looking at surface signals. Does the response match the context semantically? Does the model seem confident? These checks miss the deeper issue: reasoning errors, fact fabrication, and overconfident extrapolation beyond retrieved evidence. What Actually Catches Production Errors Cleanlab tested seven approaches across six datasets spanning different RAG challenges: Critical finding: TLM achieved the highest AUROC (area under ROC curve) across all six benchmarks. Critical finding: TLM achieved the highest AUROC (area under ROC curve) across all six benchmarks. What does this mean practically? On the SVAMP dataset (elementary math word problems): On the SVAMP dataset Using TLM scores, you can achieve near-zero error rates by manually reviewing only ~20% of responsesUsing token probabilities, you’d need to review ~40% of responses for the same error rateUsing self-evaluation, you’d need to review ~90% of responses Using TLM scores, you can achieve near-zero error rates by manually reviewing only ~20% of responses Using token probabilities, you’d need to review ~40% of responses for the same error rate Using self-evaluation, you’d need to review ~90% of responses That’s a 4.5x reduction in human review costs compared to self-evaluation, while maintaining the same quality bar. That’s a 4.5x reduction in human review costs compared to self-evaluation, while maintaining the same quality bar. Why TLM Works When Others Fail TLM doesn’t rely on a single signal. It combines three complementary uncertainty measures: 1. Self-Reflection 1. Self-Reflection Like LLM-as-a-judge, but more sophisticated: the system prompts the model to evaluate its own response quality, check for inconsistencies, and identify potential errors. 2. Multi-Response Consistency 2. Multi-Response Consistency The system generates multiple candidate responses to the same query and compares them for semantic consistency. High variability = low confidence. If your LLM gives five different answers to the same question, something’s wrong. 3. Probabilistic Measures 3. Probabilistic Measures Token-level probabilities provide a baseline confidence signal, though this alone isn’t sufficient (as the benchmarks show). The combination catches what individual methods miss. A response might have high token probabilities (the model is confident) but low consistency across samples (that confidence is unwarranted). Or it might pass self-reflection but show semantic drift from the retrieved context. This is why TLM consistently outperforms both specialized trained models and simpler heuristics across diverse applications. This is why TLM consistently outperforms both specialized trained models and simpler heuristics across diverse applications. In practice, this looks like: Air Canada’s bereavement fare hallucination would have scored: ~0.23 trustworthiness,High token probabilities (confident language)Low consistency (multiple attempts generate different policies)Failed self-reflection (claims contradict retrieved context).Set your production threshold at 0.5, and that response never reaches the customer. No lawsuit, no headline. ~0.23 trustworthiness, High token probabilities (confident language) Low consistency (multiple attempts generate different policies) Failed self-reflection (claims contradict retrieved context). Set your production threshold at 0.5, and that response never reaches the customer. No lawsuit, no headline. The Architecture Matters: Why Custom-Trained Models Will Age Poorly HHEM, Prometheus, and Lynx were trained on GPT-4’s errors. They won’t catch GPT-5's. Fine-tuned hallucination detectors learn to recognize patterns in how specific LLMs fail. HHEM knows what GPT-3.5 hallucinations look like. Prometheus was trained on Mistral’s failure modes. Lynx learned from Llama 3 errors. But LLMs are evolving rapidly. GPT-6, Claude 5, Llama 5–future models will make different types of errors. The detector you deploy today becomes less effective with every model upgrade. different types of errors TLM, by contrast, wraps any LLM and measures uncertainty through model-agnostic techniques. It works with GPT-4, Claude, Llama, your custom fine-tune, and whatever gets released next month. any You’re not locked into a specific model ecosystem. You don’t need to retrain evaluation models when you upgrade your generation LLM. You don’t need infrastructure to serve custom evaluation models. The benchmarks prove this flexibility doesn’t sacrifice performance: TLM outperformed even domain-specific trained models on their home turf. The benchmarks prove this flexibility doesn’t sacrifice performance: TLM outperformed even domain-specific trained models on their home turf. The Bottom Line RAG isn’t staying in pilot programs because teams lack technical skill. It’s stuck because hallucinations are a solved problem in theory but a crisis in practice. hallucinations are a solved problem in theory but a crisis in practice. Most teams are using detection tools that barely outperform random chance on real-world data. They’re measuring the wrong type of uncertainty. They’re optimizing for development convenience rather than production reliability. Meanwhile, Air Canada is writing checks to customers because their chatbot lied. The benchmarks are public. The tools are available. The question isn’t whether your RAG will hallucinate, it’s whether you’ll catch it before your users do. Air Canada learned this lesson in court. Your users won’t sue you for poor AUROC scores. They’ll sue you for wrong answers. Choose accordingly. Sources: Full benchmarking methodology and codeTutorial on RAG hallucination detection with Cleanlab Full benchmarking methodology and code Full benchmarking methodology and code Tutorial on RAG hallucination detection with Cleanlab Tutorial on RAG hallucination detection with Cleanlab

This story contains new, firsthand information uncovered by the writer.

Hot off the press! This story contains factual information about a recent event.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

This story will praise and/or roast a product, company, service, game, or anything else people like to review on the Internet.

Air Canada Lost a Lawsuit Because Their RAG Hallucinated. Yours Might Be Next

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

5 AI Agent Workflows for Repeatable Success (Code Included)

Build a Monster-Finding Tool For Your Next D&D Session That Picks the Right Encounter For You

The Fork Reshaping MCP Testing: How a 24-Year-Old CTO Is Taking On One of AI’s Biggest Players

How Generative Data Expands AI’s Understanding of the Real World

A Secret Technique To Sidestepping LLM Hallucinations

How to Fine-tune and Optimize GPT Assistants with OpenAI

5 AI Agent Workflows for Repeatable Success (Code Included)

Build a Monster-Finding Tool For Your Next D&D Session That Picks the Right Encounter For You

The Fork Reshaping MCP Testing: How a 24-Year-Old CTO Is Taking On One of AI’s Biggest Players

How Generative Data Expands AI’s Understanding of the Real World

A Secret Technique To Sidestepping LLM Hallucinations

How to Fine-tune and Optimize GPT Assistants with OpenAI

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps