Hallucination Detection in Retrieval-Augmented Generation Systems via Self-Consistency Scoring—A Reference-Free, Retrieval-Aware Detection Framework

Abstract

Large language models (LLMs) augmented with retrieval mechanisms have become a dominant paradigm for grounding generative output in external evidence, yet Retrieval-Augmented Generation (RAG) systems continue to produce hallucinated content even when relevant context is retrieved successfully. Existing hallucination detection methods either require access to token-level output probabilities unavailable in many deployed black-box systems, evaluate a single deterministic generation against retrieved passages, or were designed for open-ended generation without an explicit retrieval anchor. This paper proposes RAG-SC, a reference-free, claim-level hallucination detection framework tailored to the RAG setting. RAG-SC combines two complementary signals computed purely from black-box sampling access to the generator: a context-faithfulness score that checks whether each atomic claim is entailed by the retrieved passages, and a cross-sample consistency score that measures the semantic agreement of a claim across multiple stochastically resampled generations conditioned on the same retrieved context. We position RAG-SC against three established paradigms—sampling-based detection, atomic factuality scoring, and RAG-specific evaluation—and argue that none jointly exploits retrieval grounding and generation-time stochasticity. We present the method’s design, a concrete evaluation protocol across three open-domain question-answering benchmarks, and a discussion of expected behavior, limitations, and threats to validity, providing a template for empirical validation in future work.