LLM Hallucinations in RAG QA - Thomas Stadelmann, deepset.ai

[@pythontok4192]

What does the hallucination detector model compare each sentence of the answer against? If you run it against the contexts , say if k=4 snippets retrieved by the rag, wouldn’t some of them not be relevant?

[@vsrohit]

Can you please provide the links for the hallucination detection model?

[@davefar2964]

Thanks a lot for this presentation, the research papers on hallucinations as well as your BERTscore solutions were quite interesting. Another class of approaches (that causes high cost but no big latency if done in parallel) to detect and avoid hallucinations (see kzbin.info/www/bejne/oqS9dImjeKeFosUsi=-dYinw2SiAGw44df&t=1428) is getting multiple samples at test time, e.g. doing self-consistency or ensembling, deciding for the final answer by mayority voting or ranking.

[@billykotsos4642]

So even RAG can't be trusted 100% huh...