RAG
AI answers with citations: grounding an LLM in your sources
An answer with citations puts the passage that supports each claim right next to it. The model is allowed to say "I don't know", pulls the quotes first and answers second, and any claim without a quote is retracted. Hallucinations drop a lot but do not disappear, so a person checks the decisions that matter.
Updated
What is LLM grounding?
The answer rests on the text the model was given instead of its general knowledge. In practice you hand it the documents, tell it to use only the information in them, and ask it to quote the passage it relies on.
How do LLM citations work?
- The documents go into the request as PDFs, plain text or custom content blocks you split yourself.
- For PDFs and plain text, the text is chunked into sentences, so the model can cite one sentence or a whole paragraph.
- The answer comes back with the exact passage and its location: page numbers for PDFs, character positions for plain text.
- In a RAG pipeline, each retrieved chunk can go in as its own document.
How do you reduce hallucinations?
- Give the model explicit permission to say "I don't know" or that the documents lack the information.
- For long documents, ask for word-for-word quotes first and the analysis second.
- After the answer, make each claim find its supporting quote; a claim without one is removed.
- Run the same prompt several times and compare. Inconsistencies can point to a hallucination.
Frequently asked questions
Why should a model be allowed to say "I don't know"?
Anthropic's hallucination guide lists permission to admit uncertainty first and says it can sharply cut false information. An "I don't know" is easy to check; an invented answer is not.
Do citations eliminate hallucinations?
No. The same guide says the techniques reduce hallucinations but do not eliminate them entirely. Critical information still needs checking, especially when a high-stakes decision depends on it.
Can I trust that the cited passage exists in the document?
With the citations feature in the Claude API, citations are guaranteed to point to the documents you provided. That tells you where the text came from. Whether the passage really supports the claim is something you, or a separate check, still verify.
How does a multi-agent system handle citations?
In the multi-agent research system Anthropic describes, once the research loop ends, a separate citation agent goes through the documents and the report and attributes each claim to its source.
Do citations work on scanned documents?
On PDFs only when the text can be extracted. Citing images inside PDFs is not supported yet.
Sources
Further reading
Related articles
What is RAG in AI, and how does it work step by step?
RAG (Retrieval-Augmented Generation) means the model searches your documents first and writes the answer from what it found. The documents are split into chunks, each chunk becomes an embedding in an index, and for every question the chunks closest in meaning are added to the prompt next to the question.ReadHuman in the loop: when a person approves what an AI agent does
Human in the loop means a person approves an AI agent's high-impact actions before they happen. Everything else runs on its own with minimal tools and permissions, and authorization lives in the downstream systems, outside the model's judgment. That is how OWASP recommends keeping an agent with too much power in check.Read
Case studies
Related services
Want answers with sources on your documents?
We build the assistant on your company's documents, with the supporting passage next to every answer.
See the document chatbot