How to quality control the citations your AI knowledge base adds

The "phantom citation" is the most dangerous illicit artefact in an AI knowledge base.

You've found the perfect knowledge base solution, uploaded your documents ran a few test queries, everything looks brilliant.

Until you spot something strange - in the list of citations there is a document that you don't recognise. You quickly scan your list of uploaded documents, you can't see it. It might be a bug - so you check your local filesystem, no sign. It could exist in the cloud, so you scan Google Drive, no where to be seen.

You double check the answer, it sounded great at first but on closer inspection, it reveals a glaring error.

"Refunds on discontinued lines are handled at manager's discretion within 60 days of purchase (Refunds Policy - 2026)."

The problem is, your refund policy is 30 days. The cited document doesn't exist.

The Problem

When large language models (LLM) are trained they take in vast swathes of information and build a dataset that powers what is fundamentally a prediction algorithm who's only job is to predict, based on what came before, what's comes next. This works great for forming natural language but it also gives it a blind spot, when it's not properly directed it is capable of creating false information that we know as hallucinations.

Unlike creative tasks where quality of the output doesn't rest on the accuracy of the input, AI knowledge bases depend on the information being used by the LLM to be a hard guardrail. The worst outcome of an AI knowledge base is confidently citing non-existent documents.

Any providers of AI knowledge bases should be run through two thorough tests to ensure that you are suffering from the phantom citation. If either of them fail, it's time to assess your choice of software.

Test one: scope it hard, then try to make it fail

Most invented answers come from the same place - the knowledge base was asked something its sources don't cover, and it filled the gap rather than admitting the gap.

So don't build one knowledge base over everything you own. Build separate projects, each pointed at a narrow set of sources: one for HR policies, one for product documentation, one for the sales library. A knowledge base reading only the 2026 staff handbook and the pension scheme documents has fewer opportunities to be creative than one reading the entire Google Drive, including that folder of half-finished proposals from 2021.

Then query it. Write ten questions on paper before you touch the tool. Seven you know the answer to, three you know are not in the sources at all - "what's our paternity leave policy in Ireland?" when you've only uploaded the UK handbook. Run all ten.

The three unanswerable ones are the real test. You want "I don't have a source that covers this." If instead you get a fluent, reasonable-sounding Irish statutory entitlement, you have learned something important in ninety seconds, and you have learned it before your team did.

This week: build one narrowly project, write your ten questions, and keep the sheet. You'll reuse it on every vendor you trial.

“Empirical evaluation on 100 Ukrainian legal queries across five systems -- four commercial LLMs via AWS Bedrock (Claude Haiku 4.5, Mistral Pixtral Large, Amazon Nova Pro/Lite) and one RAG-augmented production system -- reveals CG ranging from 0.791 to 0.873, with 13-21% of citations hallucinated.”

arXiv (arXiv:2606.00898)

Citation Grounding: Detecting and Reducing LLM Citation Hallucinations via Legal Citation Graphs, 2026

Test two: click every citation, because a citation is not a check

Cited AI answers are the single best guardrail available, and they are also the easiest thing in this market to fake convincingly. A footnote proves a document was retrieved. It does not prove the sentence above it reflects what that document says.

This has been measured. A 2026 study evaluating citation grounding against a graph built from 100.8 million Ukrainian court decisions tested four commercial models and one production retrieval system, and found 13-21% of citations hallucinated [1]. The paper is worth reading for how it breaks the failure down: does the cited provision exist, is it contextually appropriate, and was it valid at the relevant date. Those are three separate ways a citation can be wrong, and only the first one is obvious to the eye.

Now the uncomfortable part, and it applies to us as much as anyone. Grounding answers in your own documents reduces invention. It does not remove it. Researchers working on sentence-level hallucination detection in 2026 stated it plainly - retrieval "reduces but does not eliminate hallucination" [2]. Their own detector, which is better than eyeballing, still only reaches a response-level AUC of about 0.73. Anyone selling you zero hallucinations is selling you something that has not been built yet.

What citations genuinely buy you is a shortened verification loop. Instead of asking "is this true?", your team asks "does page 3 say this?" — and that is a five-second question rather than a five-minute hunt.

This week: take the seven questions you knew the answers to, click through to the source on every one, and note which are supported word-for-word, which are a reasonable paraphrase, and which are unsupported. That third column is your hallucination rate, measured on your documents rather than a leaderboard.

The watch-outs, and what to stop an AI knowledge base hallucinating on next

Three honest ones.

Citations cannot save a bad source. If both the 2024 and 2026 returns policies are in the folder, a perfectly grounded answer can quote the dead one at you. Cleaning up which documents you import does more for accuracy than any vendor feature, and it is unpaid work you have to do.

Imports are point-in-time. When you connect Google Drive or Notion or upload a PDF, you are importing what exists then. Change the handbook, re-import the handbook. Keep good version control in place to ensure that your whole knowledge base stays current.

And your ten-question test is a smoke test, not an audit. Run it again a month in, with questions your team actually asked, which will be stranger and more specific than anything you invented on paper.

None of this is an enterprise programme any more. A scoped, cited, access-controlled AI knowledge base is now something you set up yourself, on a free trial, and put in front of six people on a Thursday. The interesting question is no longer whether you can afford to test one properly. It's whether you'd let it answer a customer-facing question yet - and what you'd need to see, on your own documents, to change your mind.

Sources

[1] arXiv, Citation Grounding: Detecting and Reducing LLM Citation Hallucinations via Legal Citation Graphs (arXiv:2606.00898), 2026 — https://arxiv.org/abs/2606.00898

[2] arXiv, Detecting Hallucinations in Retrieval-Augmented Generation through Grounding-Aware Sensitivity by Perturbation (GASP) (arXiv:2607.04223), 2026 — https://arxiv.org/abs/2607.04223