How can you test whether an AI citation supports its claim?
Compare the generated claim with the exact cited passage, then score entailment, scope, freshness, and source quality instead of counting the link as automatically correct.
Written for researchers, editors, and SEO teams auditing answer-system citations for accuracy and usefulness.
Key facts
- Citation presence does not prove the source supports the generated statement.
- Entailment asks whether the cited evidence justifies the claim without adding unsupported meaning.
- Freshness and source authority matter separately from textual support.
A useful rule: make each important claim understandable and verifiable without requiring the reader to reconstruct your meaning from the rest of the page.
What evidence should be captured?
Save the product, model or mode when visible, full prompt, relevant conversation context, generated claim, cited URL, citation position, date, and a screenshot or export allowed by the product. Open the cited page and capture the exact supporting passage with its heading and source date. Outputs can change, so a URL and memory are not enough for reproducible review. Remove personal or sensitive query data before sharing the record. The unit of analysis is one claim-to-citation pair, not the entire answer scored as a single impression.
- Store one row per claim and citation pair.
- Preserve query context and observation time.
- Handle private prompts according to data policy.
How is entailment scored?
Use a simple rubric: fully supported, partially supported, unsupported, or contradictory. Fully supported means the source justifies the same subject, relationship, conditions, and degree of certainty. Partial support may establish the general topic but omit a number, date, comparison, or exception added by the answer. Unsupported means the passage does not justify the claim, while contradictory evidence points the other way. Have a second reviewer score a sample and discuss disagreements so the categories remain consistent.
- Compare meaning, not shared keywords.
- Check numbers, dates, modality, and qualifiers.
- Calibrate reviewers on the same sample.
How should scope and freshness be checked?
Confirm that the source applies to the named country, audience, product tier, software version, and time period. A perfectly quoted policy from another jurisdiction still fails the user's claim. Record source publication and update dates separately from your access date. Look for superseding documentation, redirects, archived notices, and current official pages. When the answer uses present tense, the source must support current applicability or the wording should be changed to historical scope. Freshness is a separate dimension from entailment because an old source can accurately support a historical statement.
- Match jurisdiction, version, tier, and effective date.
- Search for superseding primary documentation.
- Score current and historical claims differently.
How is source quality evaluated?
Identify whether the source is the primary authority for its claim, an original study, a transparent secondary analysis, or an unattributed repetition. Check author, publisher, method, conflicts, and access to underlying evidence. A low-quality source can entail a statement yet remain a poor basis for a high-stakes answer. Conversely, an authoritative source can be cited incorrectly. Score authority and entailment separately so one does not hide the other. Prefer official current documentation for product behaviour and original data for measured claims.
- Separate source authority from claim support.
- Trace statistics to the original dataset or study.
- Apply stricter thresholds to high-stakes decisions.
What metrics should the audit report?
Report citation precision, the share of cited claims that are fully supported, partial-support rate, contradiction rate, source-authority distribution, freshness failures, and coverage of claims that needed citations. Include sample size, products, query set, dates, and reviewer agreement. Do not generalize from a handful of convenient examples to every answer. For your own site, connect failures back to source passages: unclear scope, stale dates, weak primary evidence, or inaccessible text. Fix those page-level causes and repeat the same test set after recrawl.
- Publish denominators and sampling method.
- Separate accuracy, authority, freshness, and coverage.
- Use repeated fixed queries for directional comparison.
Put it to work
Find the highest-impact fix on your site.
Score claim support, scope, freshness, and source authority with a reproducible evidence log.
Run a citation auditSources
- 1.Liu et al.: Evaluating verifiability in generative search enginesChecked 2026-07-26
- 2.Aggarwal et al.: GEO: Generative Engine OptimizationChecked 2026-07-26