Grounded Generation
Grounded generation is a generation method in which a model produces answers based on retrieved or supplied evidence rather than relying solely on its own parametric memory, while citing the sources behind each claim. It corresponds to the 'generation' step of a RAG pipeline and aims to reduce hallucination and improve verifiability by tying each statement back to its supporting documents.
- Grounded generation is an output method that builds answers from retrieved or supplied evidence and links each claim in the answer back to its source through citations.
- It is the 'generation' step of RAG's two stages (retrieval then generation), and unlike the grounding step that fetches the evidence, it focuses on actually writing and citing the answer from that evidence.
- Commercial APIs such as Google Vertex AI and Vectara return supporting chunks, claim-to-evidence mappings, and a grounding score (0 to 1) alongside the answer, making the answer's provenance traceable.
- Its core benefits are reduced hallucination and verifiability, constraining the answer so that it is derived only from the provided material.
- However, a citation does not guarantee trustworthiness: research has documented an 'unfaithful citation' problem in which a model attaches sources after the fact to rationalize an answer it had already formed.
What Grounded Generation Is
Grounded generation is a generation method in which a large language model does not rely solely on knowledge memorized into its parameters at training time, but instead produces answers based on retrieved or user-supplied evidence and cites the sources behind them. In the retrieval-augmentation-generation flow of a RAG pipeline, it corresponds to the final 'generation' step, and its essence is anchoring the model's output to verifiable information sources.
It is worth clarifying its relationship to grounding. Grounding is the broader concept, referring to the overall ability to connect an answer to factual sources, whereas grounded generation focuses specifically on the part that uses the retrieved evidence to write the actual answer and attaches sources at the sentence level on output. In other words, when distinguishing 'fetching the evidence (retrieval)' from 'building and citing an answer based on that evidence (generation),' grounded generation is the latter.
Google Cloud's documentation describes grounded generation (also called grounded answers, or RAG) as two stages, retrieval and generation, and defines it as a methodology that pins the model's output to specific data sources to reduce the chance of fabricating content that is not true.
Standard Generation vs. Grounded Generation
| Dimension | Standard Generation | Grounded Generation |
|---|---|---|
| Knowledge source | Parametric memory learned during training | Retrieved or supplied evidence (documents, web, databases) |
| Source citation | None (or attached arbitrarily after the fact) | Each claim linked to a supporting evidence chunk |
| Verifiability | Low (provenance hard to trace) | High (traceable via evidence links) |
| Hallucination risk | Relatively high | Relatively low, constrained by the provided material |
| Freshness | Fixed at training time | Can reflect up-to-date material at retrieval time |
| Position within RAG | Not applicable | The 'generation' step following retrieval |
How Commercial APIs Work
Google Vertex AI's grounded generation API retrieves relevant information from evidence sources such as Google Search, inline text, and Agent Search data stores, and then generates an answer from that content. The response includes the following so that provenance can be traced:
- Support chunks: snippets quoted verbatim from the source, along with metadata such as title, URI, document ID, and page.
- Grounding support: a mapping that links each claim in the answer (claimText) to the indices of the chunks that support it (supportChunkIndices).
- Grounding score: a value between 0 and 1 indicating how strongly the answer is grounded in the provided evidence.
Vectara offers this as a RAG service, explaining that it reduces hallucination by ensuring that 'generated content is verifiable and pinned to the supplied data.' Answers include, by default, citations that attribute facts derived from the data, and Vectara also operates its own evaluation model, HHEM (Hughes Hallucination Evaluation Model), to measure the factual faithfulness of answers.
Evidence and Cases
The benefits of grounded generation are clear, but the mere presence of a citation does not guarantee reliability. The paper by Wallat et al. (2024), Correctness is not Faithfulness in RAG Attributions (arXiv:2412.18004), distinguishes between the correctness and faithfulness of a citation. Correctness asks 'does the cited document actually support the claim,' while faithfulness asks 'did the model genuinely consult that document to produce the answer.' The study reports that in current citation-bearing answers up to 57% of citations may be unfaithful, owing to 'post-rationalization,' in which the model attaches a fitting source after the fact to an answer it already held. This means a citation can be formally correct yet still not actually be where the answer was derived from.
Aiming to mitigate this problem, Xia et al. (2024), Ground Every Sentence (arXiv:2407.01796), propose the ReClaim (Refer & Claim) approach. Instead of writing the entire answer and then attaching sources, it interleaves references and claims to produce sentence-level citations, reporting citation-matching accuracy of about 90%. This shows that the quality of grounded generation depends not on 'whether citations exist' but on 'how faithfully claims and evidence are linked at the sentence level.'
References
- Google Cloud — Generate grounded answers with RAG (Vertex AI Search)
- Google Cloud — Grounding overview (Generative AI on Vertex AI)
- Vectara Docs — Retrieval Augmented Generation (Grounded Generation) Overview
- Vectara — New Release: Grounded Generation
- Wallat et al. (2024), Correctness is not Faithfulness in RAG Attributions, arXiv:2412.18004
- Xia et al. (2024), Ground Every Sentence: Improving Retrieval-Augmented LLMs with Interleaved Reference-Claim Generation, arXiv:2407.01796