What is Query Rewriting?

Query rewriting transforms a user's original query into a form that search systems can understand and match more effectively, acting as an umbrella concept for the preprocessing that happens just before matching and scoring.
Common techniques include synonym and acronym expansion, typo correction, linguistic normalization such as stemming, removal of unnecessary phrasing, named-entity recognition, and intent clarification.
Query expansion is one branch of rewriting that adds related terms to boost recall, while rewriting is the broader concept that also covers dropping terms or linking entities to a knowledge graph.
RAG pipelines in which an LLM rewrites the query before retrieval have become an active research area, with the Rewrite-Retrieve-Read framework by Ma et al. (arXiv:2305.14283) as a leading example.
Query decomposition and query fan-out split a single compound query into multiple sub-queries, whereas query rewriting keeps the query as one and transforms it into a better form — that is the core difference.

What Is Query Rewriting?

Query rewriting is the practice of transforming a user's original query into a form better suited to search, before the actual matching and scoring take place. OpenSource Connections defines it as "an umbrella term covering the many things that happen before actual matching and scoring occurs in a search engine." In other words, it is not a single technique but the whole preprocessing stage that bridges the gap between the input text and the representation of indexed documents.

This gap matters because the words people type into a search box frequently diverge from the language documents actually contain. Users mix in acronyms, colloquialisms, and typos, and they compress their search intent into terse phrasing. Rewriting adds synonyms, fixes typos, strips out clutter, and identifies key entities to refine the query into a form that is easier for a search engine to understand.

Common Rewriting Techniques

The following techniques are used across both traditional search engines and LLM-based pipelines.

Linguistic normalization: absorbs inflected forms and spelling variants through stemming, ASCII folding, and similar methods.
Typo and spacing correction: fixes mistyped words (for example, "strnger things" → "Stranger Things").
Synonym and acronym expansion: adds equivalent alternatives, acronym expansions, and domain-related terms.
Noise removal: trims phrasing that is unnecessary for search and keeps only the key entities.
Named-entity recognition and linking: identifies proper nouns in the query, links them to entities in a knowledge graph, and flags them so downstream modules can handle them specially.

Elastic's experiments (Elasticsearch Labs) explore LLM-driven rewriting through core keyword enrichment, hypothetical answer generation (producing a plausible answer or document title for the query in advance), and entity enrichment. In particular, they explain that a "template-based" approach — pairing prompts with a predefined query DSL template rather than allowing free-form rephrasing — makes outputs more deterministic and prevents drift away from the original intent.

Query Rewriting vs. Query Expansion vs. Query Decomposition

It is important to clearly distinguish these from adjacent concepts. The three are often used interchangeably, but their scopes differ.

Concept	Core action	Number of queries	Primary goal
Query Rewriting	Transforms the original query into a better form (synonyms, typos, normalization, term removal, entity linking)	Stays at 1	Improve matching relevance and recall
Query Expansion	Adds related terms to broaden the query	1 (expanded)	Improve recall (a sub-branch of rewriting)
Query Decomposition / Fan-out	Splits a compound query into multiple sub-queries, searches them in parallel, then merges the results	Split into several	Satisfy a compound intent part by part

The key point is that query expansion is one branch of query rewriting. OpenSource Connections explains that expansion is a specific category contained within rewriting, citing cases like synonyms where you cannot simply substitute a term but must keep the alternatives alive alongside it. That said, as 'Pretrained Transformers for Text Ranking' (arXiv:2010.06467) notes, rewriting is more general than expansion — because rewriting can also remove terms a user added unnecessarily, or link named entities to a knowledge graph.

By contrast, query decomposition and query fan-out are a different breed. According to Microsoft Learn's documentation on agentic retrieval in Azure AI Search, rewriting keeps the tools as they are and rewrites the query in several ways against the same tool, whereas decomposition breaks a compound query into multiple sub-questions, runs each independently, and accumulates the top results as combined context. In short, rewriting is about making one query better, while decomposition and fan-out turn one query into many.

Evidence and Examples

In the LLM era, query rewriting has emerged as a core step in RAG (retrieval-augmented generation). Because of the noise and intent bias contained in the original query, direct retrieval alone often fails to fetch sufficiently relevant documents.

Rewrite-Retrieve-Read: Ma et al. (2023, arXiv:2305.14283, EMNLP 2023) proposed the Rewrite-Retrieve-Read framework, in which an LLM first rewrites the query before retrieval, rather than the conventional retrieve-then-read order. They went further by training a small rewriting model with reinforcement learning, using retrieval quality as the reward signal, so that it generates queries better suited to a fixed retriever and LLM reader.
Diverse multi-query rewriting (DMQR-RAG): an approach that generates several diverse variants of the original query to lift document recall and final response quality, compensating for the limitations of a single rewrite.
Elastic's measurements: Elasticsearch Labs reports that using only the terms produced by QR on their own can be marginal or even worse, and that the best results came from placing the original query in the must clause and combining the rewritten terms as score boosters in the should clause. They also note that for keyword extraction, a small model (Claude 3.5 Haiku) was on par with larger models while substantially lowering input and output costs.

Implementation Checklist

Do not discard the original query outright; combine it with the rewritten terms to keep the original intent at the center of retrieval.
Treat synonyms as alternatives (OR conditions) rather than substitutions, so that meaning is preserved.
Constrain rewriting with templates and prompts rather than free-form rephrasing to prevent drift away from the original intent.
Apply typo correction and linguistic normalization (stemming) as baseline preprocessing, and identify key entities to weight them.
For compound intent (when several questions are mixed into one sentence), decide whether to branch into query decomposition instead of rewriting.
In RAG, treat retrieval quality (recall and ranking) as the metric, validate the effect of rewriting via A/B testing, and then determine its scope of application.