CeQe: Grounding Lexical Retrieval in Semantic Evidence

Adam Kahirov, Umesh Deshpande, Swaminathan Sundararaman

IBM Research

Abstract

Lexical retrieval (BM25) captures exact keyword matches and weights terms by corpus-wide significance, but it is blind to the semantic vocabulary gap: when a relevant document phrases an answer differently from the query, BM25 never retrieves it, and no amount of downstream reranking or fusion can recover a document that was never in the candidate set. We present Cross-Encoder Query Expansion (CE-QE), which reads the per-token relevance attributions of a cross-encoder applied to top semantic search results, selects the terms the cross-encoder treats as decisive, and appends them to the BM25 query. Unlike classical pseudo-relevance feedback, which reuses BM25's own (possibly wrong) top results, CE-QE seeds expansion from the semantic retriever's results, avoiding self-reinforcing query drift. Unlike recent generative query expansion (HyDE, Query2doc), which prompts a large language model to hallucinate text from its parametric knowledge, every CE-QE expansion term is copied verbatim from a retrieved passage, so it cannot introduce vocabulary the corpus does not contain, and its only added cost is attribution extraction on a cross-encoder a hybrid pipeline already runs for reranking. On seven BEIR datasets, CE-QE improves lexical recall substantially where query and answer vocabulary diverge (e.g., NQ Recall@100 from 0.32 to 0.47), and its score-fusion variant (SESF) beats cross-encoder score fusion by 2.5% on Recall@100 and beats SPLADEv2 and ColBERTv2 by 5.3% and 4.6% on nDCG@10, while leaving the underlying BM25 index completely unmodified.

Keywords: query expansion, cross-encoder reranking, hybrid search, BM25, dense retrieval, BEIR, retrieval-augmented generation.

Executive Summary: Lexical retrieval with BM25 is fast and reliable for exact term matches, but it fails when relevant documents use different wording from the query. In hybrid systems that combine BM25 with semantic search, reranking and fusion cannot recover missed documents because they only operate on items already retrieved. This vocabulary-gap problem limits overall recall in retrieval-augmented generation and similar applications.

The authors developed Cross-Encoder Query Expansion (CE-QE) to address the gap. The method runs semantic search on the original query, applies a cross-encoder to the top results, extracts the tokens the model treats as most relevant, and appends those tokens to the BM25 query. A fusion variant called Semantically Enriched Score Fusion (SESF) then combines the enriched lexical results with semantic scores. Both approaches were tested on seven BEIR datasets through a consistent pipeline, using standard flat BM25 indexes and two common cross-encoder rerankers.

CE-QE raised BM25 Recall@100 substantially on datasets with large wording differences, such as Natural Questions (0.32 to 0.47) and TREC-COVID (0.56 to 0.67). SESF improved Recall@100 by 2.5 percent over ordinary cross-encoder fusion and outperformed SPLADEv2 and ColBERTv2 by 5.3 percent and 4.6 percent on nDCG@10. Gains were largest where queries and answers used different registers and were small or slightly negative where wording already overlapped. End-to-end latency rose to roughly 557 ms per query from 154 ms for basic fusion, with the added cost coming from a bounded cross-encoder pass.

These results show that injecting semantic signal directly into the lexical query can raise the recall ceiling without changing the underlying BM25 index. The approach therefore preserves the option to use any lexical index, including future large-scale versions. It also avoids the risk of introducing terms absent from the corpus, unlike methods that generate expansions with large language models.

Organizations should apply CE-QE or SESF selectively on workloads where vocabulary mismatch is common, and they should consider a simple gate to skip expansion when lexical and semantic results already agree. The next practical step is to measure the same gains on billion-chunk BM25 indexes. Attribution quality from the chosen cross-encoder remains the main variable that could affect term selection, so testing additional rerankers would increase confidence before broad deployment.

1. Introduction

Section Summary: Hybrid retrieval combines lexical methods such as BM25 with dense semantic search because each compensates for the other's weaknesses in finding relevant documents. The core limitation is that BM25 only surfaces passages sharing exact surface terms with the query, so documents using different phrasing are never retrieved and cannot be rescued by later fusion or reranking. The paper therefore proposes a query-expansion technique, CE-QE, that extracts decisive terms from a cross-encoder run on semantic results and appends them to the BM25 query, grounding the added terms in the corpus without extra model calls.

Hybrid retrieval that fuses lexical (BM25) and dense semantic search is now the default for retrieval-augmented workloads, because each retriever recovers what the other misses: lexical matching captures exact terms and corpus-wide term significance, while dense retrieval captures synonyms and paraphrase [1]. The value of the lexical stage is concrete—it captures exact matches (a dense model can rank "Apple" the company and "apple" the fruit as near-identical) and weights rare, discriminative terms by their corpus-wide significance—but it has a structural weakness that fusion alone does not fix.

The problem. BM25 retrieves only documents that share surface terms with the query. When a relevant document phrases the answer differently ("side effects" vs. "adverse reactions", "ibuprofen" vs. "NSAID"), lexical search misses it entirely, and simply fusing its result list with a semantic list does not repair the lexical ranking itself: fusion and reranking can only reorder or combine documents that were retrieved in the first place. If a relevant document shares no surface terms with the query, BM25 never puts it in the candidate set, and no amount of downstream re-scoring can recover it—the recall ceiling is fixed at the first stage.

This paper contributes a query-side fix for this problem, together with an account of why it works and how it compares to the two nearest alternatives.

• Cross-Encoder Query Expansion (CE-QE) (Section 3): a query-expansion method that reads the per-token relevance attributions of a cross-encoder applied to top semantic results, selects the terms the cross-encoder treats as decisive, and appends them to the BM25 query. This injects semantic signal into the lexical stage directly, without modifying the index. Its fusion variant, Semantically Enriched Score Fusion (SESF), combines the enriched lexical results with semantic scores and delivers the best overall quality.
• A comparison to the two nearest alternatives (Section 2.1): classical pseudo-relevance feedback (RM3), which is self-referential in a way that reinforces exactly the failure CE-QE targets, and recent generative query expansion (HyDE, Query2doc), which expands a query with text an LLM hallucinates from its parametric knowledge rather than from the corpus itself, at the cost of a separate generation call per query. CE-QE expands the query only with terms already present in passages the corpus's own semantic retriever returned, using a cross-encoder a hybrid pipeline already runs for reranking, which grounds every expansion term in the corpus and adds no separate model call.

We evaluate CE-QE and SESF on BEIR through RUMIR, a reproducible pipeline built for this work (Section 4), against standard flat BM25 as the lexical stage; a companion paper addresses lexical retrieval's separate scaling problem at billion-chunk scale, and Section 5 discusses how the two compose.

2. Background and Related Work

Section Summary: This section reviews core techniques for document retrieval, including traditional term-matching systems like BM25, embedding-based similarity search with approximate indexes, methods for merging results from multiple approaches, and more expensive reranking models that jointly score query-document pairs. It then covers query expansion strategies, contrasting classical pseudo-relevance feedback that draws terms from initial results, generative LLM methods that create hypothetical documents, and the CE-QE approach that extracts expansion terms from a cross-encoder's attributions on semantically retrieved passages. The section closes by positioning CE-QE's advantages in grounding and cost over generative methods and its ability to avoid error reinforcement compared with classical feedback, while noting the use of the BEIR benchmark to measure both coverage and ranking quality.

Lexical retrieval. BM25 scores a document $D$ for query $Q$ as a sum over query terms of an IDF weight times a saturated, length-normalized term-frequency factor [2]. Term saturation (parameter $k_1$) stops a single repeated term from dominating; length normalization (parameter $b$) discounts long documents.

Dense retrieval and ANN. Bi-encoders encode text once and search by similarity [3, 4], using approximate nearest-neighbor indexes such as HNSW [5] or IVF with product quantization [6] to remain tractable at scale.

Fusion. Two ranked lists are merged by combining ranks or scores. Reciprocal Rank Fusion (RRF) sums $1/(k+\mathrm{rank}_i)$ across lists, needs no tuning, and ignores raw scores [7]; it is used by Elasticsearch and LanceDB. Weighted fusion combines normalized scores as $\alpha, L(c)+(1-\alpha), S(c)$ (OpenSearch, Elasticsearch) but is sensitive to score normalization because BM25 and cosine live on different scales.

Reranking. Cross-encoders jointly encode the query and each candidate to produce a precise relevance score; they raise quality but cost far more than first-stage retrieval, motivating two-stage pipelines that rerank only a small candidate set [8].

Query expansion. Classical pseudo-relevance feedback (e.g., relevance models / RM3 [9]) expands a query with terms that co-occur in top-ranked documents, using term statistics. CE-QE differs in the selection signal: it picks expansion terms from a cross-encoder's per-token relevance attributions rather than from co-occurrence counts, so the added terms are those a supervised relevance model treats as decisive in the query–passage interaction. Learned sparse models such as SPLADEv2 [10] and late-interaction models such as ColBERTv2 [11] attack the same vocabulary gap by changing the index; CE-QE leaves the BM25 index unchanged and expands the query instead. A more recent line of work expands queries generatively: HyDE prompts an LLM to hallucinate a hypothetical answer document and embeds it for dense retrieval [12], and Query2doc few-shot-prompts an LLM to write a pseudo-document that is appended to the query for both sparse and dense retrieval [13]. Section 2.1 contrasts CE-QE with this generative family directly.

Evaluation. BEIR is the standard heterogeneous, zero-shot IR benchmark [1]. We report Recall@100 (coverage) and nDCG@10 (ranking quality); the two diverge—two rankings with identical Recall@5 can have very different nDCG@5 (e.g., 1.0 vs. 0.62) depending on where the relevant items land—so we track both throughout. Retrieval quality of this kind underlies retrieval-augmented generation broadly [14], which is the deployment setting motivating this work.

2.1 Positioning relative to recent work

CE-QE vs. generative (LLM) query expansion. HyDE [12] and Query2doc [13] both expand a query using text an LLM generates from its own parametric knowledge: a hypothetical answer document or pseudo-document that may contain no terms actually present in the target corpus, and that costs a full LLM decoding pass per query (typically on the order of a paragraph of generated tokens). CE-QE expands the query with terms extracted from passages the corpus's own semantic retriever actually returned, using attribution weights from a cross-encoder that a hybrid pipeline already runs for reranking (Section 3.3). This has two concrete consequences. First, grounding: every CE-QE expansion term is copied verbatim from a retrieved passage, so it cannot introduce vocabulary the corpus does not contain, whereas a generated hypothetical document can drift into fluent but corpus-absent phrasing—a documented failure mode of generative expansion on unfamiliar or ambiguous queries. Second, cost: CE-QE adds attribution extraction on a component already in the serving path (Table 2 shows the full SESF pipeline, including this step, at $\sim$ 557 ms), whereas generative expansion adds a separate LLM call whose latency and expense scale with generated length and are incurred independently of any reranking the pipeline already performs. The two approaches are not mutually exclusive—an LLM-generated expansion could be fed through the same attribution-based filtering CE-QE uses to select terms from it before appending them to the BM25 query—but CE-QE's grounding and marginal-cost advantages come specifically from sourcing expansion terms from retrieval rather than generation.

CE-QE vs. classical pseudo-relevance feedback. RM3 and relatives [9] expand a query with terms drawn from the top documents the same lexical retriever already returned—self-referential in a way that hurts exactly the queries CE-QE targets, since if BM25's top results are already wrong because of a vocabulary gap, feeding terms back from those same wrong results reinforces the error rather than correcting it (query drift). CE-QE instead seeds expansion from the semantic retriever's top passages, which remain a trustworthy seed set precisely in the cases where BM25's own results are not (Section 3).

3. Cross-Encoder Query Expansion

Section Summary: Cross-Encoder Query Expansion (CE-QE) improves lexical search like BM25 by expanding the original query with a few carefully chosen terms drawn from semantically related passages, so that relevant documents missing exact word matches can still be retrieved in the first stage. Instead of pulling terms from BM25’s own top results or using raw frequency counts, the method runs a cross-encoder over the top passages returned by a dense retriever and selects the tokens to which the model assigns the highest relevance attribution. The added terms are deliberately limited in number, appended rather than substituted, and produced by a one-time neural step that leaves the underlying BM25 index unchanged.

This section motivates each design choice before describing the mechanism, because the choices are not arbitrary—each responds to a specific way naive alternatives fail.

3.1 Why fix the query, not just the ranking

The standard way to add semantic awareness to a lexical result is to rerank it: retrieve with BM25, then re-score the candidates with a cross-encoder or fuse them with a semantic list. Reranking and fusion, however, can only reorder or combine documents that were retrieved in the first place. If a relevant document shares no surface terms with the query, BM25 never puts it in the candidate set, and no amount of downstream re-scoring can recover it—the recall ceiling is fixed at the first stage. This is precisely the vocabulary-gap failure mode motivating this paper (Section 1): lexical search retrieves the wrong set, not just ranks the right set poorly. Fixing it therefore requires intervening before retrieval, at the query itself, so that the missing document has a chance to be retrieved at all. CE-QE is a query-side fix for exactly this reason: it changes what BM25 searches for, not how BM25's results are consumed afterward.

3.2 Why semantic passages, not BM25's own top results, seed expansion

Classical pseudo-relevance feedback (RM3 and relatives [9]) expands the query with terms drawn from the top documents that the same lexical retriever already returned. This is self-referential in a way that hurts exactly the queries CE-QE targets: if BM25's top results are wrong because of a vocabulary gap, feeding terms back from those same wrong results reinforces the error rather than correcting it (query drift). CE-QE instead seeds expansion from the semantic retriever's top passages. Dense retrieval finds topically related passages by meaning, independent of surface term overlap, so its top results remain a trustworthy seed set precisely in the cases where BM25's own results are not. This is the central design insight: use the retriever that does not have the failure mode to repair the retriever that does.

3.3 Why cross-encoder attribution, not raw term statistics

Given a set of semantically relevant passages, the next question is which of their words to add to the query. A pseudo-relevance-feedback-style answer would score candidate terms by frequency or TF-IDF within those passages. This is a weak signal here: a passage relevant to "side effects of ibuprofen" also contains many generic, high-frequency words (include, common, effects itself) that are not what makes the passage relevant, and frequency statistics cannot tell the two apart. A cross-encoder, by contrast, is trained end-to-end to predict query–passage relevance, and its per-token attributions reveal which tokens the model actually used to make that judgment. For the query "side effects of ibuprofen" and the passage "Common side effects of ibuprofen include nausea and dizziness", the cross-encoder assigns high attribution to nausea (0.94) and dizziness—content terms absent from the query—and low attribution to the generic scaffolding around them. This is a supervised, discriminative-term selector obtained for free from a component the pipeline needs anyway for reranking, rather than a separate model that must be trained or tuned.

A second reason to prefer attribution over whole-passage injection is query compactness. Appending an entire top passage to the query would dilute BM25's IDF weighting with a burst of common terms and inflate query length past the point where term-frequency saturation ($k_1$) still discriminates—long, diffuse queries under-reward the terms that matter. Extracting only a handful of high-attribution tokens keeps the expanded query short and each added term individually discriminative, which is what BM25's scoring function is designed to exploit.

3.4 The CE-QE algorithm

CE-QE (Figure 1) runs semantic search for the original query, retrieves the top passages, applies the cross-encoder, and extracts the highest-attribution content tokens across those passages (e.g., nausea, NSAID, pain). It appends the extracted tokens to the original query and runs BM25 on the expanded query. In our experiments up to 10 tokens are extracted from up to 10 documents; both limits are deliberate rather than incidental. Bounding the number of source passages caps the risk of topic drift—attribution scores taper quickly past the first few passages, and terms drawn from lower-ranked, less relevant passages are increasingly likely to be generic or off-topic—while bounding the token budget keeps the expanded query short for the reason given in Section 3.3. Terms are appended to the original query rather than replacing it, so the original terms keep contributing their own IDF weight and BM25 can still reward an exact match; expansion only adds recall opportunities, it does not remove the precision the original query already had.

The BM25 index itself is untouched; only the query changes. This is a deliberate separation of concerns: selecting expansion terms is the one part of the pipeline that needs a neural model, and it runs once per query over a small, bounded set of candidate passages (at most 10) rather than over the corpus, so its cost does not grow with corpus size regardless of which BM25 index answers the expanded query. In this paper we evaluate CE-QE against standard, flat BM25 (Section 4.1); we have not run CE-QE against a hierarchical, billion-scale BM25 index end-to-end. The separation of concerns is what makes that combination architecturally straightforward—the expanded query is ordinary BM25 input, so it needs no special handling by any particular lexical index's internal structure—but we present it as a compositional argument in Section 5 rather than as a measured result.

CE-QE targets a specific failure mode. Consider the query "Brown State Fishing Lake is in a country that has a population of how many inhabitants?" whose answer document is about "Brown County, Kansas … the county population was 9,984." Lexical search fixates on population/country; semantic search drifts to lakes/countries. CE-QE extracts county, kansas, census from the top passages and adds them to the BM25 query, which then matches the answer document that neither stage found alone.

3.5 Why the benefit concentrates on high vocabulary-divergence datasets

CE-QE's mechanism predicts where it should help: datasets where queries and their answer passages are phrased in different registers should benefit most, and datasets where they already share vocabulary should benefit least. This matches the evaluation (Section 4). Natural Questions and TREC-COVID pair informal or lay-phrased queries against encyclopedic or technical-scientific answer text, respectively—a wide register gap that expansion closes. FEVER, by contrast, is a fact-verification dataset whose claims are written to closely paraphrase the Wikipedia sentences they check against, so query and answer vocabulary already overlap and expansion has little left to add (and, as Section 4 shows, can slightly hurt by adding terms the original query did not need). This dataset-dependent pattern is evidence that CE-QE is doing what it is designed to do—closing a vocabulary gap—rather than improving retrieval through some unrelated, dataset-independent effect.

3.6 Semantically Enriched Score Fusion (SESF)

SESF uses CE-QE as the lexical stage of a score-fusion hybrid: the enriched BM25 results are combined with the semantic results, and the merged set is reranked. The motivation for fusing rather than using CE-QE's lexical list alone is that fusion and expansion attack the same problem from different angles and their gains are not redundant: expansion raises the ceiling on what BM25 can retrieve, while fusion still contributes independent evidence from the semantic embedding whenever the two retrievers disagree. Because the lexical list going into fusion is now itself semantically aware, the two lists overlap more on genuinely relevant documents than plain BM25 $+$ vector fusion would, giving the fusion step—and the reranker downstream—a stronger and more concentrated candidate pool to work with. A reranker can only reorder the candidates it is given, so improving the pool improves the ceiling on what reranking can achieve, which is why SESF's gains over CE score fusion (Section 4) show up primarily as higher Recall@100 rather than as a change in how well already-retrieved items are ranked.

4. Evaluation

Section Summary: The evaluation tested a range of retrieval methods on seven standard benchmark datasets, each with up to thousands of documents, using a consistent pipeline that combined keyword search, semantic embeddings, fusion techniques, and optional reranking. Hybrid fusion of keyword and semantic results improved recall by roughly 4–10 percent over either method alone, while weighted score fusion edged out simpler merging for final ranking quality. Adding a cross-encoder reranker or query-expansion step delivered further gains in both recall and ranking accuracy, but only when limited to a small candidate set; otherwise latency rose by hundreds of times, motivating the bounded approaches tested.

4.1 Setup

Experiments run through RUMIR, a pipeline built for this work that indexes BEIR datasets, executes lexical, semantic, fusion, reranking, and expansion variants, and evaluates them consistently. Table 1 lists the seven BEIR datasets; each configuration is evaluated over 1,000 queries at top- $k{=}100$. Embeddings use granite-embedding-30m-english; the semantic index is IVF_PQ (LanceDB) or HNSW (OpenSearch). Rerankers are bge-reranker-v2-m3 (568M params) and ms-marco-MiniLM-L12-v2 (33.4M params, the OpenSearch default). The lexical stage in every result below—BM25, RRF, CE score fusion, CE-QE, and SESF alike—is standard, flat BM25 over each dataset's own (at most 8.8M-document) corpus.

: Table 1: BEIR datasets used in the evaluation.

Dataset	Documents	Domain
MSMARCO	8.8M	Web / QA
HotpotQA	5.2M	Wikipedia
NQ	2.7M	Wikipedia
FEVER	5.4M	Wikipedia (fact check)
Climate-FEVER	5.4M	Climate claims
TREC-COVID	171K	Biomedical
FiQa	57K	Finance

4.2 Hybrid fusion vs. single retrievers

Fusing BM25 with semantic search via RRF improves Recall@100 on every dataset (Figure 2): on average $\sim$ 4.3% over semantic-only and $\sim$ 10% over BM25-only, with the largest gains where the two retrievers disagree most (MSMARCO, FiQa). Comparing the two fusion strategies, RRF and weighted fusion reach near-identical Recall@100, but weighted fusion ranks better—higher nDCG@10 on five of seven datasets (Figure 3)—because it preserves score magnitude that RRF discards.

4.3 Two-stage reranking: gain and cost

Adding a cross-encoder reranker (CE score fusion) over the fused candidates raises quality, and the gain is larger in nDCG@10 than in Recall@100—the reranker mostly reorders items already retrieved rather than surfacing new ones. On TREC-COVID, for instance, nDCG@10 rises from 0.71 (RRF) to 0.79 while Recall@100 rises from 0.70 to 0.83. The cost is the cross-encoder pass, which scales with the number of passages reranked (Figure 4): at 128 passages, MiniLM costs 43.5 ms per query but BGE costs 409.6 ms on an A100—a $\sim$ 9 $\times$ gap that makes reranker choice the dominant latency lever once the candidate set is fixed.

That per-passage cost compounds quickly if reranking is applied naively over a full, unbounded candidate set rather than a small one. We measure this directly, single-threaded, with the MiniLM-12L reranker (118M parameters) over the full hybrid candidate list for each of the seven BEIR datasets, comparing hybrid retrieval alone against hybrid retrieval followed by full-candidate-set reranking (Figure 5). Hybrid retrieval alone answers in 29–58 ms per query across datasets; adding full reranking raises this to 25–86 s per query—a 485–2051 $\times$ slowdown, worst on TREC-COVID (86.17 s, $2051\times$) and best on HotpotQA (26.21 s, $485\times$), tracking each dataset's average candidate-set size more than any other factor. This is the result that motivates confining reranking to a small, bounded candidate set rather than the full retrieved list: SESF and CE score fusion both rerank only the fused top- $k$ (Table 2), which is why their end-to-end latency lands in the hundreds of milliseconds rather than tens of seconds.

4.4 CE-QE: enriching the lexical stage

CE-QE improves BM25 directly (Figure 6). The largest gains appear where lexical and semantic retrieval otherwise diverge: NQ Recall@100 rises from 0.32 to 0.47, TREC-COVID from 0.56 to 0.67, and Climate-FEVER from 0.19 to 0.26. FEVER dips slightly (0.71 $\rightarrow$ 0.70) where surface terms already match, so added tokens contribute little. Because CE-QE only rewrites the query, these gains require no change to the underlying BM25 index—here, standard flat BM25—and the same argument applies unchanged if a different, larger-scale BM25 index answers the expanded query instead (Section 5).

4.5 SESF: fusion quality and comparison to SOTA

Using CE-QE inside score fusion (SESF) beats cross-encoder score fusion by 2.5% on Recall@100 at comparable nDCG@10 (Figure 7 a), with the clearest wins on HotpotQA, TREC-COVID, and Climate-FEVER. Against strong learned models, SESF beats ColBERTv2 by 4.6% and SPLADEv2 by 5.3% on nDCG@10 (Figure 7 b), while leaving the lexical index a standard BM25 index rather than a specialized sparse or late-interaction structure. Overall, query expansion with cross-encoders improves 6.6% over RRF, the common production default.

4.6 End-to-end latency

Table 2 places the query-time cost of each method (top- $k{=}100$, MSMARCO). RRF is cheapest because it fuses ranks only; CE score fusion adds the reranker pass; SESF adds semantic retrieval and cross-encoder token extraction on top, trading $\sim$ 400 ms of extra latency for its recall and SOTA-beating ranking. The extra cost is spent on CPU-friendly BM25 and a bounded reranker call, not on scaling the index.

\begin{tabular}{@lcl@}
\toprule
Method & Latency & Quality note \\
\midrule
RRF & $\sim$153.8\, ms & fusion baseline \\
CE score fusion & $\sim$400.3\, ms & $+$reranker; better nDCG@10 \\
SESF (ours) & $\sim$556.9\, ms & best recall; beats SOTA nDCG@10 \\
\midrule
\multicolumn{3}{@l@}{\footnotesize Reranker cost @128 passages: MiniLM 43.5\, ms, BGE 409.6\, ms (A100).}\\
\bottomrule
\end{tabular}

5. Discussion and Limitations

Section Summary: The approach has not yet been tested end-to-end on billion-scale document collections, though its design of simply expanding the input query text should allow it to work with any standard BM25 index without special modifications. Because the method adds extra computation at query time, it is best used selectively on queries that stand to benefit rather than applied in every case. Its effectiveness also depends on the quality of the underlying reranker’s token attributions, and sensitivity to other reranker choices has not been examined.

Composing with a scalable lexical index. CE-QE is evaluated here against standard flat BM25 over BEIR-scale corpora (at most 8.8M documents); we have not run it against a billion-chunk lexical index end-to-end. What makes that composition architecturally straightforward is exactly the separation of concerns in Section 3: CE-QE's output is an ordinary, expanded text query, not a modified index, a custom scoring function, or a change to term statistics, so any BM25 index—including a sharded or hierarchical one—answers it without special handling. We would expect CE-QE's recall gain to hold at scale, since it is a property of the query, not of how the index is internally partitioned, but we present this as an architectural expectation, not a measured result, and confirming it end-to-end is the natural next step.

Cost is reserved for queries that need it. CE-QE adds a semantic-search plus cross-encoder pass at query time (Table 2), so it trades latency for quality and is best applied selectively rather than unconditionally—on datasets whose surface terms already match the answer vocabulary (FEVER), the expansion adds cost for little gain (Section 3.5). A deployment could use a cheap signal (e.g., lexical-semantic agreement on the unexpanded query) to decide when expansion is worth its cost, though we have not evaluated such a gate here.

Attribution quality bounds expansion quality. CE-QE's expansion terms are only as good as the cross-encoder's attributions; a reranker with poorly calibrated or noisy token-level attributions would select less discriminative terms, and we have not studied sensitivity to reranker choice beyond the two models used in Section 4.

6. Conclusion

Section Summary: Traditional keyword search often fails when questions and documents use different words, and simply reordering already-retrieved results cannot recover the missing documents. CE-QE solves this by letting a cross-encoder judge relevance inside the initial results and then feeding those judged terms back into the original keyword query, all without any extra language-model generation step. The resulting expansions raise recall on vocabulary-mismatched queries and, when fused with other signals, outperform several strong neural baselines while leaving the underlying keyword index unchanged.

Lexical retrieval's blindness to the semantic vocabulary gap cannot be fixed by reranking or fusion alone, because both operate only on documents already retrieved. CE-QE fixes it at the source by importing a cross-encoder's token-level relevance judgments into the BM25 query, grounding every expansion term in a passage the corpus's own semantic retriever actually returned rather than in text a language model hallucinates, and adding no separate generation call. Lexical recall improves substantially where query and answer vocabulary diverge (e.g., NQ Recall@100 from 0.32 to 0.47), and its fusion form, SESF, beats cross-encoder score fusion by 2.5% on Recall@100 and beats SPLADEv2 and ColBERTv2 by 5.3% and 4.6% on nDCG@10, all while leaving the underlying BM25 index completely unmodified—a property that should let it compose with any lexical index, including one built for scale, without architectural change, pending direct measurement of that composition.

References

Section Summary: This section lists key academic papers that underpin research on information retrieval systems. The references span foundational techniques like probabilistic ranking models and efficient nearest-neighbor search, as well as newer neural approaches that use embeddings and language models to find relevant passages. They also cover benchmarks for evaluating these methods and applications such as question answering and retrieval-augmented generation.

[1] N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, and I. Gurevych. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In NeurIPS Datasets and Benchmarks, 2021.

[2] S. Robertson and H. Zaragoza. The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in IR, 3(4):333–389, 2009.

[3] V. Karpukhin, B. Oğuz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih. Dense Passage Retrieval for Open-Domain Question Answering. In EMNLP, 6769–6781, 2020.

[4] N. Reimers and I. Gurevych. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In EMNLP-IJCNLP, 3982–3992, 2019.

[5] Y. A. Malkov and D. A. Yashunin. Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE TPAMI, 42(4):824–836, 2018.

[6] H. Jégou, M. Douze, and C. Schmid. Product Quantization for Nearest Neighbor Search. IEEE TPAMI, 33(1):117–128, 2011.

[7] G. V. Cormack, C. L. A. Clarke, and S. Büttcher. Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. In SIGIR, 758–759, 2009.

[8] R. Nogueira and K. Cho. Passage Re-ranking with BERT. arXiv:1901.04085, 2019.

[9] V. Lavrenko and W. B. Croft. Relevance-Based Language Models. In SIGIR, 120–127, 2001.

[10] T. Formal, C. Lassance, B. Piwowarski, and S. Clinchant. SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval. arXiv:2109.10086, 2021.

[11] K. Santhanam, O. Khattab, J. Saad-Falcon, C. Potts, and M. Zaharia. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In NAACL, 3715–3734, 2022.

[12] L. Gao, X. Ma, J. Lin, and J. Callan. Precise Zero-Shot Dense Retrieval without Relevance Labels. In ACL, 1762–1777, 2023.

[13] L. Wang, N. Yang, and F. Wei. Query2doc: Query Expansion with Large Language Models. In EMNLP, 9414–9423, 2023.

[14] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997, 2023.