Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

neural retriever

A neural retriever is an information retrieval model that uses neural networks to search and extract relevant documents or text passages from a large collection in response to a query. Unlike traditional retrieval systems that rely on exact keyword matching and lexical statistics, a neural retriever encodes queries and documents into continuous dense vector representations within a shared embedding space. Relevance is determined by measuring semantic proximity using mathematical similarity metrics such as dot products or cosine similarity, enabling the system to understand context, synonyms, and conceptual intent beyond surface-level wording. These models serve as fundamental components in advanced natural language processing pipelines, including open-domain question answering and retrieval-augmented generation systems, where they dynamically retrieve external knowledge to support language understanding and text generation.

1 item

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela

OrganizationsMetaNew York UniversityUniversity College London

Why you should read this

Proposes the definitive retrieval-augmented architecture linking a pre-trained retriever with a sequence-to-sequence generator trained end-to-end to mitigate hallucinations.

Part of:Essential research papers for AI engineering interviews,Prep reading for the retrieval-augmented generation interview,Prep reading for the search and retrieval systems interview

Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, their ability to access and precisely manipulate knowledge is still limited, and hence on knowledge-intensive tasks, their performance lags behind task-specific architectures. Additionally, providing provenance for their decisions and updating their world knowledge remain open research problems. Pre-trained models with a differentiable access mechanism to explicit non-parametric memory can overcome this issue, but have so far been only investigated for extractive downstream tasks. We explore a general-purpose fine-tuning recipe for retrieval-augmented generation (RAG) -- models which combine pre-trained parametric and non-parametric memory for language generation. We introduce RAG models where the parametric memory is a pre-trained seq2seq model and the non-parametric memory is a dense vector index of Wikipedia, accessed with a pre-trained neural retriever. We compare two RAG formulations, one which conditions on the same retrieved passages across the whole generated sequence, the other can use different passages per token. We fine-tune and evaluate our models on a wide range of knowledge-intensive NLP tasks and set the state-of-the-art on three open domain QA tasks, outperforming parametric seq2seq models and task-specific retrieve-and-extract architectures. For language generation tasks, we find that RAG models generate more specific, diverse and factual language than a state-of-the-art parametric-only seq2seq baseline.

Added

2026-02-14