Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

large-scale recommendation

Large-scale recommendation refers to the automated process and system architecture designed to identify, rank, and deliver personalized item suggestions to users from massive candidate pools containing millions or billions of items. Because evaluating complex scoring models over an entire corpus in real time is computationally prohibitive, these systems typically operate through a multi-stage pipeline comprising candidate retrieval, ranking, and re-ranking. The initial retrieval stage rapidly filters the vast item corpus down to a manageable subset of promising candidates using efficient indexing and vector similarity search methods, after which downstream stages apply more computationally intensive models to accurately score, personalize, and order the final recommendations under strict latency and throughput constraints.

1 item

Retrieval with Learned Similarities

Retrieval with Learned Similarities

Bailu Ding, Jiaqi Zhai

OrganizationsMetaMicrosoft

Why you should read this

Develops Mixture-of-Logits and an approximate top-k search algorithm to enable efficient retrieval with complex learned similarities, cutting latency by up to 66x while maintaining over 99% recall across recommendation and question answering tasks.

Retrieval plays a fundamental role in recommendation systems, search, and natural language processing (NLP) by efficiently finding relevant items from a large corpus given a query. Dot products have been widely used as the similarity function in such tasks, enabled by Maximum Inner Product Search (MIPS) algorithms for efficient retrieval. However, state-of-the-art retrieval algorithms have migrated to learned similarities. These advanced approaches encompass multiple query embeddings, complex neural networks, direct item ID decoding via beam search, and hybrid solutions. Unfortunately, we lack efficient solutions for retrieval in these state-of-the-art setups. Our work addresses this gap by investigating efficient retrieval techniques with expressive learned similarity functions. We establish Mixture-of-Logits (MoL) as a universal approximator of similarity functions, demonstrate that MoL's expressiveness can be realized empirically to achieve superior performance on diverse retrieval scenarios, and propose techniques to retrieve the approximate top-k results using MoL with tight error bounds. Through extensive experimentation, we show that MoL, enhanced by our proposed mutual information-based load balancing loss, sets new state-of-the-art results across heterogeneous scenarios, including sequential retrieval models in recommendation systems and finetuning language models for question answering; and our approximate top-kk algorithms outperform baselines by up to 66x in latency while achieving >.99 recall rate compared to exact algorithms.

Added

2026-09-29