Relevance assessments are evaluations that determine whether and to what degree a retrieved document or piece of information satisfies a specific search query or user information need. Used primarily in information retrieval testing and benchmark evaluations, these assessments establish the ground truth against which search engines and retrieval algorithms are measured. Human annotators or domain experts typically review query-document pairs and assign ratings, which can be binary classifications of relevant or non-relevant, or multi-level graded scales that capture varying depths of topical relevance and utility. These assessment datasets provide the foundational basis for calculating standard retrieval performance metrics such as precision, recall, and rank-based cumulative gain.
