Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

multiple experts

In machine learning and automated decision-making systems, multiple experts refers to a set of two or more distinct decision-makers, such as human specialists, secondary predictive models, or domain-specific algorithms, available to provide outputs or labels for a given task. In frameworks like learning to defer and adaptive routing, these individual experts typically exhibit diverse capabilities, error profiles, and operational costs. A primary model or allocation mechanism assesses the input context and the relative strengths of each expert to decide whether to generate a prediction autonomously or route the input to the most appropriate expert in the group, aiming to optimize overall system accuracy, efficiency, and resource utilization.

1 item

Two-Stage Learning to Defer with Multiple Experts

Two-Stage Learning to Defer with Multiple Experts

Anqi Mao, Christopher Mohri, Mehryar Mohri, Yutao Zhong

Why you should read this

Proposes a theoretically grounded framework for two-stage learning to defer with multiple experts by designing surrogate losses with rigorous H\mathcal{H}-consistency guarantees that enable efficient post-hoc deferral without retraining the underlying predictor.

We study a two-stage scenario for learning to defer with multiple experts, which is crucial in practice for many applications. In this scenario, a predictor is derived in a first stage by training with a common loss function such as cross-entropy. In the second stage, a deferral function is learned to assign the most suitable expert to each input. We design a new family of surrogate loss functions for this scenario both in the score-based and the predictor-rejector settings and prove that they are supported by H-consistency bounds, which implies their Bayes-consistency. Moreover, we show that, for a constant cost function, our two-stage surrogate losses are realizable H-consistent. While the main focus of this work is a theoretical analysis, we also report the results of several experiments on CIFAR-10 and SVHN datasets.

Added

2026-09-26