Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

convex risk minimization

Convex risk minimization is a framework in statistical machine learning where a predictive model is trained by minimizing an expected or empirical objective function defined by a convex surrogate loss. In many practical tasks, directly minimizing the true performance metric involves discrete or non-convex loss functions, such as the zero-one misclassification loss, which makes exact optimization computationally intractable. Convex risk minimization resolves this difficulty by replacing the non-convex target loss with a convex surrogate, such as hinge loss, cross-entropy loss, or logistic loss. This convexity ensures that the optimization landscape lacks problematic local minima and can be solved efficiently with standard convex optimization algorithms, while theoretical properties like calibration, excess risk bounds, and Bayes consistency guarantee that minimizing the surrogate loss effectively optimizes performance on the underlying target task.

1 item

Two-Stage Learning to Defer with Multiple Experts

Two-Stage Learning to Defer with Multiple Experts

Anqi Mao, Christopher Mohri, Mehryar Mohri, Yutao Zhong

Why you should read this

Proposes a theoretically grounded framework for two-stage learning to defer with multiple experts by designing surrogate losses with rigorous H\mathcal{H}-consistency guarantees that enable efficient post-hoc deferral without retraining the underlying predictor.

We study a two-stage scenario for learning to defer with multiple experts, which is crucial in practice for many applications. In this scenario, a predictor is derived in a first stage by training with a common loss function such as cross-entropy. In the second stage, a deferral function is learned to assign the most suitable expert to each input. We design a new family of surrogate loss functions for this scenario both in the score-based and the predictor-rejector settings and prove that they are supported by H-consistency bounds, which implies their Bayes-consistency. Moreover, we show that, for a constant cost function, our two-stage surrogate losses are realizable H-consistent. While the main focus of this work is a theoretical analysis, we also report the results of several experiments on CIFAR-10 and SVHN datasets.

Added

2026-09-26