Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

multinomial logistic loss

Multinomial logistic loss, commonly known as categorical cross-entropy loss or multiclass log loss, is a performance metric and optimization objective used in machine learning and statistics for multiclass classification problems. It measures the discrepancy between the true categorical label of an input and the predicted probability distribution generated by a model, which is typically computed across multiple mutually exclusive classes using the softmax function. Formally derived as the negative log-likelihood of the correct class under a multinomial distribution, the function assigns a severe penalty to predictions that assign low probability to the actual target. Because it provides a continuous, differentiable, and convex surrogate for discrete classification error, it is widely utilized for parameter estimation via gradient-based optimization in multinomial logistic regression and deep neural networks.

1 item

Two-Stage Learning to Defer with Multiple Experts

Two-Stage Learning to Defer with Multiple Experts

Anqi Mao, Christopher Mohri, Mehryar Mohri, Yutao Zhong

Why you should read this

Proposes a theoretically grounded framework for two-stage learning to defer with multiple experts by designing surrogate losses with rigorous H\mathcal{H}-consistency guarantees that enable efficient post-hoc deferral without retraining the underlying predictor.

We study a two-stage scenario for learning to defer with multiple experts, which is crucial in practice for many applications. In this scenario, a predictor is derived in a first stage by training with a common loss function such as cross-entropy. In the second stage, a deferral function is learned to assign the most suitable expert to each input. We design a new family of surrogate loss functions for this scenario both in the score-based and the predictor-rejector settings and prove that they are supported by H-consistency bounds, which implies their Bayes-consistency. Moreover, we show that, for a constant cost function, our two-stage surrogate losses are realizable H-consistent. While the main focus of this work is a theoretical analysis, we also report the results of several experiments on CIFAR-10 and SVHN datasets.

Added

2026-09-26