Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

local input perturbations

Local input perturbations refer to small, bounded modifications or noise introduced directly to an input data point within its immediate neighborhood in the input space. In machine learning and neural network analysis, these variations are used to evaluate or enforce model robustness, smoothness, and stability around specific data samples. By analyzing how a model responds to slight deviations—such as random noise or targeted shifts within a confined radius—practitioners can assess the behavior of internal representations and decision boundaries. Restricting sensitivity to local input perturbations, often through mathematical constraints like Lipschitz continuity, prevents models from generating disproportionately large changes in output for minor changes in input, thereby improving generalization and preventing overfitting to isolated training points.

1 item

On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning

On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning

Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello, Concetto Spampinato, Simone Calderara

OrganizationsPeRCeiVe LabUniversity of CataniaUniversity of Modena and Reggio Emilia

Why you should read this

Proposes a surrogate objective that constrains layer-wise Lipschitz constants on replay data to prevent unstable decision boundaries and buffer overfitting across standard continual learning methods.

Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize on the latter to prevent catastrophic forgetting. This work draws attention to a hidden pitfall of this widespread practice: repeated optimization on a small pool of data inevitably leads to tight and unstable decision boundaries, which are a major hindrance to generalization. To address this issue, we propose Lipschitz-DrivEn Rehearsal (LiDER), a surrogate objective that induces smoothness in the backbone network by constraining its layer-wise Lipschitz constants w.r.t. replay examples. By means of extensive experiments, we show that applying LiDER delivers a stable performance gain to several state-of-the-art rehearsal CL methods across multiple datasets, both in the presence and absence of pre-training. Through additional ablative experiments, we highlight peculiar aspects of buffer overfitting in CL and better characterize the effect produced by LiDER. Code is available at https://github.com/aimagelab/LiDER.

Added

2026-09-26