Built independently by an author, for readers. Read the story and support ChapterPal
ChapterPal Logo
Log inSign up

keyword

Lipschitz-driven rehearsal

Lipschitz-driven rehearsal is a continual learning technique that enhances experience replay by enforcing mathematical smoothness across a neural network through constraints on its layer-wise Lipschitz constants. In standard rehearsal strategies, models repeatedly optimize over a small memory buffer of past examples to prevent catastrophic forgetting, which frequently causes overfitting and brittle decision boundaries around replayed data points. Lipschitz-driven rehearsal addresses this issue by bounding the rate of change of the network activations with respect to buffer samples, thereby promoting smoother representation spaces and more resilient decision boundaries. This regularizing objective helps maintain stability across previously learned tasks while improving overall generalization without compromising the acquisition of new knowledge.

1 item

On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning

On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning

Lorenzo Bonicelli, Matteo Boschini, Angelo Porrello, Concetto Spampinato, Simone Calderara

OrganizationsPeRCeiVe LabUniversity of CataniaUniversity of Modena and Reggio Emilia

Why you should read this

Proposes a surrogate objective that constrains layer-wise Lipschitz constants on replay data to prevent unstable decision boundaries and buffer overfitting across standard continual learning methods.

Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize on the latter to prevent catastrophic forgetting. This work draws attention to a hidden pitfall of this widespread practice: repeated optimization on a small pool of data inevitably leads to tight and unstable decision boundaries, which are a major hindrance to generalization. To address this issue, we propose Lipschitz-DrivEn Rehearsal (LiDER), a surrogate objective that induces smoothness in the backbone network by constraining its layer-wise Lipschitz constants w.r.t. replay examples. By means of extensive experiments, we show that applying LiDER delivers a stable performance gain to several state-of-the-art rehearsal CL methods across multiple datasets, both in the presence and absence of pre-training. Through additional ablative experiments, we highlight peculiar aspects of buffer overfitting in CL and better characterize the effect produced by LiDER. Code is available at https://github.com/aimagelab/LiDER.

Added

2026-09-26