Learning to Prompt for Continual Learning

Zifeng WangZizhao ZhangChen-Yu LeeHan ZhangRuoxi SunXiaoqi RenGuolong SuVincent PerotJennifer DyTomas Pfister

article2021CVPR1,401 citations

Introduces a prompt-based continual learning framework that dynamically selects learnable prompts to guide pre-trained models across sequential tasks, matching rehearsal-based performance without storing past data or requiring task identities at test time.

Listen

Modern artificial intelligence models struggle to learn new tasks sequentially because training on new information typically overwrites previously acquired knowledge, a phenomenon known as catastrophic forgetting. Conventional solutions mitigate this by storing past user data in memory buffers to periodically retrain the system or by relying on explicit task indicators during testing to select task-specific components. However, retaining historical data introduces significant data privacy risks and memory costs, while requiring explicit task labels at test time makes systems impractical for real-world scenarios where incoming data streams lack predefined boundaries.

The article evaluates a new continual learning framework called Learning to Prompt (L2P). The primary objective is to demonstrate that a pre-trained vision model can sequentially learn diverse tasks without updating its core weights, without relying on stored past examples, and without needing task identity labels at test time.

To achieve this, the article establishes a method that keeps a large pre-trained transformer model completely frozen and instead trains a small external memory space termed a prompt pool. Prompts act as compact, learnable instructions that guide the frozen model to perform specific tasks. Using an automated query-key matching mechanism, the system dynamically retrieves the most relevant prompt instructions for each input image. The framework was evaluated across standard image classification benchmarks covering class-incremental, domain-shifting, and boundary-free streaming environments, comparing performance against leading regularization, memory buffer, and architecture-expanding methods.

The evaluation revealed several key findings. First, L2P achieved state-of-the-art results across all evaluated benchmarks without retaining any past training data, reaching an average accuracy of 83.83% on Split CIFAR-100 and 81.14% on a diverse 5-dataset benchmark. Second, the system maintained low forgetting rates between 4.64% and 7.63%, vastly outperforming baseline methods without replay buffers, which suffered from severe forgetting rates between 27.77% and 94.63%. Third, in buffer-free settings, L2P performed competitively against or outperformed sophisticated rehearsal-based methods that required storing up to 50 samples per class. Fourth, prompt memory required minimal overhead, adding between 0.05% and 0.11% in additional parameters relative to the original model. Finally, the framework succeeded in continuous, task-agnostic streaming settings where task boundaries were entirely absent, achieving 88.34% accuracy.

These findings indicate that continual learning can be effectively achieved by learning dynamic instructions rather than continually retraining large model weights. For organizations deploying machine learning, this approach offers substantial reductions in computational training costs and eliminates the compliance and security liabilities associated with storing sensitive historical user data. Furthermore, decoupling shared and task-specific instructions enables fine-grained knowledge transfer across related tasks while preserving model flexibility.

Organizations seeking to maintain adaptable computer vision models in privacy-sensitive or resource-constrained environments should consider adopting prompt-pool architectures over traditional data-replay buffers. Before broad operational deployment, teams should conduct internal pilots to calibrate hyperparameter settings, particularly prompt pool capacity and selection sizes, based on task diversity. Future work should expand validation to non-transformer architectures, test modalities outside of vision such as natural language or audio, and evaluate the robustness of the system against adversarial security threats.

Google Northeastern University classification image classification random access memory continual learning prompt learning catastrophic forgetting

arXiv: 2112.08654 google-research/l2p

Paper: The Power of Scale for Parameter-Efficient Prompt Tuning, Brian Lester et al. (2021). Introduces continuous soft prompt tuning for frozen models, which provides the underlying parameter-efficient adaptation mechanism that L2P extends to continual learning.
Paper: GPT Understands, Too, Xiao Liu et al. (2021). Establishes continuous prompt optimization as an effective and stable alternative to model fine-tuning, laying the groundwork for prompt-based knowledge isolation.
Paper: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing, Pengfei Liu et al. (2021). Provides a comprehensive taxonomy and formulation of prompt-based learning methods that motivate prompting as a memory and task instruction mechanism.
Paper: Dark Experience for General Continual Learning: a Strong, Simple Baseline, Pietro Buzzega et al. (2020). Offers a key baseline in general continual learning against which non-rehearsal prompt memory frameworks are compared.
Paper: Gradient Episodic Memory for Continual Learning, David Lopez-Paz et al. (2017). Defines core principles of episodic memory and knowledge transfer in continual learning that motivated alternative, buffer-free memory designs like prompt pools.
Paper: A Continual Learning Survey: Defying Forgetting in Classification Tasks, Matthias De Lange et al. (2019). Provides a structured overview and taxonomy of traditional continual learning paradigms, setting up the catastrophic forgetting challenges L2P seeks to resolve.
Paper: iCaRL: Incremental Classifier and Representation Learning, Sylvestre-Alvise Rebuffi et al. (2016). Serves as the foundational exemplar-based incremental learning approach that L2P seeks to surpass without using replay buffers.
Paper: Learning without Forgetting, Zhizhong Li et al. (2016). Presents early foundational work on adapting neural networks sequentially without storing past task training data.

Paper: Visual Prompt Tuning, Menglin Jia et al. (2022). Extends prompt-based parameter-efficient adaptation systematically across vision Transformer backbones.
Paper: Conditional Prompt Learning for Vision-Language Models, Kaiyang Zhou et al. (2022). Builds upon prompt learning concepts by generating dynamic, instance-conditioned prompts to enhance generalization across novel categories.
Paper: Towards a Unified View of Parameter-Efficient Transfer Learning, Junxian He et al. (2022). Unifies prompt tuning with other parameter-efficient adaptation methods to analyze their functional components.
Paper: A Comprehensive Survey of Continual Learning: Theory, Method and Application, Liyuan Wang et al. (2023). Synthesizes modern continual learning approaches, framing prompt-based and representation-based memory systems within the broader theoretical landscape.
Paper: LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning, Bo Liu et al. (2023). Applies and benchmarks lifelong learning transfer mechanisms across complex multi-task decision-making domains.
Paper: Learning, Fast and Slow: Towards LLMs That Adapt Continually, Rishabh Tiwari et al. (2026). Explores combining slow parameter adaptation with fast prompt optimization for continual learning in large language models.

Cover for Learning to Prompt for Continual Learning

Abstract

The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehensive experiments under popular image classification benchmarks with different challenging continual learning settings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a rehearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at this https URL.

Topics

classification image classification random access memory source codes memory systems distribution functions online learning

Keywords

continual learning prompt learning catastrophic forgetting prompt pool learnable prompts task-agnostic learning prompt selection mechanism episodic memory rehearsal-based methods

1 Introduction
2 Related Work
3 Prerequisites
3.1 Continual learning protocols
3.2 Prompt-based learning and baselines
4 Learning to Prompt (L2P)
4.1 From prompt to prompt pool
4.2 Instance-wise prompt query
4.3 Optimization objective for L2P
5 Experiments
5.1 Comparing methods
5.2 Datasets and experimental details
5.3 Main results
5.4 Effectiveness of core designs
6 Conclusion
References
Potential negative societal impact
Limitations
Dataset details and licensing information
Algorithm details

Knowls

Knowl 1 — Learning to Prompt (L2P) Framework and Prompt Pool
model/method
Learning to Prompt (L2P) is a continual learning framework that keeps a pre-trained sequence backbone model frozen and instead adapts to sequential tasks by learning a memory space of prompt parameters, termed a prompt pool.

Let the pre-trained vision transformer model be $f = f_r \circ f_e$ , where $f_e : \mathbb{R}^{L \times (S^2 \cdot C)} \to \mathbb{R}^{L \times D}$ is the input patch embedding layer ( $L$ is the number of token patches, $S$ is the patch spatial size, $C$ is the channel count, and $D$ is the embedding dimension) and $f_r$ denotes the sequence of self-attention Transformer blocks. Given an input image $x \in \mathbb{R}^{H \times W \times C}$ , its patch embedding sequence is $x_e = f_e(x) \in \mathbb{R}^{L \times D}$ , which includes the pre-trained [class] token.

A prompt pool $\mathcal{P}$ consists of $M$ learnable prompt tensors: $\mathcal{P} = \{P_1, P_2, \dots, P_M\}, \quad P_j \in \mathbb{R}^{L_p \times D}$ where $L_p$ is the token length of an individual prompt and $D$ matches the embedding dimension of $x_e$ .

For an input sample, a subset of $N$ prompt indices $\{s_i\}_{i=1}^N \subseteq \{1, \dots, M\}$ (with $1 \le N \le M$ ) is selected dynamically. The extended token sequence $x_p$ is constructed by concatenating the selected prompt tokens with the input embeddings along the sequence length dimension: $x_p = [P_{s_1}; P_{s_2}; \dots; P_{s_N}; x_e] \in \mathbb{R}^{(N L_p + L) \times D}$

This parameterization adds negligible learnable parameters (approximately $0.05\%$ to $0.11\%$ of the backbone size) and allows parameter sharing across similar tasks while assigning dedicated prompt parameters to dissimilar inputs, eliminating the need for a rehearsal memory buffer.
Knowl 2 — Instance-Wise Prompt Query and Key-Value Memory Lookup
model/method
To dynamically retrieve relevant prompts without requiring task identity at training or test time, L2P structures the prompt pool as a key-value memory $\{(k_1, P_1), (k_2, P_2), \dots, (k_M, P_M)\}$ , where each prompt $P_j \in \mathbb{R}^{L_p \times D}$ is paired with a learnable key vector $k_j \in \mathbb{R}^{D_k}$ , and $\mathcal{K} = \{k_j\}_{j=1}^M$ .

An input instance $x$ is mapped to query feature space via a deterministic, parameter-frozen query function $q(x) = f(x)[0, :] \in \mathbb{R}^{D_k}$ , using the [class] token feature produced by the pre-trained backbone $f$ . The match between the query $q(x)$ and a key $k_i$ is scored using a distance function $\gamma(q(x), k_i)$ , implemented as cosine distance: $\gamma(u, v) = 1 - \frac{u \cdot v}{\|u\|_2 \|v\|_2}$

The subset of top- $N$ keys $\mathcal{K}_x = \{k_{s_1}, \dots, k_{s_N}\}$ for instance $x$ is selected by minimizing the matching distance: $\mathcal{K}_x = \underset{\{s_i\}_{i=1}^N \subseteq \{1, \dots, M\}}{\operatorname{argmin}} \sum_{i=1}^N \gamma(q(x), k_{s_i})$

Diversifying Prompt Selection with Task Boundary Prior (Optional): When discrete task boundaries are known during training, prompt selection can be encouraged to diversify across tasks. A selection frequency vector $H_t = [h_1, h_2, \dots, h_M]$ tracks the normalized selection frequency of each prompt up to task $t - 1$ . During training of task $t$ , the lookup objective penalizes frequently chosen prompts: $\mathcal{K}_x = \underset{\{s_i\}_{i=1}^N \subseteq \{1, \dots, M\}}{\operatorname{argmin}} \sum_{i=1}^N \gamma(q(x), k_{s_i}) \cdot h_{s_i}$ At test time, prompt lookup strictly reverts to the unpenalized formulation.
Knowl 3 — L2P Optimization Objective
equation
The prompt pool tensors $\mathcal{P} = \{P_j\}_{j=1}^M$ , prompt keys $\mathcal{K} = \{k_j\}_{j=1}^M$ , and classification head parameters $\phi$ of model $g_\phi$ are trained end-to-end by minimizing the loss function:

$\min_{\mathcal{P}, \mathcal{K}, \phi} \mathcal{L}\left(g_\phi\left(f_r^{\text{avg}}(x_p)\right), y\right) + \lambda \sum_{k_{s_i} \in \mathcal{K}_x} \gamma\left(q(x), k_{s_i}\right) \quad \text{s.t.} \quad \mathcal{K}_x = \underset{\{s_i\}_{i=1}^N \subseteq [1, M]}{\operatorname{argmin}} \sum_{i=1}^N \gamma\left(q(x), k_{s_i}\right)$

where:
- $x_p = [P_{s_1}; \dots; P_{s_N}; x_e]$ is the prompt-prepended token sequence for input $x$ .
- $f_r^{\text{avg}}(x_p) = \operatorname{AvgPool}\left(f_r(x_p)[0 : N L_p, :]\right) \in \mathbb{R}^D$ is the average-pooled representation of the output hidden vectors corresponding exclusively to the $N \cdot L_p$ prompt token positions from Transformer layers $f_r$ .
- $\mathcal{L}(\cdot, y)$ is the standard softmax cross-entropy classification loss against target class label $y$ .
- $\gamma(q(x), k_{s_i})$ is the cosine distance between the frozen input query $q(x)$ and the selected prompt key $k_{s_i}$ . This surrogate loss pulls selected keys closer to the query features of inputs that use them.
- $\lambda > 0$ is a scalar balancing weight (empirically set to $\lambda = 0.5$ ).
Knowl 4 — L2P Training Algorithm
algorithm
The complete training procedure for Learning to Prompt (L2P) across a sequence of $T$ continual tasks is defined below:
Input: Pre-trained embedding layer $f_e$ , pre-trained Transformer layers $f_r$ , classification layer $g_\phi$ , prompt pool $\mathcal{P} = \{P_j\}_{j=1}^M$ , prompt keys $\mathcal{K} = \{k_j\}_{j=1}^M$ , task training sets $\{\mathcal{D}_t\}_{t=1}^T$ with $\mathcal{D}_t = \{(x_i^t, y_i^t)\}_{i=1}^{n_t}$ , epochs per task $E_t$ , learning rate $\eta$ , balance parameter $\lambda$ .
Initialize: Head parameters $\phi$ , prompt pool $\mathcal{P}$ , prompt keys $\mathcal{K}$ .
for $t = 1, \dots, T$ do
    for $e = 1, \dots, E_t$ do
        for each mini-batch $\mathcal{B} = \{(x_i, y_i)\}_{i=1}^B \subset \mathcal{D}_t$ do
            Initialize batch chosen keys $\mathcal{K}_\mathcal{B} = \emptyset$ , batch chosen prompts $\mathcal{P}_\mathcal{B} = \emptyset$
            Initialize batch loss $\mathcal{L}_\mathcal{B} = 0$
            for $(x, y) \in \mathcal{B}$ do
                Compute frozen query feature $q(x) = f(x)[0, :]$
                Find top- $N$ keys $\mathcal{K}_x = \operatorname{argmin}_{\{s_i\}_{i=1}^N \subseteq [1, M]} \sum_{i=1}^N \gamma(q(x), k_{s_i})$
                Retrieve corresponding prompts $\mathcal{P}_x = \{P_{s_1}, \dots, P_{s_N}\}$
                Compute embedding tokens $x_e = f_e(x)$
                Concatenate prompt and input tokens $x_p = [P_{s_1}; \dots; P_{s_N}; x_e]$
                Compute pooled prompt features $f_r^{\text{avg}}(x_p) = \operatorname{AvgPool}(f_r(x_p)[0 : N L_p, :])$
                Compute sample loss $\mathcal{L}_x = \mathcal{L}(g_\phi(f_r^{\text{avg}}(x_p)), y) + \lambda \sum_{k \in \mathcal{K}_x} \gamma(q(x), k)$
                Accumulate batch loss $\mathcal{L}_\mathcal{B} \leftarrow \mathcal{L}_\mathcal{B} + \mathcal{L}_x$
                Update tracking sets $\mathcal{K}_\mathcal{B} \leftarrow \mathcal{K}_\mathcal{B} \cup \mathcal{K}_x$ , $\mathcal{P}_\mathcal{B} \leftarrow \mathcal{P}_\mathcal{B} \cup \mathcal{P}_x$
            end for
            for $(k, P) \in \operatorname{zip}(\mathcal{K}_\mathcal{B}, \mathcal{P}_\mathcal{B})$ do
                 $k \leftarrow k - \eta \nabla_k \mathcal{L}_\mathcal{B}$
                 $P \leftarrow P - \eta \nabla_P \mathcal{L}_\mathcal{B}$
            end for
             $\phi \leftarrow \phi - \eta \nabla_\phi \mathcal{L}_\mathcal{B}$
        end for
    end for
end for

Knowl 5 — Class-Incremental Performance on Split CIFAR-100 and 5-Datasets

data/table

Evaluation of L2P and L2P-R (L2P with rehearsal buffer) against fine-tuning, regularization, and rehearsal baselines on class-incremental benchmarks using a pre-trained ViT-B/16 backbone. Split CIFAR-100 contains 10 tasks (10 classes each). 5-datasets contains CIFAR-10, MNIST, Fashion-MNIST, SVHN, and notMNIST sequentially.

Method	Buffer Size	Split CIFAR-100		5-datasets
		Average Acc (%) $\uparrow$	Forgetting (%) $\downarrow$	Average Acc (%) $\uparrow$	Forgetting (%) $\downarrow$
FT-seq-frozen	0	$17.72 \pm 0.34$	$59.09 \pm 0.25$	$39.49 \pm 0.12$	$42.62 \pm 0.20$
FT-seq	0	$33.61 \pm 0.85$	$86.87 \pm 0.20$	$20.12 \pm 0.42$	$94.63 \pm 0.68$
EWC	0	$47.01 \pm 0.29$	$33.27 \pm 1.17$	$50.93 \pm 0.09$	$34.94 \pm 0.07$
LwF	0	$60.69 \pm 0.63$	$27.77 \pm 2.17$	$47.91 \pm 0.33$	$38.01 \pm 0.28$
L2P (ours)	0	$\mathbf{83.83 \pm 0.04}$	$\mathbf{7.63 \pm 0.30}$	$\mathbf{81.14 \pm 0.93}$	$\mathbf{4.64 \pm 0.52}$
ER	10/class	$67.87 \pm 0.57$	$33.33 \pm 1.28$	$80.32 \pm 0.55$ (5/class)	$15.69 \pm 0.89$ (5/class)
GDumb	10/class	$67.14 \pm 0.37$	–	$56.99 \pm 0.06$ (5/class)	–
BiC	10/class	$66.11 \pm 1.76$	$35.24 \pm 1.64$	$78.74 \pm 1.41$ (5/class)	$21.15 \pm 1.00$ (5/class)
DER++	10/class	$61.06 \pm 0.87$	$39.87 \pm 0.99$	$80.81 \pm 0.07$ (5/class)	$14.38 \pm 0.35$ (5/class)
Co2L	10/class	$72.15 \pm 1.32$	$28.55 \pm 1.56$	$82.25 \pm 1.17$ (5/class)	$17.52 \pm 1.35$ (5/class)
L2P-R (ours)	10/class	$\mathbf{84.21 \pm 0.53}$	$\mathbf{7.72 \pm 0.77}$	$\mathbf{85.56 \pm 0.95}$ (5/class)	$\mathbf{4.22 \pm 0.03}$ (5/class)
ER	50/class	$82.53 \pm 0.17$	$16.46 \pm 0.25$	$84.26 \pm 0.84$ (10/class)	$12.85 \pm 0.62$ (10/class)
GDumb	50/class	$81.67 \pm 0.02$	–	$70.76 \pm 0.12$ (10/class)	–
BiC	50/class	$81.42 \pm 0.85$	$17.31 \pm 1.02$	$85.53 \pm 2.06$ (10/class)	$10.27 \pm 1.32$ (10/class)
DER++	50/class	$83.94 \pm 0.34$	$14.55 \pm 0.73$	$84.88 \pm 0.57$ (10/class)	$10.46 \pm 1.02$ (10/class)
Co2L	50/class	$82.49 \pm 0.89$	$17.48 \pm 1.80$	$86.05 \pm 1.03$ (10/class)	$12.28 \pm 1.44$ (10/class)
L2P-R (ours)	50/class	$\mathbf{86.31 \pm 0.59}$	$\mathbf{5.83 \pm 0.61}$	$\mathbf{88.95 \pm 0.78}$ (10/class)	$\mathbf{4.92 \pm 0.71}$ (10/class)
Upper-bound	–	$90.85 \pm 0.12$	–	$93.93 \pm 0.18$	–

Without any rehearsal buffer (buffer size 0), L2P achieves $83.83\%$ average accuracy on Split CIFAR-100 and $81.14\%$ on 5-datasets, substantially exceeding non-rehearsal methods (EWC at $47.01\%$ , LwF at $60.69\%$ ) and surpassing 10/class rehearsal baselines (Co2L at $72.15\%$ ).

Knowl 6 — Comparison with Architecture-Based Continual Learning Methods

data/table

Performance comparison on Split CIFAR-100 against architecture-based continual learning methods that operate without test-time task identity. Performance is evaluated by the absolute average accuracy and the difference metric $\text{Diff} = \text{Upper-bound Acc} - \text{Method Acc}$ (where lower difference is better) to account for differing backbone architectures.

Method	Backbone	Avg. Acc (%) $\uparrow$	Diff (%) $\downarrow$
Upper-bound	ResNet18	80.41	–
SupSup	ResNet18	$28.34 \pm 2.45$	52.07
DualNet	ResNet18	$40.14 \pm 1.64$	40.27
Upper-bound	ViT-B/16	90.85	–
L2P (ours)	ViT-B/16	$\mathbf{83.83 \pm 0.04}$	$\mathbf{7.02}$

L2P reduces the gap to the multi-task upper-bound from $>40\%$ (in DualNet and SupSup) down to $7.02\%$ , while adding only $\sim 0.1\%$ parameters compared to architecture expansion methods that add substantial parameters or sub-networks per task.

Knowl 7 — Domain-Incremental and Task-Agnostic Continual Learning Benchmark Results

data/table

Evaluation on domain-incremental continual learning (CORe50 dataset across 8 training domains) and task-agnostic continual learning (Gaussian scheduled CIFAR-100, where class distributions drift continuously without explicit task switches).

Method	CORe50 (Domain-Incremental)		Gaussian CIFAR-100 (Task-Agnostic)
	Buffer Size	Test Acc (%) $\uparrow$	Buffer Size	Test Acc (%) $\uparrow$
EWC	0	$74.82 \pm 0.60$	0	$63.04 \pm 0.42$
LwF	0	$75.45 \pm 0.40$	–	–
L2P (ours)	0	$\mathbf{78.33 \pm 0.06}$	0	$\mathbf{88.34 \pm 0.14}$
ER	50/class	$80.10 \pm 0.56$	50/class	$82.63 \pm 0.27$
GDumb	50/class	$74.92 \pm 0.25$	50/class	$81.67 \pm 0.02$
BiC	50/class	$79.28 \pm 0.30$	–	–
DER++	50/class	$79.70 \pm 0.44$	50/class	$85.24 \pm 0.71$
Co2L	50/class	$79.75 \pm 0.84$	–	–
L2P-R (ours)	50/class	$\mathbf{81.07 \pm 0.13}$	50/class	$\mathbf{88.92 \pm 0.39}$
Upper-bound	–	$82.15 \pm 0.37$	–	$90.85 \pm 0.12$

In the task-agnostic Gaussian scheduled setting, L2P without a buffer ( $88.34\%$ ) outperforms all baseline methods even when they are equipped with a 50/class replay buffer (DER++ at $85.24\%$ , ER at $82.63\%$ ).

Knowl 8 — Ablation of L2P Core Architectural Components

data/table

Ablation study on the 5-datasets continual learning benchmark measuring the necessity of each core mechanism in L2P:

Ablated Component	Average Acc (%) $\uparrow$	Forgetting (%) $\downarrow$
w/o prompt pool (single prompt trained sequentially)	51.96	26.60
w/o key-value pair (using mean of prompt as key)	58.33	20.45
w/o diversified selection (free selection across tasks)	62.26	17.84
None (full L2P framework)	81.14	4.64

Prompt pool: Replacing the pool with a single shared prompt causes average accuracy to collapse from $81.14\%$ to $51.96\%$ and forgetting to increase from $4.64\%$ to $26.60\%$ , demonstrating that a single prompt suffers catastrophic interference.
Key-value decoupling: Removing separate learnable keys and using prompt mean embeddings as keys drops performance to $58.33\%$ , showing the necessity of decoupling prompt content learning from query matching.
Diversified selection: Omitting the task-frequency penalty on diverse multi-domain datasets decreases accuracy to $62.26\%$ , indicating that frequency penalization prevents negative transfer between unrelated distributions.

Knowl 9 — Prompt Hyperparameter Sensitivity and Sharing Dynamics
empirical result
The performance of L2P depends on three primary hyperparameters:
1. Prompt pool size $M$ : On datasets with high domain diversity (e.g., 5-datasets), increasing $M$ from 5 to 30 steadily improves average accuracy from $\approx 67\%$ to over $81\%$ . On datasets with high intra-task similarity (e.g., Split CIFAR-100), increasing $M$ beyond 10 provides diminishing returns (accuracy remains plateaued around $83.5\%$ to $83.8\%$ ), indicating that diverse tasks require a larger prompt capacity to allocate task-specific prompts.
2. Prompt token length $L_p$ : An extremely short prompt length ( $L_p = 1$ ) reduces accuracy across datasets (e.g., drops to $77.01\%$ on Split CIFAR-100 and $70.87\%$ on 5-datasets for $N=1$ ). Increasing length to $L_p = 5$ or $10$ delivers optimal capacity, whereas oversized prompts ( $L_p = 20$ ) degrade accuracy due to underfitting.
3. Selection count $N$ : Selecting $N = 4$ or $5$ prompts per sample achieves optimal performance across benchmarks. When $N$ matches $M$ (all prompts selected for every instance), performance degrades to $79.65\%$ on Split CIFAR-100.
Empirical prompt selection histograms reveal that tasks with high semantic similarity (CIFAR-100) automatically share the majority of prompt indices across tasks, whereas distinct task domains (5-datasets) recruit non-overlapping prompt subsets.
Knowl 10 — Limitations of L2P
limitation
The L2P framework has several documented constraints and open challenges:
1. Pre-trained sequence model dependency: L2P relies on pre-trained sequence-based architectures (such as Vision Transformers) where continuous prompt tokens can be prepended directly to token embeddings. It is not directly compatible with standard convolutional neural network (ConvNet) architectures without structural modification.
2. Modalities evaluated: Experimental validation is conducted exclusively on computer vision classification benchmarks; application to other modalities (e.g., speech or multi-modal domains) remains unverified within the work.
3. Synthetic task-agnostic evaluation: Benchmarking in the task-agnostic setting relies on synthetic datasets (Gaussian scheduled CIFAR-100) that do not capture the complexity of real-world non-stationary streaming distributions.
4. Inherited pre-training biases and robustness: Pre-trained backbones risk propagating representation biases to downstream continual tasks, and the robustness of prompt-guided representations against adversarial perturbations in safety-critical settings is not assessed.

Coverage note — None was omitted; all key methodology components (prompt pool, key-value query mechanism, loss objective, algorithm), experimental results (class-incremental, domain-incremental, task-agnostic, architecture baselines), component ablations, hyperparameter sensitivity, and limitations are fully covered.

References

1.Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, 2018. 2
2.Yaroslav Bulatov. notmnist dataset, 2011. 12
3.Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In NeurIPS, 2020. 1, 2, 5, 6, 7
4.Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co2l: Contrastive continual learning. In ICCV, 2021. 1, 3, 5, 6
5.Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, pages 532–547, 2018. 8
6.Arslan Chaudhry, Albert Gordo, Puneet Kumar Dokania, Philip Torr, and David Lopez-Paz. Using hindsight to anchor past knowledge in continual learning. arXiv preprint arXiv:2002.08165, 2(7), 2020. 2
7.Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420, 2018. 2, 7
8.Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc’Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902.10486, 2019. 1, 2, 6, 7
9.Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. TPAMI, 2021. 5
10.Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. ICLR, 2021. 4
11.Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR. OpenReview.net, 2021. 5
12.Sayna Ebrahimi, Franziska Meier, Roberto Calandra, Trevor Darrell, and Marcus Rohrbach. Adversarial continual learning. In ECCV, 2020. 6, 12
13.Sebastian Farquhar and Yarin Gal. Towards robust evaluations of continual learning. arXiv preprint arXiv:1805.09733, 2018. 1
14.Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio G ´ omez Colmenarejo, Edward ´ Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471–476, 2016. 5
15.Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu. A survey of deep learning techniques for autonomous driving. Journal of Field Robotics, 37(3):362–386, 2020. 12
16.Raia Hadsell, Dushyant Rao, Andrei A Rusu, and Razvan Pascanu. Embracing change: Continual learning in deep neural networks. Trends in cognitive sciences, 2020. 1, 4
17.Tyler L Hayes, Nathan D Cahill, and Christopher Kanan. Memory efficient experience replay for streaming learning. In ICRA, 2019. 2, 6
18.Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3
19.Zixuan Ke, Bing Liu, and Xingchang Huang. Continual learning of a mixed sequence of similar and dissimilar tasks. NeurIPS, 33, 2020. 1, 3, 5
20.Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 7
21.James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. PNAS, 114(13):3521–3526, 2017. 2, 6, 7
22.Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 6, 12
23.Dharshan Kumaran, Demis Hassabis, and James L McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in cognitive sciences, 20(7):512–534, 2016. 1
24.Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/, 1998. 12
25.Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021. 2, 3, 4
26.Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In ICML, pages 3925–3934. PMLR, 2019. 1, 3
27.Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021. 2, 3
28.Zhizhong Li and Derek Hoiem. Learning without forgetting. TPAMI, 40(12):2935–2947, 2017. 2, 6
29.Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586, 2021. 2, 3, 4
30.Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. In Conference on Robot Learning, 2017. 6, 7, 12
31.Noel Loo, Siddharth Swaroop, and Richard E Turner. Generalized variational continual learning. arXiv preprint arXiv:2011.12328, 2020. 3
32.David Lopez-Paz and Marc’Aurelio Ranzato. Gradient episodic memory for continual learning. NeurIPS, 2017. 5, 7
33.Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017. 12
34.Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. arXiv preprint arXiv:2101.10423, 2021. 1, 2, 3, 5, 6, 7, 12
35.Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In CVPR, 2018. 3
36.James L McClelland, Bruce L McNaughton, and Randall C O’Reilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995. 1
37.Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109–165. Elsevier, 1989. 1
38.Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6):1–35, 2021. 3, 12
39.Sanket Vaibhav Mehta, Darshan Patil, Sarath Chandar, and Emma Strubell. An empirical investigation of the role of pre-training in lifelong learning. ICML Workshop on Theory and Foundation of Continual Learning, 2021. 12
40.Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. In NIPS, 2011. 12
41.Aaron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. arXiv preprint arXiv:1711.00937, 2017. 5
42.German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019. 3
43.Jonas Pfeiffer, Aishwarya Kamath, Andreas Ruckl ¨ e,´ Kyunghyun Cho, and Iryna Gurevych. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247, 2020. 3
44.Quang Pham, Chenghao Liu, and Steven Hoi. Dualnet: Continual learning, fast and slow. NeurIPS, 2021. 3, 6
45.Quang Pham, Chenghao Liu, Doyen Sahoo, et al. Contextual transformation networks for online continual learning. In ICLR, 2020. 1, 3, 5
46.Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In ECCV, 2020. 5, 6, 7
47.Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 21:1–67, 2020. 2, 3
48.Dushyant Rao, Francesco Visin, Andrei Rusu, Razvan Pascanu, Yee Whye Teh, and Raia Hadsell. Continual unsupervised representation learning. NeurIPS, 32, 2019. 3
49.Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In CVPR, pages 2001–2010, 2017. 2
50.Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. 3
51.Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In ICML, pages 4548–4557. PMLR, 2018. 3
52.Murray Shanahan, Christos Kaplanis, and Jovana Mitrovic. Encoders and ensembles for task-free con- ´ tinual learning. arXiv preprint arXiv:2105.13327, 2021. 6, 7, 8, 12
53.Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, 2020. 2
54.Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proc SIGSAC conference on computer and communications security, 2015. 1, 3
55.Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv:1904.07734, 2019. 5
56.Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 2017. 4
57.Tom Veniat, Ludovic Denoyer, and Marc’Aurelio Ranzato. Efficient continual learning with modular networks and task-driven priors. arXiv preprint arXiv:2012.12631, 2020. 5
58.Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Guihong Cao, Daxin Jiang, Ming Zhou, et al. K-adapter: Infusing knowledge into pre-trained models with adapters. arXiv preprint arXiv:2002.01808, 2020. 3
59.Zifeng Wang, Tong Jian, Kaushik Chowdhury, Yanzhi Wang, Jennifer Dy, and Stratis Ioannidis. Learn-prune-share for lifelong learning. In ICDM, 2020. 3
60.Mitchell Wortsman, Vivek Ramanujan, Rosanne Liu, Aniruddha Kembhavi, Mohammad Rastegari, Jason Yosinski, and Ali Farhadi. Supermasks in superposition. NeurIPS, 33:15173–15184, 2020. 3, 6
61.Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. In CVPR, pages 374–382, 2019. 2, 6
62.Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017. 12
63.Shipeng Yan, Jiangwei Xie, and Xuming He. Der: Dynamically expandable representation for class incremental learning. In CVPR, pages 3014–3023, 2021. 3
64.Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547, 2017. 3
65.Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICML, 2017. 2
66.Chen Zeno, Itay Golan, Elad Hoffer, and Daniel Soudry. Task agnostic continual learning using online variational bayes. arXiv preprint arXiv:1803.10123, 2018. 5
67.Zizhao Zhang, Han Zhang, Long Zhao, Ting Chen, , Sercan O. Arık, and Tomas Pfister. Nested hierarchical transformer: Towards accurate, data-efficient and interpretable visual understanding. In AAAI, 2022. 5
68.Tingting Zhao, Zifeng Wang, Aria Masoomi, and Jennifer Dy. Deep bayesian unsupervised lifelong learning. Neural Networks, 2022. 3

Citation

MLA

Wang, Z., et al. “Learning to Prompt for Continual Learning”. arXiv, 2021, http://arxiv.org/abs/2112.08654v2.

APA

Wang, Z., Zhang, Z., Lee, C.-Y., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., & Pfister, T. (2021). Learning to Prompt for Continual Learning. arXiv. http://arxiv.org/abs/2112.08654v2

Chicago

Wang, Z., Z. Zhang, C.-Y. Lee, et al. 2021. “Learning to Prompt for Continual Learning”. arXiv. http://arxiv.org/abs/2112.08654v2.

Harvard

Wang, Z. et al. (2021) “Learning to Prompt for Continual Learning”, arXiv [Preprint]. Available at: http://arxiv.org/abs/2112.08654v2.

Vancouver

1. Wang Z, Zhang Z, Lee C-Y, Zhang H, Sun R, Ren X, Su G, Perot V, Dy J, Pfister T (2021) Learning to Prompt for Continual Learning. arXiv

BibTeX

@article{wang2021learning,
  title = {Learning to Prompt for Continual Learning},
  author = {Wang, Zifeng and Zhang, Zizhao and Lee, Chen-Yu and Zhang, Han and Sun, Ruoxi and Ren, Xiaoqi and Su, Guolong and Perot, Vincent and Dy, Jennifer and Pfister, Tomas},
  year = {2021},
  journal = {arXiv},
  url = {http://arxiv.org/abs/2112.08654v2},
  eprint = {2112.08654}
}

Metadata:arXiv

Source Code

This paper has an official code repository available. Click below to access the source code.

View Repository

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

Learning to Prompt for Continual Learning

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — Learning to Prompt (L2P) Framework and Prompt Pool

Knowl 2 — Instance-Wise Prompt Query and Key-Value Memory Lookup

Knowl 3 — L2P Optimization Objective

Knowl 4 — L2P Training Algorithm

Knowl 5 — Class-Incremental Performance on Split CIFAR-100 and 5-Datasets

Knowl 6 — Comparison with Architecture-Based Continual Learning Methods

Knowl 7 — Domain-Incremental and Task-Agnostic Continual Learning Benchmark Results

Knowl 8 — Ablation of L2P Core Architectural Components

Knowl 9 — Prompt Hyperparameter Sensitivity and Sharing Dynamics

Knowl 10 — Limitations of L2P

References

Citation

BibTeX

Source Code

Access the Paper