A desirable dialog system should be able to continually learn new skills without forgetting old ones, and thereby adapt to new domains or tasks in its life cycle. However, continually training a model often leads to a well-known catastrophic forgetting issue. In this paper, we present Continual Prompt Tuning, a parameter-efficient framework that not only avoids forgetting but also enables knowledge transfer between tasks. To avoid forgetting, we only learn and store a few prompt tokens' embeddings for each task while freezing the backbone pre-trained model. To achieve bi-directional knowledge transfer among tasks, we propose several techniques (continual prompt initialization, query fusion, and memory replay) to transfer knowledge from preceding tasks and a memory-guided technique to transfer knowledge from subsequent tasks. Extensive experiments demonstrate the effectiveness and efficiency of our proposed method on continual learning for dialog state tracking, compared with state-of-the-art baselines. Recently, most studies have focused on developing dialog systems for specific domains in an offline manner, assuming the data distribution stays the same. However, this is far from realistic because a deployed dialog system is often required to support new domains and provide more services constantly over time. Therefore, it is crucial for a dialog system to continually learn new tasks without forgetting old ones with high efficiency. Previous studies on continual learning (Kirk-patrick et al., 2017; Li and Hoiem, 2018) mainly focused on solving the catastrophic forgetting (CF) problem (McCloskey and Cohen, 1989): when a neural model is trained on a sequence of tasks, new tasks may interfere catastrophically with old tasks. Simply storing a model version for each task to * Corresponding author. 26C2A1 Figure 1: An illustration of Continual Prompt Tuning . We train a soft prompt for each task and freeze the pre-trained model. Several techniques are proposed to transfer knowledge from preceding tasks (green solid arrows) and subsequent tasks (red dashed arrows). mitigate forgetting is prohibitive as the number of tasks grows, especially when the model size is large. To mitigate catastrophic forgetting with low computation and storage overhead, recent methods freeze the backbone model and propose to train a weight/feature mask (Mallya et al., 2018; Geng et al., 2021) or an adapter (Madotto et al., 2021) for each task independently. However, the techniques above are still not efficient enough, and they largely ignore knowledge transfer among tasks. In this paper, we develop prompt tuning (Lester et al., 2021) for continual learning. We freeze the backbone pre-trained model and train a few prompt tokens' embeddings for each task, which is highly parameter-efficient to avoid forgetting. As illustrated by yellow components in Figure 1, we concatenate the input with a few tunable task-specific prompt tokens before feeding it to a frozen pre-trained model. Since these prompt tokens have only a small number of parameters (0.1% of the pre-trained model's parameters in our experiments), we can efficiently train and store the prompt for each task. During inference, the same pre-trained model can handle different tasks by inputting different prompts, which is friendly for deployment. Unlike the vanilla approach of training each task's prompt from scratch and fixing it afterward, we propose Continual Prompt Tuning , a framework that enables knowledge transfer between tasks. We consider transferring knowledge from both preceding tasks (forward) and subsequent tasks (back-ward). To realize forward transfer, we propose several techniques, including continual prompt initialization, query fusion, and memory replay (green solid arrows in Figure 1). To achieve positive backward transfer, we propose a memory-guided technique that uses subsequent tasks' data to update the previous tasks' prompts selectively (red dashed arrows in Figure 1). We conduct experiments on Dialog State Tracking (DST), a core component of a dialog system, using the Schema-Guided Dialog dataset (Rastogi et al., 2020). The model continually learns new services that have multiple slots to fill. We concatenate all slots' descriptions with the input and insert a sentinel token after each description, formulating DST as a masked spans recovering task, which is similar to the pre-training objective of T5 (Raffel et al., 2020). We empirically show that our proposed framework effectively outperforms state-of-the-art baselines on continual learning for DST, and is extremely efficient in terms of computation and storage. 1 To summarize, our main contributions are: 1. For the first time, we develop prompt tuning for continual learning, which avoids forgetting efficiently and is friendly for deployment. 2. We investigate several techniques for forward and backward knowledge transfer based on prompt tuning, further boosting the continual learning performance. 3. Our experiments on continual DST demonstrate the superior performance and efficiency of our proposed method. Continual Learning (CL) studies the problem of continually acquiring knowledge from a data stream and reusing it for future learning while avoiding forgetting. Three kinds of CL methods have been developed. Rehearsal methods store and replay some training samples from previous tasks (Rebuffi et al., 2017; Lopez-Paz and Ranzato, 2017). Regularization methods apply additional loss to aid knowledge consolidation (Kirkpatrick 1 Code and data are publicly available at https:// github.com/thu-coai/CPT4DST et al., 2017; Li and Hoiem, 2018). Architectural methods introduce task-specific parameters for new tasks and fix parameters for old tasks to prevent forgetting, to which our method belongs. Previous architectural methods include dynamic expanding network structure (Rusu et al., 2016), iterative network pruning and re-training (Mallya and Lazeb-nik, 2018), learning a parameter mask for each task individually (Mallya et al., 2018), etc. For continual learning in dialog system, variants of general CL methods have been applied (Lee, 2017; Shen et al., 2019; Wu et al., 2019; Mi et al., 2020; Geng et al., 2021). AdapterCL (Madotto et al., 2021) is the most related to our work, which freezes the pre-trained model and learns an adapter (Houlsby et al., 2019) for each task independently. Compared with AdapterCL, our method is more parameter-efficient, and we explore the effect of both forward and backward transfer. Recent studies have found that using a textual prompt to convert downstream tasks to the language modeling task is a more effective way to use pre-trained language models than typical fine-tuning (Brown et al., 2020; Schick and Schtze, 2021). Prompts can be manual designed (Petroni et al., 2019) or generated automatically (Shin et al., 2020; Jiang et al., 2020; Gao et al., 2021). Since searching prompts in discrete spaces is sub-optimal, some works (Qin and Eisner, 2021; Liu et al., 2021; Han et al., 2021) combine hard text prompts and soft prompts whose embeddings are learned through back-propagation. Lester et al. (2021) show that freezing the pre-trained model and only tuning soft prompts, known as prompt tuning, is parameter-efficient and becomes more competitive with fine-tuning as the model size grows. Prompt tuning differs from embedding adapter (Zhu et al., 2021) that aims to address the multilingual embedding deficiency. An embedding adapter transforms all tokens embeddings but do not affect transformer layers' computation, while prompt tuning does not change tokens embeddings but adds new tunable prompt tokens to the input, serving as context and affecting all following transformer layers. Gu et al. (2021) and Vu et al. (2021) further explore the transferability of soft prompts across tasks. While they investigate one-step adaptation, we are interested in prompt transfer in the continual learning setting. Dialog State Tracking (DST) aims to capture user goals in the form of (slot, value) pairs. Traditional ontology-based classification methods (Mrkic et al., 2017; Lee et al., 2019) require access to all candidate values. To alleviate the reliance on the ontology and improve generalization to unseen values, some work extract values from a dialog context (Xu and Hu, 2018; Gao et al., 2019) while others generate values directly to handle situations where values are missing from the context (Wu et al., 2019; Hosseini-Asl et al., 2020). Generation-based models either generate all (slot, value) pairs in one pass (Hosseini-Asl et al., 2020; Madotto et al., 2021) or generate value for each given slot separately (Wu et al., 2019). The former are more efficient but can only predict in-domain slots and lack transferability while the latter can incorporate more information about a slot as a query, such as a brief natural language description (Rastogi et al., 2020), slot type information (Lin et al., 2021), possible values (Lee et al., 2021), and the task definition and constraint (Mi et al., 2022). Our proposed method integrates multiple slot descriptions into a single query and generates all values in one pass, which improves performance without losing efficiency. The goal of continual learning is to sequentially learn a model f : X  T  Y from a stream of tasks T 1 ... TT that can predict the target y given the input x and task T k  T . We denote the data for each task T k as D k . Our method is based on pre-trained language models. Instead of fine-tuning a pre-trained model in a traditional manner (Figure 2(a)), we freeze the model but "reprogram" it to solve task T k by adding m new soft prompt tokens P k = P 1 k P 2 k ...P mk to the textual input and tuning the embeddings of P k only. Since the prompt's parameters are much less than the model's, we save P k for each task to avoid forgetting. We treat each service/API as a task in continual DST (service and task are used interchangeably). To incorporate informative slot descriptions and ease the decoding process, we convert the descriptions into a query with masked spans and formulate DST as a masked spans recovering task (Sec. 3.2). To enhance knowledge transfer between tasks, we propose continual prompt initialization, query fusion, and memory replay for forward transfer (Sec. 3.3) and explore a memory-guided technique for backward transfer (Sec. 3.4). In DST, each service T k has a set of pre-defined slots S k = { s 1 , ..., s n k } to be tracked. The input x is a dialog and the output y consists of slot-value pairs: { ( s 1 , v 1 ) , ( s 2 , v 2 ) , ..., ( s n k , v n k ) } . Similar to many NLP tasks, DST can be formulated as a text-to-text generation task. Formally, we define a function g k : X  Y  V   V  for each service T k to transform the original data ( x, y ) to:  x,  y = g k ( x, y ) (1) where V is the vocabulary and  x,  y are texts that serve as the model input and output, respectively. For example,  x can be the concatenation of x and service name, while  y is a sequence of slot-value pairs (Madotto et al., 2021) (Figure 2(a)). Previous research has shown that incorporating a natural language description d i for each slot s i is beneficial (Lin et al., 2021; Lee et al., 2021). They concatenate the dialog x with each slot description d i and decode the value v i independently. However, separately decoding is inefficient, especially when there are many slots. To solve this, we concatenate all slot descriptions and insert a sentinel token after each description to form a query added to the input, formulating DST as a masked spans recovering task that generates all slot values in one pass:  x = [ x ; Q k ; P k ] Q k =  d k 1 : (cid:104) M 1 (cid:105) . ... d kn k : (cid:104) M n k (cid:105) .   y =  (cid:104) M 1 (cid:105) v k 1 ... (cid:104) M n k (cid:105) v kn k (2) where [  ;  ] is the concatenation operation and (cid:104) M (cid:105) are distinct sentinel tokens representing masked spans. The query Q k contains all n k slot descriptions for task T k with n k masked spans and  y contains corresponding slot values leaded by the sentinel tokens. If the value of a slot can not be inferred from the input, we set it to "None". We freeze the pre-trained model's parameters  and only optimize the prompt's parameters  P k for each service T k . The loss function is: L  Pk (a) Fine-tuning takes the dialog and current service's name as input and tunes T5 to generate slot-value pairs. (b) Continual Prompt Tuning feeds the dialog, query consisting of slot descriptions and sentinel tokens, and prompt tokens to frozen T5 and tunes the prompt's embeddings to generate values for all slots in the query. Continual prompt initialization, query fusion, and memory replay are proposed to enhance forward transfer while subsequent services' data will be used for backward transfer. We show an example dialog, service name, fused query, and expected outputs. Slot names and descriptions are in italic and values are underlined. Note that the second slot description in the query belongs to another service ("banks") and is inserted by query fusion. Prompt Pre-trained Model However, when training on the current task, there is only one query that consists of the slot descriptions of that task in a fixed order, which may hinder the model from learning the general skill. Therefore, we propose to augment the query by mixing slot descriptions from the current and previous tasks to help the prompt better understand the correspondence between slot descriptions and values. We fuse the query Q k with previous tasks' queries { Q j } j<k for each sample, including three steps: 1) sample n 1 slots from S k randomly, where n 1 is sampled from [1 , |S k | ] uniformly. 2) sample n 2 slots from previous tasks' slots (cid:83) i<k S i randomly, where n 2 is sampled from [1 , n 1 ] uniformly. 3) combine the above n 1 and n 2 slots' descriptions in a random order as new Q (cid:48) k , and modify  y accordingly. Note that some original slots are dropped, and values for added slots are set to "None". Input Task 1 transform 3.3 Forward Transfer Reusing the knowledge acquired from preceding tasks often improves and accelerates the learning on future tasks. Therefore, we propose three types of techniques for forward transfer that can be employed in combination. 3.3.1 Continual Prompt Initialization An intuitive way to transfer knowledge is parameter initialization. We explore two continual prompt initialization strategies. CLInit uses last task's prompt P k  1 to initialize current task's prompt P k . SelectInit evaluates all { P j } j<k on the validation set of T k without training and selects the one with the lowest loss to initialize P k . The initial prompt of CLInit has been continually trained on all previous tasks, while SelectInit only considers the most relevant task without interference from its subsequent tasks. We empirically compare these two strategies in Sec. 5.3. We hope the model can learn to generate values according to any slot descriptions, which is a general skill that may improve performance on future tasks. Previous studies (Rebuffi et al., 2017; Lopez-Paz and Ranzato, 2017) store a few samples for each task and replay them when training on new tasks to mitigate forgetting. Since our prompt tuning framework has already resolved forgetting, we focus on how these samples benefit the current task. We assume we can store | M | samples for each task ( | M | should be small) and denote M i as the memory for task T i . When a new task T k comes, we optimize P k on D k and M <k = (cid:83) i<k M i jointly, changing the loss function to L  Pk ( D k + M <k ) . When combined with query fusion, query Q i for samples in the memory M i are also fused with queries { Q j } j  k,j (cid:54) = i from other seen tasks, including the current task. Note that in this way, samples from other tasks can be viewed as "positive" samples to those added slots in Q (cid:48) i since these samples may have not "None" values for those added slots. Although fixing P k immediately after training on task T k can avoid forgetting, it also blocks the backward knowledge transfer from future tasks. Motivated by Chaudhry et al. (2019), we explore whether it is possible to improve the performance on previous tasks with the help of memory when a new task comes. Specifically, for each previous task T i , i < k , we initialize a new prompt P ( k ) i to P i and trained it on current task's data D k with memory M i as regularization. During training, we sample a batch from D k and a batch from M i synchronously and denote the gradient from each batch as g ori and g ref , respectively. We decide the gradient for update according to the angle between g ori and g ref : g = (cid:40) g ori , if g Tori g ref > 0 0 , otherwise (4) which means we abort the update that will increase the loss on memory batch. We empirically find that this simple abortion is better than projecting g ori onto the normal plane of g ref (Chaudhry et al., 2019). After training, we update P i to P ( k ) i if P ( k ) i obtains lower loss and better (or equal) performance on M i than P i . Recently, Madotto et al. (2021) proposed a continual learning benchmark for task-oriented dialog systems and compared several classic CL methods. We adapt their data processing steps and baselines in our experiments. We conduct experiments on Schema-Guided Dialog dataset (SGD) (Rastogi et al., 2020) that has 44 services over 19 domains. It also provides a one-sentence description for each slot. We treat each service as a task and only consider dialogs involving a single service. We randomly split a service's dialogs into train/val/test sets at the ratio of 7:1:2. The number of training samples of each service ranges from 112 to 4.7K, and there are 2 to 10 slots for one service. More details about data statistics can be found in the Appendix (Table 8). We evaluate DST performance using the widely adopted Joint Goal Accuracy (JGA) (Wu et al., 2019), which requires all slots' values are correctly predicted. We assign the target service during testing to avoid ambiguity since the same dialog can be parsed differently under different services. We denote a j,i as the JGA on the test set of task T i right after training on task T j . We evaluate the CL performance as the average JGA on all tasks after training on the final task TT : Avg . (6) FWT is the averaged zero-shot performance on new tasks, evaluating a model's generalization ability. BWT assesses the impact that learning on subsequent tasks has on a previous task. Negative BWT indicates that the model has forgotten some previously acquired knowledge. We adopt the following models from Madotto et al. (2021) as baselines:  Fine-tuning : Fine-tune the model on new task data continually. Replay : Save | M | samples randomly sampled from the training set of each task T i to memory M i and jointly train the model on new task data D k and memory M <k . EWC : Maintain the memory in the same way as Replay but use it to compute the Fisher information matrix for regularization (Kirkpatrick et al., 2017). AdapterCL : Freeze the pre-trained model and train a residual Adapter (Houlsby et al., 2019) for each task independently (Madotto et al., 2021). Above methods use the same input and output format as in Figure Prompt tuning based methods including our proposed Continual Prompt Tuning are list below:  Prompt Tuning : Formulate DST as a masked spans recovering task (Sec. 3.2) and only tune the prompt for each task independently. Multi-task Prompt Tuning : Prompt Tuning in a multi-task manner instead of CL. Train a single prompt using all tasks' data concurrently. Continual Prompt Tuning : Prompt Tuning with CLInit (Sec. 3.3.1) and query fusion (Sec. 3.3.2). w/ memory with memory replay (Sec. 3.3.3). w/ memory & backward with memory replay and memory-guided backward transfer (Sec. 3.4). We use the following setting in the experiments unless otherwise specified. Training task sequences Since a sequence of all (44) tasks is too long for the evaluation purpose, we conduct most of the experiments on 15 tasks chosen at random to save computing resources. We run AdapterCL , Prompt Tuning , and Multi-task Prompt Tuning 5 times with different random seeds because they are agnostic to task order. The FWT and BWT metrics for these models are left blank. We run other methods in the same 5 task orders created by random permutation. The selected tasks and ordering are listed in the Appendix (Table 9). Hyper-parameters We use T5-small as the backbone model and reuse its sentinel tokens (Raffel et al., 2020). For each task, Continual Prompt Tuning first trains 10 epochs with fused query (and using memory if available) for forward transfer. Afterward, it concentrates on the current task and continues training 10 epochs on the original data of the current task. When using backward transfer, we train 5 epochs for each previous task. Other methods train 20 epochs for each task. We use AdamW and set the learning rate to 3e-5 for Fine-tuning , Replay , and EWC , 3e-3 for AdapterCL , and 0.5 for all prompt tuning based methods. We set the batch size to 16 for prompt tuning based methods and 8 for other methods. To avoid overfitting, we perform early stopping if validation performance does not improve for 5 consecutive epochs. The weight for EWC regularization loss is 0.01. We set the memory size | M | to 50 for each task and save the same samples for all methods that require memory. We initialize prompt tokens with the tokens randomly drawn from the vocabulary. For prompt tuning based methods, we tune 100 soft prompt tokens with the embedding size 512 for each task, resulting in 51.2K parameters. To compare parameter efficiency, we adjust AdapterCL 's parameters for each task to be nearly 1x or 20x as ours. The experiments are organized as follows. We compare our method with baselines in Sec. 5.1, and present a comprehensive ablation study in Sec. 5.2. We investigate the effect of prompt initialization in Sec. 5.3, and the effect of model size and prompt length in Sec. 5.4. Computation Resource Analysis. In CL, there is a trade-off between performance and computation resources. Ideally, we hope to utilize the least amount of computation resources to achieve the best performance. We take three vital resources into our consideration. Memory saves previous tasks' samples, which may involve privacy issue and requires extra storage. Additional parameters are the extra parameters we add to our model to cope with different tasks along the CL process, which should be kept to a minimum in order to scale to long task sequences. Tunable parameters are the trainable parameters when we learn a task, which is important for GPU memory and computation. We show the usage of these resources in Table 1 (right). Replay stores | M | samples for each task and does not need extra parameters. EWC saves the Fisher information matrix and original parameters, requiring two times additional parameters. AdapterCL , Prompt Tuning , and Continual Prompt Tuning require no memory and only add a small number (2% or 0.1%) of additional parameters for each task, largely reducing the computational and storage overhead. Apart from the vanilla form, Continual Prompt Tuning can also utilize the memory if available. Consistent with Madotto et al. (2021), both Fine-tuning and EWC suffer from catastrophic forgetting while replaying memory can alleviate the problem to a large extend. Fine-tuning and EWC have a low Avg. JGA because of the large negative BWT, while Replay improves BWT a lot thus has a high Avg. JGA. Our proposed Prompt Tuning with masked spans recovering is more parameter efficient than AdapterCL . In terms of Avg. JGA, Prompt Tuning is much better than AdapterCL with the same size and comparable to AdapterCL with 20x parameters. Forward transfer through CLInit and query fusion is effective for Prompt Tuning . Continual Prompt Tuning improves over Prompt Tuning significantly and outperforms baselines. When memory is available, our method achieves the best results w.r.t. all metrics, closing the gap between CL and multi-task learning. Memory improves zero-shot performance (FWT) on new tasks as Replay is better than Fine-tuning and Continual Prompt Tuning w/ memory is better than without memory. Our memory-guided backward transfer effectively utilizes subsequent tasks to help previous tasks. Although minor, Continual Prompt Tuning w/ memory & backward is the only method that exhibits positive BWT. To understand the effect of different proposed techniques, we conduct an in-depth ablation study and show the result in Table 2. Row 1 and 2 do not formulate DST as a masked spans recovering (MSR) task: the input is the concatenate of the dialog, service name, and soft prompt, while the output is a sequence of slot-value pairs as in Fine-tuning (Fig-ure 2(a)). Several interesting observations can be noted: First , formulating DST as MSR is benefi-MSR CLInit QF MR Avg. cial. Using MSR achieves better CL performance regardless of learning each task independently (row 3 v.s. row 1) or continually using CLInit (row 4 v.s. row 2). Besides, MSR formulation improves zero-shot generalization on new tasks (row 4 v.s. row 2). Second , forward transfer through CLInit brings large improvement for CL. CLInit outperforms random initialization greatly for both using MSR formulation (row 4 v.s. 3) and not (row 2 v.s. 1). Third , both query fusion and memory replay are effective. When they are used separately, memory replay (row 6) boosts the performance more than query fusion (row 5), while applying them altogether achieves the best performance (row 7). In this experiment (Table 3), we compare CLInit with other prompt initialization strategies for Prompt Tuning in CL. SelectInit (see Sec. 3.3.1) Initialization Avg. selects the prompt that has the best zero-shot performance on the current task from all previous tasks' prompts for initialization. We could see that both SelectInit and CLInit outperform random initialization significantly, demonstrating the effectiveness of transferring knowledge from previous tasks through prompt initialization. CLInit is slightly better than SelectInit in both Avg. JGA and zero-shot generalization (FWT), which reveals the benefit of accumulating knowledge from all seen tasks. In contrast, the prompt initialized by SelectInit has seen fewer tasks and thus contains less knowledge, which might explain the slightly worse result. Based on the observation above, we further study that whether seeing more preceding tasks further helps CLInit. To this end, we choose a task order of all 44 tasks at random (see Table 8 in the Appendix) and perform Prompt Tuning with CLInit on the last 5, last 15, last 30, and all 44 tasks separately. Formally, we train on four CL curriculums T 40:44 , T 30:44 , T 15:44 , and T 1:44 , which have the same ending. We calculate the Avg. JGA on the T 40:44 , T 30:44 , and T 15:44 if possible. As illustrated in Table 4, performance on the same tasks (in the same column) increases monotonously as the number of preceding tasks grows. This pattern validates that the benefit of CLInit becomes more evident as the number of tasks increases. This finding suggests that our method is suitable for long task sequences. In this experiment, we analyze the influence of pre-trained model size and prompt length. We vary the pre-trained model in {T5-small, T5-base, T5-large} and prompt length in {1, 5, 20, 100, 150} for Continual Prompt Tuning on the 15 tasks (the task order is in Table 9 in the Appendix). Figure 3 shows Avg. JGA and Table 5 shows FWT. We can observe that: First , when fixing the prompt length, increasing the model size improves the Avg. JGA as well as the generalization ability measured by FWT in most cases. Second , when the backbone model size is fixed, increasing the prompt length improves the overall performance in general. Furthermore, we found that increasing prompt token length from 20 to 100 improves Avg. JGA and FWT more than increasing it from 100 to 150, which is consistent with the finding in Lester et al. (2021). Third , our method becomes more parameter-efficient as the backbone model size grows. With the same number of tunable parameters (x-axis), using a larger pre-trained model achieves better Avg. JGA. In this section, we compare the role of memory in Replay and our method. We vary the memory size per task | M | in {10, 50, 100} and show the performance of Replay and Continual Prompt Tuning with memory replay (and memory-guided backward transfer) in Table 6. We can find that increasing the memory size benefits Replay significantly. This is not surprising because Replay and other rehearsal methods rely on memory to solve the challenging forgetting problem. When the memory size is unlimited, Replay degenerates to multi-task learning, which is powerful but costly in storage and computation. For Continual Prompt Tuning , however, the memory is not used for retaining the performance on previous tasks since parameters for previous tasks are saved. In forward transfer, the memory helps recall previous tasks' knowledge and serves as a complement to CLInit and query fusion. The influence on Avg. JGA depends on the effect of transfer learning on the current task via multi-task training ( L  Pk ( D k + M <k ) ). As shown in the row 2 in Table 6, increasing the memory size does not improve Avg. JGA significantly and may even distract the model from learning the current domain. This result suggests that our method does not need a large memory for forward transfer. In backward transfer, the memory gives reference gradients to guide the updates and serves as a filter to decide whether to accept the updates. Thus larger memory gives more accurate guidance. From the bottom row in Table 6, we can find that increasing memory size can improve the effect of backward transfer. We also conduct experiments using a percentage memory budget, setting the memory size for each task proportional to task data size: | M i |  | D i | . This means low-resource tasks have fewer samples Memory Size fixed = 50 proportional Replay 58.6 3 . stored in the memory than in the original setting. We set the total memory size to 50 * T, where T is the number of tasks. As shown in Table 7, Replay performs much worse (58.6  55.8) in the unbalanced task memory setting while the effect on Continual Prompt Tuning w/ mem. is slight (60.7  60.3). Besides, our proposed backward transfer technique is still effective. Overall, these results indicate that compared with Replay , our method uses the memory differently and benefits less from enlarging the memory. In this paper, we develop prompt tuning for continual learning for the first time. We propose Continual Prompt Tuning , a highly parameter-efficient framework that avoids forgetting and enables for-ward/backward knowledge transfer among tasks. For forward transfer, we explore continual prompt initialization, query fusion, and memory replay techniques. For backward transfer, we devise a memory-guided technique. Extensive experiments on continual learning for DST demonstrate the effectiveness and efficiency of our proposed method compared with state-of-the-art baselines. Our method and findings will foster more future studies towards building more scalable, adaptable task-oriented dialog systems. This work was supported by the National Science Foundation for Distinguished Young Scholars (with No. 62125604) and the NSFC projects (Key project with No. 61936010 and regular project with No. 61876096). This work was also supported by the Guoqiang Institute of Tsinghua University, with Grant No. 2019GQG1 and 2020GQG0005, and sponsored by Tsinghua-Toyota Joint Research Fund.
abstain abstain method method objective objective abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method method abstain objective method objective objective method abstain abstain objective objective objective objective other other other other objective other other method objective other other other other other other other objective other other other other other objective abstain abstain method method abstain method method method method method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain method abstain method abstain abstain abstain method method abstain method method method abstain abstain abstain method method method method method method abstain method method abstain method method abstain abstain method method method method abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method method method abstain method abstain method abstain abstain method abstain method method method abstain method method method method abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain result objective objective objective abstain objective method other other