Building models of natural language processing (NLP) is challenging in low-resource scenarios where only limited data are available. Optimization-based meta-learning algorithms achieve promising results in low-resource scenarios by adapting a well-generalized model initialization to handle new tasks. Nonetheless, these approaches suffer from the memorization overfitting issue, where the model tends to memorize the meta-training tasks while ignoring support sets when adapting to new tasks. To address this issue, we propose a memory imitation meta-learning (MemIML) method that enhances the model's reliance on support sets for task adaptation. Specifically, we introduce a task-specific memory module to store support set information and construct an imitation module to force query sets to imitate the behaviors of some representative support-set samples stored in the memory. A theoretical analysis is provided to prove the effectiveness of our method, and empirical results also demonstrate that our method outperforms competitive baselines on both text classification and generation tasks. Building natural language processing (NLP) models in low-resource scenarios is of great importance in practical applications because labeled data are scarce. Meta-learning-based methods (Thrun and Pratt, 2012) have been commonly used in such scenarios owing to their fast adaptation ability. Notable successes have been achieved by meta-learning on low-resource NLP tasks, such as multi-domain sentiment classification (Yu et al., 2018; Geng et al., 2019) and personalized dialogue generation (Madotto et al., 2019; Song et al., 2020; Zheng et al., 2020). proaches have been widely used in various low-resource NLP scenarios (Madotto et al., 2019; Qian and Yu, 2019; Li et al., 2020; Mi et al., 2019) because they are model-agnostic and easily applicable. Concretely, optimization-based meta-learning algorithms aim to learn a well-generalized global model initialization  that can quickly adapt to new tasks within a few steps of gradient updates. In the meta-training process, we first train  on a support set (i.e., a few training samples of a new task i ) to obtain task-specific parameters  (cid:48) i . Then, we optimize  based on the performance of  (cid:48) i on a query set (i.e., another set of samples in task i ). Despite its effectiveness, optimization-based meta-learning algorithms usually suffer from the memorization overfitting issue 1 (Yin et al., 2020; Rajendran et al., 2020), where the learned model tends to solve all the meta-training tasks by memorization, rather than learning how to quickly adapt from one task to another via support sets. This is acceptable for training process, but results in poor generalization on the meta-testing sets, because the memorized model does not have knowledge of those tasks and does not know how to utilize the base learner to learn new tasks. Hence, this issue hinders the model from capturing task-specific characteristics from support sets and thus prevents the model from adapting to distinct new tasks (Ra-jendran et al., 2020). For instance, in personalized dialogue generation, this implies that the dialog model cannot adapt to individual users based on short conversation histories and hence fails to generate personalized responses. Several works have been proposed to tackle the memorization overfitting issue for regression and image classification tasks. Some studies try to explicitly regularize the model parameters (Yin et al., 1 Memorization overfitting is different from the overfitting in conventional supervised learning (Hawkins, 2004). The latter means that the model overfits to the training tasks and fails to generalize to the testing tasks. 2020; Rajendran et al., 2020), but this restricts the complexity of model initialization and reduces the model capacity. Another line of research integrates samples from support sets into the corresponding query sets via data augmentation (Yao et al., 2021). However, data augmentation on textual data may result in noisy labels or distribution shifts, which impairs the model performance (Chen et al., 2021). In this paper, we address the memorization overfitting issue by enhancing the model's dependence on support sets when learning the model initialization, which forces the model to better leverage information from support sets. As an analogy, consider a young investor who has the ability to adapt to new circumstances rapidly but little memory of learned experiences, and an old investor who is experienced but refuses to be flexible. Our idea is to make the young investor adaptive to the various situations when he assesses his benefits so that he can not only take advantage of the old one's experience but also learn from the old investor how to leverage the learned experience. In this paper, the young investor stands for a standard meta-learning algorithm (e.g., MAML), which is prone to memorization overfitting, and the old investor is a memory module we integrate into the method, carrying information of support sets. Specifically, we propose a Memory-Imitation Meta-Learning (MemIML) method that forces query set predictions to depend on their corresponding support sets by dynamically imitating behaviors of the latter. We therefore, introduce a memory module and an imitation module to enhance such dependence. The memory module is task-specific, storing representative information of support sets. The imitation module assists in predicting samples of query sets by dynamically imitating the memory construction. In this way, the model has to access the support set by memory imitation each time it makes a prediction on a query-set sample, hence it's no longer feasible for the model to memorize all meta tasks. The contributions of this work are: 1. A novel method MemIML is proposed to alleviate the memorization overfitting for optimization-based meta-learning algorithms. It encourages the utilization of support sets with the help of a memory module and an imitation module when adapting to new tasks. 2. Comprehensive experiments on text classification and generation tasks show that MemIML significantly outperforms competitive baselines. Meta-Learning. Meta-Learning aims to improve the learning algorithm itself based on the previously learned experience (Thrun and Pratt, 1998; Hospedales et al., 2021). In general, there are three categories of meta-learning methods: model-based methods, (Santoro et al., 2016; Obamuyide et al., 2019) which depend on the particular model design to facilitate fast learning; metric-based methods, (Vinyals et al., 2016; Snell et al., 2017; Geng et al., 2019) which encode samples into an embedding space and classify them based on the learned distance metric; optimization-based methods (Finn et al., 2017; Mi et al., 2019) that learn a well-generalized model initialization which allows for fast adaptation to new tasks. For low-resource scenarios in NLP, optimization-based meta-learning methods achieved promising results on tasks such as personalized dialog generation (Madotto et al., 2019; Song et al., 2020; Tian et al., 2021), low-resource machine translation (Gu et al., 2018; Sharaf et al., 2020) and question answering (Yan et al., 2020), few-shot slot tagging (Wang et al., 2021), and so on. Memorization overfitting of Meta-learning. Meta-learning algorithms suffer from memorization overfitting. Yin et al. (2020) build an information bottleneck to the model, while this approach decreases the model performance with this passive regularization. Rajendran et al. (2020) inject random noise to the ground truth of both support and query sets, while little extra knowledge is introduced to learn a good initialization. Yao et al. (2021) address overfitting issues by augmenting meta-training tasks through mixing up support and query sets. However, such augmentation for text needs to be based on the assumption of keeping the label and the data distribution unchanged, which is often not true in practice (Chen et al., 2021). Instead of regularization and data augmentation, we leverage the support sets information stored in the memory to augment the meta-learning. External Memory for Few-shot Learning. Memory mechanism has proven to be powerful for few-shot learning (Geng et al., 2019; Santoro et al., 2016; Munkhdalai et al., 2019). Current methods 584 either refine representations stored in the memory (Ramalho and Garnelo, 2018) or refining parameters using the memory (Munkhdalai and Yu, 2017; Cai et al., 2018; Wang et al., 2020). In the NLP domain, some methods store encoded contextual information into a memory (Kaiser et al., 2017; Holla et al., 2020; Zheng et al., 2019). Geng et al. (2019) propose a memory induction module with a dynamic routing algorithm for few-shot text classification tasks. Munkhdalai et al. (2019) augment the model with an external memory by learning a neural memory. Wang et al. (2021) reuse learned features stored in the memory on the few-shot slot tagging. We first formulate model-agnostic meta-learning (MAML) (Finn et al., 2017). Specifically, denote the base model used in MAML as f  and assume each task T i sampled from a task distribution p ( T ) associates with a dataset D i . Each dataset D i consists of a support set D si = { ( X sj , Y sj ) } N s j =1 and a query set D qi = { ( X qj , Y qj ) } N q j =1 , where X and Y denote the input and ground truth of a sample, respectively. During the meta-training stage, a task-specific (a.k.a., post-update) model f  (cid:48) i is first obtained for each task T i via gradient descent over its support set D si . Then MAML updates its initialization (a.k.a., pre-update)  according to the performance of f  (cid:48) i on the query set D qi as in Eq.1:   = min  ET i  p ( T ) (cid:104) L (cid:16) f  (cid:48) i ( X qi ) , Y qi (cid:17)(cid:105) (1) s.t.  (cid:48) i =      L ( f  ( X si ) , Y si ) (2) where  is the inner loop learning rate. During the meta-testing stage, the learned initialization   is fine-tuned on the support set D st for task T t , and the resulting model is evaluated on the query set D qt with the post-update parameters  (cid:48) t . To alleviate the memorization overfitting issue in meta-learning, we propose MemIML, which includes a memory module and an imitation module on the grounds of a base model. The memory module is task-specific, recording the mapping behaviors between inputs and outputs of support sets for each task. The imitation module is shared across tasks and predicts values for each query-set sample by dynamically imitating the memory construction. The acquired support set information leveraged by the imitation module augments the model initialization learning, enhancing the dependence of the model's task adaptation on support sets. Fig. 1 shows our model architecture. We design a memory module M i for each task T i and incorporate it in the MAML framework. In order to fully leverage information from support sets, we construct key-value pairs from support-set samples and store them in the memory module. The key is the sentence representation of a sample input from support sets obtained from an introduced key network. The corresponding value is constructed to store the information of the sample output (ground truth) as in Sec. 4.3: in NLG tasks, the value is the sentence embedding of the output sentence; in NLU tasks, the value is the one hot embedding of the class label (a scalar) of the sample. Our memory has two operations: memory writing that constructs the memory and memory reading that acquires information from memory. In the following, we elaborate on these contents in detail. Key Network represents a sample with a vector. Specifically, we use a frozen pre-trained BERT model (Devlin et al., 2019) as the key network. The input of the key network is the sample input sentence X sj  D si ( X qj  D qi ), and the output is the encoded representation of the first token (i.e. [CLS] token) of the sentence. The acquired representation is regarded as the key K sj for X sj ( K qj for X qj ). Memory Writing constructs the memory using the information of samples in the support set D si . For each task T i , the task-specific memory M i consists of N i memory slots (i.e. key-value pairs { K sl , V sl } N i l =1 ). To build these memory slots, we select samples from support sets and write their information into the memory. The sample selection is according to a diversity-based selection criterion (Xie et al., 2015) to ensure the diversity and representativeness of the memory content. The detailed description of this criterion is in Appendix. D. For each task-specific memory module M i , we adopt the diversity score as S ( M i ) on the stored keys. Here, a more diverse memory gets a higher diversity score. When the memory is not full, we directly write support-set samples without selection; otherwise, we compute the diversity score of the current memory and scores after every old key-value pair is replaced with a new key-value pair. Then we replace the old pair with the new one 585 N e i ghbo r s Memory Local Adaptation 1 5 6 2 3 4 Reading Output Global optimization of Local adaptation of Locally Adapted Value Predictor Query Set Support Set Write Inner Loop for Each Task Outer Loop Read MAML Optimization Memory Module Imitation Module Predicted value Value Support set information Query set information Inner loop Outer loop Key Global Value Predictor Local Adaptation Figure 1: The architecture of our model, MemIML. where the replacement can maximize the diversity score. In this way, the memory we build can carry more distinguishable and representative information and efficiently utilize the storage space. Memory Reading obtains information from memory to enhance the meta-learning. The input is the sentence representation of the sample in query sets encoded by the key network, and the output is the memory slots similar to the query sample. Specifically, given the key representation K qj of a sample X qj  D qi , we retrieve the top N most similar slots from its task-specific memory M i . The similarity is measured based on the Euclidean distance between K qj and each key K sl in the memory slots. The retrieved key-value pairs { K sl , V sl } Nl =1 act as the output of memory reading. In order to better leverage the retrieved memory and enhance the dependence of our model on support sets, we propose an imitation module to encourage the imitation of support sets behaviors when making predictions on query sets. For each sample X qj in the query set, the inputs of the imitation module are the key K qj and its retrieved N memory slots, and the output is the predicted value  V qj for X qj . To achieve the imitation, we construct a value predictor that can model the behaviors of support-set samples (i.e. key-value matching) stored in the memory. For estimating the value of each query-set sample, we conduct local adaptation on the value predictor to adapt the matching. In this way, the proposed imitation module is customized for each query-set sample, which facilitates better capture of specific task information than directly using the memory reading output, especially when tasks are versatile. The reason is that the similarity measurement of previous memory reading operations is based on the fixed BERT representations, which ignores the task-specific information. In MemIML, the proposed value predictor aims to build a mapping from keys to values of the memory module mentioned in Sec. 4.1. The input of the value predictor is a key obtained from the key network, and the output is the associated value. Specifically, we use a two-layer fully-connected network g  with parameters  to build the mapping. The value predictor is learned over constructed key-value pairs of support sets across all tasks. Given the key K qj of a query-set sample input X qj , we can then estimate its associated value as  V qj . To train the value predictor, we minimize the recon-struction loss L rec (  V , V ) to make the predicted values as close as possible to values constructed from the ground truths of support-set samples, where L rec is the cross-entropy loss if the value V is a label and is the mean square loss if V is a vector. The training procedure includes the global optimization shared across tasks and the local adaptation for each specific task. Specifically, we first train the value predictor with samples from support sets of all tasks. After feeding the memory reading output of a query-set sample to this network, we perform local adaptation and employ the adapted network to estimate the value for the query sample. Global Optimization. To obtain the task-independent global parameters  , we train the value predictor over constructed keys (i.e., as inputs) and values (i.e., as outputs) from support-set samples of all tasks. The global optimization keeps updating in the whole meta-training phase. Local Adaptation. To make the value predictor adaptive to each query-set sample X qj , inspired by (Sprechmann et al., 2018), we propose local adaptation that fine-tunes the global value predictor g  to get an adapted one with parameters  qj . The local adaptation only works when predicting X qj . Based on the initial parameters  from the global optimization, we perform several gradient descent steps to minimize the loss L loc , which is: L loc =  (cid:107)     (cid:107) 22 + 1 NN (cid:88) l =1 L rec   (  V sl , V sl ) (3) Here,  V sl = g   ( K sl ) , { K sl , V sl } Nl =1 is the memory reading output of the query-set sample, and the factor  restricts the distance between  qj and  . Minimizing the second term encourages g  qj to better estimate the retrieved memory values { V sl } Nl =1 . Then we can acquire the locally adapted value prediction network g  qj with parameters  qj = arg min   L loc (  ) . Given a query-sample key K qj , we can thus predict its associated value as  V qj = g  qj ( K qj ) , (4) where the adapted parameters  qj are discarded thereafter, and the model does not back-propagate through  V qj . In this sense, besides the task-specific parameter  (cid:48) i provided by MAML, there will also be  qj learned from support sets specific to each query-set sample. This guarantees that the model relies more on support sets for task adaptation. Fig. 1 (right part) illustrates the mechanism of local adaptation. In this part, we will elaborate on two few-shot applications in NLP (i.e., text generation and text classification) to solve the memorization overfitting problem of MAML. The model structures of these applications are basically the same, except for the following three points: the base model, the way to get the value V sl stored in the memory module, and the way to leverage the output  V qj of Sec. 4.2. Personalized Dialogue Generation. The base model is the transformer (Vaswani et al., 2017) consisting of an encoder and a decoder. In this task, each sample consists of an input utterance and a ground truth utterance, so the value V sl stored in the memory is obtained from the ground truth utterance Y sl of a support-set sample, which is embedded by the key network followed by an LSTM (Hochreiter and Schmidhuber, 1997). This LSTM is optimized with the base model. The  V qj , concatenated with the encoder outputs, serves as a new input for the decoder. Hence, we acquire the prediction of a query-set sample via  Y qj = Decoder ([  V qj ; Encoder ( X qj )]) . Multi-domain Sentiment Classification. The base model is a BERT (Devlin et al., 2019) followed by a fully-connected network. Each sample consists of an input sentence and a sentiment label (ground truth), so the memory value V s l is the sentiment label. To leverage  V qj , we interpolate it with the original output of the base model  Y qj as  Y qj =   Y qj + (1   )  V qj (5) where  balances  Y qj and  V qj . Notice that the interpolation not only works on the prediction output but also guides the training via gradient descent based on the interpolated output. We verify the effectiveness of the interpolation in Appendix. C. Algorithm 1 Memory Imitation Meta-training Require: p ( T ) : task distribution,  1  4 : step sizes 1: Initialize  from pretrained model; initialize  randomly; initialize memory for T tasks as { M i } Ti =1 = {  } Tj =1 2: while not converge do 3: Sample batch of tasks {T i } ni =1 , where T i  p ( T ) 4: for all task T i do 5: Sample support set D si and query set D qi from T i 6: Obtain the keys { K s l } N s l =1 and the values { V s l } N s l =1 for the support set D si as in Sec. 4.1 7: M i  { < K sl , V sl > } N s l =1 # Write memory 8:      1   L rec # Global optimization 9:  (cid:48) i     2   L base # Learn  (cid:48) i in Eq. We theoretically investigate how our method helps to alleviate the memorization overfitting problem. Following Yin et al. (2020), we use mutual information I (  Y qi ; D si | , X qi ) to measure the level of the memorization overfitting. When the learned model ignores support sets to predict query sets, I (  Y qi ; D si ) | , X qi ) = 0 occurs, which indicates the complete memorization overfitting in meta-learning (Yin et al., 2020). Hence, lower mutual information means more serious memorization overfitting issues. We propose a criterion similar to (Yao et al., 2021) to measure the validity of our method for tackling this problem. For a task T i = { D si , D qi } , the criterion aims to mitigate the memorization overfitting by enhancing the model's dependence on the support set D si , i.e. increasing the mutual information between support set and  Y qi as follows: I (  Y qi ;[ D si , M i ] | , X qi ) > I (  Y qi ; D si | , X qi ) , (6) where M i means additional memory information we provide, which contains support sets information to augment the inference of the sample X qi in D qi . We demonstrate our method MemIML meets the above criterion (See details in Appendix. A.). In the meta-training phase (shown in Alg. 1), MemIML first constructs an empty memory for each task and then follows the bi-level optimization process of MAML. In the inner loop, MemIML adapts the base model initialization  to task-specific parameters via training on the support set. At the same time, from each support-set sample, MemIML obtains a key-value pair and determines whether to write it into the memory or not. Then, MemIML conducts the global optimization of the value predictor over these key-value pairs. In the outer loop, each sample of the query set reads the memory to retrieve the most similar memory slots. Local adaptation fine-tunes the value predictor on those retrieved slots. Next, the adapted value predictor estimates the value of each query sample and uses it to augment the learning of the model initialization. The total loss function in the inner loop is L total = L base + L rec , where L base = L ( f ( X s ) , Y s ) is the cross-entropy loss. The procedure of meta-training and meta-testing are almost the same except that meta-testing does not optimize the learned model initialization  and the initial parameter  of the value predictor. For each task T t in the meta-testing phase, MemIML also adapts  to task-specific parameters  (cid:48) i in the inner-loop and constructs the task-specific memory. In the outer-loop, MemIML retrieves key-value pairs from the memory to conduct local adaptation based on the initial parameter  . The estimated value  V qt from local adaptation helps the base model to infer the final output  Y qt . Experiments on personalized dialogue generation and multi-domain sentiment classification verify our model on text generation and classification, respectively, where we use Persona-Chat and ARSC datasets. Dataset. Following (Zhang et al., 2018), we use Persona-chat (Madotto et al., 2019) by regarding building a dialog model for each person as a task. The dataset consists of a training/validation/testing set with 1137/99/100 persons (tasks) separately. In the Persona-Chat dataset, each persona description has 8.3 unique dialogues on average, and each task consists of three samples. Baselines. We compare our methods with the following baselines: Base Model : We pretrain a conventional transformer-based dialog generation 588 Type Methods Accuracy Non meta-learning Fine-tune 80.73 Matching Net 81.22 Metric-based Prototypical Net 80.13 Proto ++ 82.41 meta-learning Relation Net 81.32 Induction Net 79.31 MAML 82.17 Optimization-based MR-MAML 78.14 Meta-Aug 83.57 meta-learning MetaMix 83.63 MemIML (Ours) 85.69* Table 2: The results of mean accuracy over the ARSC. model over all the training tasks ignoring the speak-ers' personality. Fine-tune : We fine-tune the pretrained base model on the support sets of each meta-testing task. MAML : We apply MAML (Madotto et al., 2019) to the base model. MR-MAML : Yin et al. (2020) tackle the memorization overfitting of MAML via regularization. Metrics. Automatic evaluation has three aspects,  Quality : BLEU-n (Papineni et al., 2002), CIDEr (Vedantam et al., 2015), and ROUGE (Lin, 2004) measures the n-gram matching between the generated response and ground truth. PPL (perplex-ity) measures the sentence fluency. Diversity. Dist-n (Li et al., 2016) evaluates the response diversity by counting unique n-grams. Consistency : C score (Madotto et al., 2019) measures the consistency between the generated responses and persona descriptions through a pretrained natural language inference model. Overall Performance. As shown in Table 1. Fine-tune outperforms Base Model in all metrics, which verifies that the task-specific data is helpful to its performance on specific tasks. Compared to Fine-tune , MAML behaves better on diversity and consistency but behaves worse on quality. Pretraining the base model achieves the best perplexity (lowest PPL) as shown by Base Model and Fine-tune . We analyze that it's because pretraining leads to a considerable degree of fluency in their generated utterances and is careless about each task's specific information, resulting in low consistency with tasks. Our model, MemIML , performs the best in most aspects, including quality, diversity, and task consistency. In particular, MemIML significantly improves MR-MAML in alleviating the memorization overfitting issue, suggesting that memory imitation is more effective than only regularizing model initialization. Dataset. Amazon Review sentiment classification dataset (ARSC) (Yu et al., 2018) contains 69 tasks in total. Following (Geng et al., 2019), we build a 2-way 5-shot meta-learning with 57 tasks for meta-training and 12 tasks for meta-testing. We conduct experiments on the ARSC (Yu et al., 2018). It contains English reviews of 23 types of Amazon products, where each product consists of three different binary classification tasks. Following Geng et al. (2019), we select 12 tasks from 4 domains ( Books, DVD, Electronics, Kitchen ) for meta-testing tasks, and the support sets of these tasks are fixed (Yu et al., 2018). Baselines. We compare our methods with the following baselines: Fine-tune : We fine-tune a pre-trained BERT on the support set of meta-testing tasks (non-meta-learning method) as in Appendix. B.2. We choose five metric-based meta-learning baselines: Matching Net (Vinyals et al., 2016), Prototypical Net (Snell et al., 2017), Proto ++ , (Ren et al., 2018), Relation Net (Sung et al., 2018), and Induction Net (Geng et al., 2019). We apply an optimization-based baseline ( MAML ) (Finn et al., 2017) to the base model, and implement some approaches tackling the memorization overfitting problem based on MAML: MR-MAML (Yin et al., 2020), MetaMix , (Yao et al., 2021) and Meta-Aug (Rajendran et al., 2020). Overall Performance. Table 2 shows the performance measured by the mean accuracy of meta-testing tasks. Our model, MemIML outperforms all competing approaches including non-meta-learning, metric-based meta-learning, and optimization-based meta-learning methods. Particularly, our model surpasses the current solutions to the memorization overfitting problem ( MR-MAML, Meta-Aug, MetaMix ), indicating that our method is more effective compared to regularization and textual augmentation. In Figure 2, the gaps of the losses on query sets between pre-update  (before training on support sets) and post-update  (cid:48) i (after training on support sets) indicate the memorization overfitting problem. The gap between sky-blue and blue curves measures the memorization overfitting of meta-training (the gap between pink and red curves measures meta-testing). Small loss gaps indicate a severe memorization overfitting where support sets are almost useless for task adaptation. Those loss gaps between  and  (cid:48) i collapse in MAML and MR-MAML after about 3000 steps. This indicates that the post-update  (cid:48) i barely benefits from the support set, and thus the memorization overfitting issue is severe. In Figure 2 (c), MemIML has large gaps between  and  (cid:48) i , implying that  (cid:48) i better leverages support sets when adapting to new tasks and thus alleviates the memorization overfitting issue. In Table 3, we conduct ablation studies to verify the effectiveness of each component. Removing Similarity-Search means the memory reading operation randomly outputs memory slots instead of searching for similar memory slots. This variant underperforms MemIML, indicating that similar samples stored in the memory provide more useful information to improve the model performance. Removing the value predictor means directly using the memory output without a learnable network. Its results are not too bad, indicating that the memory module helps to mitigate the memorization overfitting problem. However, this usage simply aggregates the support set information into the query set, which is not as precise as learning the information required by the query set itself. Therefore, it is still inferior to our model. Removing Local adaptation means we only use the global value predictor to estimate the memory output. It is crucial to the value predictor since removing it from the value predictor results in an even worse performance than removing the value predictor . Besides, the significant drop in task consistency (C-score) shows that local adaptation contributes a lot to making the model adaptive to specific tasks, as it learns to adapt to each query-set sample. Memory Size. In Table 4 and 5, we investigate the variants of our task-specific memory module of different sizes. We control the memory size through | M | = store ratio  | D s | . The results demonstrate that our model is able to maintain high performance even with only a 20% memory size by storing diverse and representative samples of support sets. Besides, as the ratio of stored samples increases, the model's performance is improved since it provides more information for the inference of query samples and the optimization of the model initialization. Storing all the encountered samples (i.e., with store ratio 100%) in the memory instead introduces some noise that damages the model performance. Number of Neighbors. We also investigate the effects of different numbers of neighbors for the model performance in Table 4 and Table 5. In both datasets, the model performs better with a larger number of neighbors. However, when the number of neighbors is too large, the model retrieves some dissimilar slots from the memory module. These dissimilar slots bring much noise, which makes the predictions of query samples inaccurate. We present two generated cases in personalized dialog in Table. 6. Base Model , Fine-tune , and MAML generate general responses with little useful information or responses that are not consistent with the personality of personas. MR-MAML generates irrelevant responses to the dialogue context. Our model not only responds coherently to the dialog history but also caters to the persona descriptions of each user. In this paper, we tackle the memorization overfitting problem of meta-learning for text classification and generation applications. We propose MemIML to enhance the dependence of the model on the support sets for task adaptation. MemIML introduces a memory module storing the information of support sets, and propose an imitation module to better leverage the support set information by imitating the behaviors of the memory. Both empirical and theoretical results demonstrate that our method MemIML effectively alleviates the memorization overfitting problem. The persona-based dialogue generation task aims to build a dialogue model which generates meaningful, fluent, and consistent responses. It will facilitate human-computer interactions in practice. However, the training of the model for personalized Persona A I am a professional singer. dialogues may lead to the leakage of personal privacy information. In this work, the data source we use is from a published dataset and does not involve privacy issues for the data collection. Our proposed method does not include inference or judgments about individuals and does not generate any discriminatory, insulting responses. Our work validates the proposed method and baseline models on human evaluation which involves manual labor. We hire five annotators to score 750 generated sentences in total (250 sentences for each model we evaluate). The hourly pay is set to 15 US$ per person, which is higher than the local statutory minimum wage. Research on this paper was supported by Hong Kong Research Grants Council (Grant No. 16204920) and National Natural Science Foundation of China (Grant No. 62106275).
abstain abstain abstain objective method objective abstain abstain abstain abstain abstain objective method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain objective abstain abstain abstain abstain objective objective objective other other other other other other other other other other objective other other other other other other other abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain abstain other abstain method abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain method abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method objective abstain objective abstain abstain abstain abstain method abstain abstain method abstain other