Cost-Effective Interactive Neural Attention Learning
We propose a novel interactive attention learning framework which we refer to as Interactive Attention Learning (IAL), in which the human annotators interactively manipulate the allocated attentions to correct the model’s behavior, by updating the attention-generating model without having to retrain the network. For efficient update of the attention generator without retraining, we propose a novel attention mechanism Neural Attention Process (NAP), which can generate stochastic attentions based on scarce training instances, and can incorporate new training instances without retraining. Further, to minimize human interaction cost, we propose a cost-effective algorithm that selects the most negative training instances that yield incorrect and non-intuitive interpretation with influence function and re-rank the attentions on the input features by their uncertainties, such that the annotators label the instances and attentions that are more influential to the prediction first. We validate IAL on multiple datasets from the healthcare domain, on which it not only significantly outperforms other baselines, but also achieves explicitly trained attention for interpretability that agrees well with human interpretation without expensive human labeling procedure.
The authors introduce an interactive attention learning framework called Neural Attention Process (NAP) which produces weight predictions over the input variables in addition to the usual target predictions. The motivation behind this type of model is to obtain an interpretable explanation of the model’s output which can be shaped by a small set of human expert annotations. At a higher level NAP seems to be an extension of a previously introduced model called RETAIN with an additional component inspired by Neural Processes (NPs) which incorporates the annotations of the experts. The authors test their model on five different tasks from 3 datasets (health records, fitness and real estate) and compare to a number of related baselines. While I think the idea is interesting, in my opinion this paper is not ready for publication and I would argue for it to be rejected, for the following reasons: * Presentation: The paper is over badly written, sloppy and not well structured. * Badly written: there are lots of grammatical errors in the paper to the point that it severely affects its clarity (for example, the subject of a sentence often changes from plural to singular or vice versa within the same sentence and it’s unclear what the authors are referring to). * Sloppy: many variables are not defined in the paper, and some are called different names depending on the figure or part of the text (i.e. the use of ‘s’ or ‘t’ interchangeably to refer to the iterations). At some point even the name of the model (referred to as NAP for the majority of the paper) is just referred to as NP-Selective in the text and some figures (compare nomenclature in table 1 vs figure 5). This lack of attention to detail (although the name of the main model should hardly count as a detail) makes me worry about the scientific rigorousness of the work. * Not well structured: The paper is not very easy to follow as there are jumps between different models (RETAIN, NPs, NAPs) which are not always evident. On that note, I would have liked to see an objective function to know what exactly is being used to optimise theta. * Significance: Overall the results don’t seem incredibly significant (e.g. figure 6). If one of the main claims is that the number or annotated examples is smaller with NAP the authors should provide experimental evidence that this is the case (as this would be one of the most significant benefits of the model). * Motivation: a key motivation for this paper, as stated by the authors, is the desire to obtain interpretable representations, which in itself is an important research problem. However, I am not sure that the suggested solution is addressing this issue. If I understand correctly, what the model learns is, in essence, a weight mask over the inputs that determines how much each of the dimensions of the input contribute to the output. This could be achieved with any attention model in and is in itself not novel. I think a more interesting motivation for this model might be its ability to incorporate some human annotations that teach the model where to attend. However if this was the main story I think the authors should focus on making this a stronger argument and proving that narrowing down the attention with said annotations actually leads to better/faster/more robust learning, rather than focussing on the interpretability narrative. Some more specific comments and recommendations for the authors (I think there are some positive aspects of this paper and I hope that I can give a few points of advice to help improve its quality.) I will go through them in the order of the paper: 1) The name Neural attention process might be a bit confusing given that there are already Attentive Neural Processes. I would strongly recommend changing the name. 2) Check the language and make sure it’s correct, the introduction is particularly full of errors and you don’t want to lose the reader from the beginning. 3) If you are talking about interpretability you would have to mention as well all the work on interpretable representations (e.g. disentangled representations etc) and if you talk about attention, it is probably worth mentioning transformers. 4) Algorithm 1: this is not very clear. I would define the variables at the top that are not self-explanatory (such as S). Also, from the look of the algorithm theta is not updated in the else loop. 5) The change in notation is very confusing. It’s not always clear when you use ‘v’ vs ‘l’ or ‘s’ vs ‘t’. Sometimes it seems it depends on which of the different models you are talking about, but either way you should try to make it all uniform. 6) I would strongly recommend to specify the loss function and the regularisation term as this will make the task much clearer to the reader. 7) I would make the difference to RETAIN very clear and argue why you think your addition improves on the original (from the text at the moment it’s not even clear when you are describing RETAIN and when you talk about your model, maybe making a subsection would help?) 8) Figure 3 is too small. 9) You claim to have 5 datasets, but the way I would interpret your setup, you have 5 tasks from 3 datasets. 10) I would say a sentence or two about why you chose the baselines you chose and why you are hoping that they prove your different points. 11) Figure 6 has no axis labels on the plot itself, which are definitely necessary. I also recommend not using red and green as the two colours as it will be hard for colour blind to distinguish between those two.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Cost-Effective Interactive Neural Attention Learning
We propose a novel interactive attention learning framework which we refer to as Interactive Attention Learning (IAL), in which the human annotators interactively manipulate the allocated attentions to correct the model’s behavior, by updating the attention-generating model without having to retrain the network. For efficient update of the attention generator without retraining, we propose a novel attention mechanism Neural Attention Process (NAP), which can generate stochastic attentions based on scarce training instances, and can incorporate new training instances without retraining. Further, to minimize human interaction cost, we propose a cost-effective algorithm that selects the most negative training instances that yield incorrect and non-intuitive interpretation with influence function and re-rank the attentions on the input features by their uncertainties, such that the annotators label the instances and attentions that are more influential to the prediction first. We validate IAL on multiple datasets from the healthcare domain, on which it not only significantly outperforms other baselines, but also achieves explicitly trained attention for interpretability that agrees well with human interpretation without expensive human labeling procedure.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The authors introduce an interactive attention learning framework called Neural Attention Process (NAP) which produces weight predictions over the input variables in addition to the usual target predictions. The motivation behind this type of model is to obtain an interpretable explanation of the model’s output which can be shaped by a small set of human expert annotations. At a higher level NAP seems to be an extension of a previously introduced model called RETAIN with an additional component inspired by Neural Processes (NPs) which incorporates the annotations of the experts. The authors test their model on five different tasks from 3 datasets (health records, fitness and real estate) and compare to a number of related baselines. While I think the idea is interesting, in my opinion this paper is not ready for publication and I would argue for it to be rejected, for the following reasons: * Presentation: The paper is over badly written, sloppy and not well structured. * Badly written: there are lots of grammatical errors in the paper to the point that it severely affects its clarity (for example, the subject of a sentence often changes from plural to singular or vice versa within the same sentence and it’s unclear what the authors are referring to). * Sloppy: many variables are not defined in the paper, and some are called different names depending on the figure or part of the text (i.e. the use of ‘s’ or ‘t’ interchangeably to refer to the iterations). At some point even the name of the model (referred to as NAP for the majority of the paper) is just referred to as NP-Selective in the text and some figures (compare nomenclature in table 1 vs figure 5). This lack of attention to detail (although the name of the main model should hardly count as a detail) makes me worry about the scientific rigorousness of the work. * Not well structured: The paper is not very easy to follow as there are jumps between different models (RETAIN, NPs, NAPs) which are not always evident. On that note, I would have liked to see an objective function to know what exactly is being used to optimise theta. * Significance: Overall the results don’t seem incredibly significant (e.g. figure 6). If one of the main claims is that the number or annotated examples is smaller with NAP the authors should provide experimental evidence that this is the case (as this would be one of the most significant benefits of the model). * Motivation: a key motivation for this paper, as stated by the authors, is the desire to obtain interpretable representations, which in itself is an important research problem. However, I am not sure that the suggested solution is addressing this issue. If I understand correctly, what the model learns is, in essence, a weight mask over the inputs that determines how much each of the dimensions of the input contribute to the output. This could be achieved with any attention model in and is in itself not novel. I think a more interesting motivation for this model might be its ability to incorporate some human annotations that teach the model where to attend. However if this was the main story I think the authors should focus on making this a stronger argument and proving that narrowing down the attention with said annotations actually leads to better/faster/more robust learning, rather than focussing on the interpretability narrative. Some more specific comments and recommendations for the authors (I think there are some positive aspects of this paper and I hope that I can give a few points of advice to help improve its quality.) I will go through them in the order of the paper: 1) The name Neural attention process might be a bit confusing given that there are already Attentive Neural Processes. I would strongly recommend changing the name. 2) Check the language and make sure it’s correct, the introduction is particularly full of errors and you don’t want to lose the reader from the beginning. 3) If you are talking about interpretability you would have to mention as well all the work on interpretable representations (e.g. disentangled representations etc) and if you talk about attention, it is probably worth mentioning transformers. 4) Algorithm 1: this is not very clear. I would define the variables at the top that are not self-explanatory (such as S). Also, from the look of the algorithm theta is not updated in the else loop. 5) The change in notation is very confusing. It’s not always clear when you use ‘v’ vs ‘l’ or ‘s’ vs ‘t’. Sometimes it seems it depends on which of the different models you are talking about, but either way you should try to make it all uniform. 6) I would strongly recommend to specify the loss function and the regularisation term as this will make the task much clearer to the reader. 7) I would make the difference to RETAIN very clear and argue why you think your addition improves on the original (from the text at the moment it’s not even clear when you are describing RETAIN and when you talk about your model, maybe making a subsection would help?) 8) Figure 3 is too small. 9) You claim to have 5 datasets, but the way I would interpret your setup, you have 5 tasks from 3 datasets. 10) I would say a sentence or two about why you chose the baselines you chose and why you are hoping that they prove your different points. 11) Figure 6 has no axis labels on the plot itself, which are definitely necessary. I also recommend not using red and green as the two colours as it will be hard for colour blind to distinguish between those two.
assistant
The authors introduce an interactive attention learning framework called Neural Attention Process (NAP) which produces weight predictions over the input variables in addition to the usual target predictions. The motivation behind this type of model is to obtain an interpretable explanation of the model’s output which can be shaped by a small set of human expert annotations. At a higher level NAP seems to be an extension of a previously introduced model called RETAIN with an additional component inspired by Neural Processes (NPs) which incorporates the annotations of the experts. The authors test their model on five different tasks from 3 datasets (health records, fitness and real estate) and compare to a number of related baselines. While I think the idea is interesting, in my opinion this paper is not ready for publication and I would argue for it to be rejected, for the following reasons: * Presentation: The paper is over badly written, sloppy and not well structured. * Badly written: there are lots of grammatical errors in the paper to the point that it severely affects its clarity (for example, the subject of a sentence often changes from plural to singular or vice versa within the same sentence and it’s unclear what the authors are referring to). * Sloppy: many variables are not defined in the paper, and some are called different names depending on the figure or part of the text (i.e. the use of ‘s’ or ‘t’ interchangeably to refer to the iterations). At some point even the name of the model (referred to as NAP for the majority of the paper) is just referred to as NP-Selective in the text and some figures (compare nomenclature in table 1 vs figure 5). This lack of attention to detail (although the name of the main model should hardly count as a detail) makes me worry about the scientific rigorousness of the work. * Not well structured: The paper is not very easy to follow as there are jumps between different models (RETAIN, NPs, NAPs) which are not always evident. On that note, I would have liked to see an objective function to know what exactly is being used to optimise theta. * Significance: Overall the results don’t seem incredibly significant (e.g. figure 6). If one of the main claims is that the number or annotated examples is smaller with NAP the authors should provide experimental evidence that this is the case (as this would be one of the most significant benefits of the model). * Motivation: a key motivation for this paper, as stated by the authors, is the desire to obtain interpretable representations, which in itself is an important research problem. However, I am not sure that the suggested solution is addressing this issue. If I understand correctly, what the model learns is, in essence, a weight mask over the inputs that determines how much each of the dimensions of the input contribute to the output. This could be achieved with any attention model in and is in itself not novel. I think a more interesting motivation for this model might be its ability to incorporate some human annotations that teach the model where to attend. However if this was the main story I think the authors should focus on making this a stronger argument and proving that narrowing down the attention with said annotations actually leads to better/faster/more robust learning, rather than focussing on the interpretability narrative. Some more specific comments and recommendations for the authors (I think there are some positive aspects of this paper and I hope that I can give a few points of advice to help improve its quality.) I will go through them in the order of the paper: 1) The name Neural attention process might be a bit confusing given that there are already Attentive Neural Processes. I would strongly recommend changing the name. 2) Check the language and make sure it’s correct, the introduction is particularly full of errors and you don’t want to lose the reader from the beginning. 3) If you are talking about interpretability you would have to mention as well all the work on interpretable representations (e.g. disentangled representations etc) and if you talk about attention, it is probably worth mentioning transformers. 4) Algorithm 1: this is not very clear. I would define the variables at the top that are not self-explanatory (such as S). Also, from the look of the algorithm theta is not updated in the else loop. 5) The change in notation is very confusing. It’s not always clear when you use ‘v’ vs ‘l’ or ‘s’ vs ‘t’. Sometimes it seems it depends on which of the different models you are talking about, but either way you should try to make it all uniform. 6) I would strongly recommend to specify the loss function and the regularisation term as this will make the task much clearer to the reader. 7) I would make the difference to RETAIN very clear and argue why you think your addition improves on the original (from the text at the moment it’s not even clear when you are describing RETAIN and when you talk about your model, maybe making a subsection would help?) 8) Figure 3 is too small. 9) You claim to have 5 datasets, but the way I would interpret your setup, you have 5 tasks from 3 datasets. 10) I would say a sentence or two about why you chose the baselines you chose and why you are hoping that they prove your different points. 11) Figure 6 has no axis labels on the plot itself, which are definitely necessary. I also recommend not using red and green as the two colours as it will be hard for colour blind to distinguish between those two.