Generalized Zero-shot ICD Coding
The International Classification of Diseases (ICD) is a list of classification codes for the diagnoses. Automatic ICD coding is in high demand as the manual coding can be labor-intensive and error-prone. It is a multi-label text classification task with extremely long-tailed label distribution, making it difficult to perform fine-grained classification on both frequent and zero-shot codes at the same time. In this paper, we propose a latent feature generation framework for generalized zero-shot ICD coding, where we aim to improve the prediction on codes that have no labeled data without compromising the performance on seen codes. Our framework generates pseudo features conditioned on the ICD code descriptions and exploits the ICD code hierarchical structure. To guarantee the semantic consistency between the generated features and real features, we reconstruct the keywords in the input documents that are related to the conditioned ICD codes. To the best of our knowledge, this works represents the first one that proposes an adversarial generative model for the generalized zero-shot learning on multi-label text classification. Extensive experiments demonstrate the effectiveness of our approach. On the public MIMIC-III dataset, our methods improve the F1 score from nearly 0 to 20.91% for the zero-shot codes, and increase the AUC score by 3% (absolute improvement) from previous state of the art. We also show that the framework improves the performance on few-shot codes.
The paper deals with the problem of text classification when the number of class is large (17000) and most of the classes do not have examples in the training set. This problem is known as Zero-shot learning. The paper proposes to use adversarial methods and the hierarchical organisation of the classes to improve current models. The paper lacks a clear description of the complete system : Figure 1 seems to be the one but there is no mention of the classification part. The description of each bloc is clear enough independently but many question remains on the global picture. The author should for example emphasis the fact that in their problemn the classes come with a short text description. This is somehow unususal for a text classification problem where usually the classes are not defined by a description. In the figure, the difference of the processing of the text from the clinical document and from the class description is not clear. 3.1 is related to feature extraction for the clinical document, but it is also said that this processing is also apply to class description. 3.2 label encoder : if the notations were the same as on figure A, it would help understanding. Results : the claim *our methods improve the F1 score from nearly 0 to 20.91% for the zero-shot codes* is not true : state-of-the-art models such as Xian2018 and Felix2018 are already over 20% F1. When looking at Figure 3 and the analysis, WGAN-Z seem to provide better representation that WGAN with it has almost no impact on the classification results (F1 20.48 versus F1 20.30) In conclusion, it seems that the proposed methods make the model more complex without bringing a significant improvements.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Generalized Zero-shot ICD Coding
The International Classification of Diseases (ICD) is a list of classification codes for the diagnoses. Automatic ICD coding is in high demand as the manual coding can be labor-intensive and error-prone. It is a multi-label text classification task with extremely long-tailed label distribution, making it difficult to perform fine-grained classification on both frequent and zero-shot codes at the same time. In this paper, we propose a latent feature generation framework for generalized zero-shot ICD coding, where we aim to improve the prediction on codes that have no labeled data without compromising the performance on seen codes. Our framework generates pseudo features conditioned on the ICD code descriptions and exploits the ICD code hierarchical structure. To guarantee the semantic consistency between the generated features and real features, we reconstruct the keywords in the input documents that are related to the conditioned ICD codes. To the best of our knowledge, this works represents the first one that proposes an adversarial generative model for the generalized zero-shot learning on multi-label text classification. Extensive experiments demonstrate the effectiveness of our approach. On the public MIMIC-III dataset, our methods improve the F1 score from nearly 0 to 20.91% for the zero-shot codes, and increase the AUC score by 3% (absolute improvement) from previous state of the art. We also show that the framework improves the performance on few-shot codes.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The paper deals with the problem of text classification when the number of class is large (17000) and most of the classes do not have examples in the training set. This problem is known as Zero-shot learning. The paper proposes to use adversarial methods and the hierarchical organisation of the classes to improve current models. The paper lacks a clear description of the complete system : Figure 1 seems to be the one but there is no mention of the classification part. The description of each bloc is clear enough independently but many question remains on the global picture. The author should for example emphasis the fact that in their problemn the classes come with a short text description. This is somehow unususal for a text classification problem where usually the classes are not defined by a description. In the figure, the difference of the processing of the text from the clinical document and from the class description is not clear. 3.1 is related to feature extraction for the clinical document, but it is also said that this processing is also apply to class description. 3.2 label encoder : if the notations were the same as on figure A, it would help understanding. Results : the claim *our methods improve the F1 score from nearly 0 to 20.91% for the zero-shot codes* is not true : state-of-the-art models such as Xian2018 and Felix2018 are already over 20% F1. When looking at Figure 3 and the analysis, WGAN-Z seem to provide better representation that WGAN with it has almost no impact on the classification results (F1 20.48 versus F1 20.30) In conclusion, it seems that the proposed methods make the model more complex without bringing a significant improvements.
assistant
The paper deals with the problem of text classification when the number of class is large (17000) and most of the classes do not have examples in the training set. This problem is known as Zero-shot learning. The paper proposes to use adversarial methods and the hierarchical organisation of the classes to improve current models. The paper lacks a clear description of the complete system : Figure 1 seems to be the one but there is no mention of the classification part. The description of each bloc is clear enough independently but many question remains on the global picture. The author should for example emphasis the fact that in their problemn the classes come with a short text description. This is somehow unususal for a text classification problem where usually the classes are not defined by a description. In the figure, the difference of the processing of the text from the clinical document and from the class description is not clear. 3.1 is related to feature extraction for the clinical document, but it is also said that this processing is also apply to class description. 3.2 label encoder : if the notations were the same as on figure A, it would help understanding. Results : the claim *our methods improve the F1 score from nearly 0 to 20.91% for the zero-shot codes* is not true : state-of-the-art models such as Xian2018 and Felix2018 are already over 20% F1. When looking at Figure 3 and the analysis, WGAN-Z seem to provide better representation that WGAN with it has almost no impact on the classification results (F1 20.48 versus F1 20.30) In conclusion, it seems that the proposed methods make the model more complex without bringing a significant improvements.