The International Classification of Diseases (ICD) is a list of classification codes for the diagnoses. Automatic ICD coding is a multi-label text classification task with noisy clinical document inputs and extremely long-tailed label distribution, making it difficult to perform fine-grained classification on both frequent and zeroshot codes at the same time. In this paper, we propose a latent feature generation framework for generalized zero-shot ICD coding, where we aim to improve the prediction on codes that have no labeled data without compromising the performance on seen codes. Our framework generates semantically meaningful features for zero-shot codes by exploiting ICD code hierarchical structure and a novel cycle architecture that reconstructs the relevant keywords. To the best of our knowledge, this is the first adversarial generative model for the generalized zero-shot learning on multi-label text classification. Extensive experiments demonstrate the effectiveness of our approach. On the public MIMIC-III dataset, our methods improve the F1 score from nearly 0 to 20.91% for the zero-shot codes, and increase the AUC score by 3% (absolute improvement) from previous state of the art.
The paper proposes a method to do zero-shot ICD coding, which comes down to determining which elements from a set of natural language labels (ICD codes) apply to a given text (diagnostic summary). The main problem is that many codes have zero or very few examples. The proposed solution for this problem is to learn a feature-space generator for examples which can be conditioned on a code. This generator is trained using a GAN, which moves the few-/zero-shot problem to training the discriminator. Here, the tree structure of the ICD codes is used, by using examples with sibling labels as approximate examples for codes with few or on labelled data points. Additionally, a keyword reconstruction loss is used, based on the idea that the keywords of the corresponding ICD code can be reconstructed from a good feature vector.
The paper is written and executed well; the setup, which involves a fair number of components, is described and motivated clearly. The experiments include comparisons to state of the art results, finding that applying the GAN-based technique they choose (introduced in Xian et al., 2018 for computer vision problems) produces significant gains in recall of low-data classes (few or zero examples). The precision decreases, probably due to a shift from false negatives to false positives, while the AUC shows modest gains.
The main potential issue with the paper is the degree of (effective) novelty. The bulk of the gain relative to the SotA seems to be achievable by applying the GAN-based method, which is described in the Xian et al. (2018) reference, or by applying the label-distribution-aware margin, which are known techniques even if they have not been applied to this particular dataset yet. The additional elements introduced by the authors - the use of sibling codes and the keyword reconstruction loss - are good ideas, and it is worthwhile having a documented test of the benefit they provide, but they don’t seem to have a major influence on the quality of the model.
All in all, I would argue for the paper to be accepted. The work done here is valuable to have on record, and the presentation and execution are well done.
Two questions for the authors:
1. Could the model benefit from having a self-attention model, like a transformer? This applies mostly to the diagnostic text encoder, as interpreting the ICD code descriptions seems not to depend strongly on structure or context. From the text of the paper it appears that a 1D convolution was used to process the diagnostic texts, but I would expect that longer-distance links between words can be quite relevant there.
2. Could the precision-recall curves be added to the supplementary material?