Learning to diagnose from scratch by exploiting dependencies among labels
The field of medical diagnostics contains a wealth of challenges which closely resemble classical machine learning problems; practical constraints, however, complicate the translation of these endpoints naively into classical architectures. Many tasks in radiology, for example, are largely problems of multi-label classification wherein medical images are interpreted to indicate multiple present or suspected pathologies. Clinical settings drive the necessity for high accuracy simultaneously across a multitude of pathological outcomes and greatly limit the utility of tools which consider only a subset. This issue is exacerbated by a general scarcity of training data and maximizes the need to extract clinically relevant features from available samples -- ideally without the use of pre-trained models which may carry forward undesirable biases from tangentially related tasks. We present and evaluate a partial solution to these constraints in using LSTMs to leverage interdependencies among target labels in predicting 14 pathologic patterns from chest x-rays and establish state of the art results on the largest publicly available chest x-ray dataset from the NIH without pre-training. Furthermore, we propose and discuss alternative evaluation metrics and their relevance in clinical practice.
 This paper presents an impressive set of results on predicting lung pathologies from chest x-ray images. Authors present two architectures: one based on denseNet, and one based on denseNet + LSTM on output dimensions (i.e. similar to NADE model), and compare it to state of the art on the chest x-ray classification. Experiments are clearly described and results are significantly better compared to state of the art. The only issue with this paper is, that their proposed method, in practice is not tractable for inference on estimating probability of a single output, a task which would be critical in medical domain. Considering that their paper is titled as a work to use "dependencies" among labels, not being able to evaluate their network*s, and lack of interpretable evaluation results on this model in the experiment section is a major limitation. On the other hand, there are many alternative models where one could simply use multi-task learning and shared parameter, to predict multiple outcomes extremely efficiently. To be able to claim that this paper improved the prediction by better modeling of *dependencies* among labels, I would need to see how the (much simpler) multi-task setting works as well. That said, the paper has several positive aspects in all areas: Originality - the paper presents first combination of DenseNets with LSTM-based output factorization, Writing clarity - the paper is very well written and clear. Quality - (apart from the missing multi-task baseline), the results are significantly better than state of the art, and experiments are well done, Significance - Apart from the issue of intractable inference which is arguably a large limitation of this work, the application in medical field is significant.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Learning to diagnose from scratch by exploiting dependencies among labels
The field of medical diagnostics contains a wealth of challenges which closely resemble classical machine learning problems; practical constraints, however, complicate the translation of these endpoints naively into classical architectures. Many tasks in radiology, for example, are largely problems of multi-label classification wherein medical images are interpreted to indicate multiple present or suspected pathologies. Clinical settings drive the necessity for high accuracy simultaneously across a multitude of pathological outcomes and greatly limit the utility of tools which consider only a subset. This issue is exacerbated by a general scarcity of training data and maximizes the need to extract clinically relevant features from available samples -- ideally without the use of pre-trained models which may carry forward undesirable biases from tangentially related tasks. We present and evaluate a partial solution to these constraints in using LSTMs to leverage interdependencies among target labels in predicting 14 pathologic patterns from chest x-rays and establish state of the art results on the largest publicly available chest x-ray dataset from the NIH without pre-training. Furthermore, we propose and discuss alternative evaluation metrics and their relevance in clinical practice.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper presents an impressive set of results on predicting lung pathologies from chest x-ray images. Authors present two architectures: one based on denseNet, and one based on denseNet + LSTM on output dimensions (i.e. similar to NADE model), and compare it to state of the art on the chest x-ray classification. Experiments are clearly described and results are significantly better compared to state of the art. The only issue with this paper is, that their proposed method, in practice is not tractable for inference on estimating probability of a single output, a task which would be critical in medical domain. Considering that their paper is titled as a work to use "dependencies" among labels, not being able to evaluate their network*s, and lack of interpretable evaluation results on this model in the experiment section is a major limitation. On the other hand, there are many alternative models where one could simply use multi-task learning and shared parameter, to predict multiple outcomes extremely efficiently. To be able to claim that this paper improved the prediction by better modeling of *dependencies* among labels, I would need to see how the (much simpler) multi-task setting works as well. That said, the paper has several positive aspects in all areas: Originality - the paper presents first combination of DenseNets with LSTM-based output factorization, Writing clarity - the paper is very well written and clear. Quality - (apart from the missing multi-task baseline), the results are significantly better than state of the art, and experiments are well done, Significance - Apart from the issue of intractable inference which is arguably a large limitation of this work, the application in medical field is significant.
assistant
 This paper presents an impressive set of results on predicting lung pathologies from chest x-ray images. Authors present two architectures: one based on denseNet, and one based on denseNet + LSTM on output dimensions (i.e. similar to NADE model), and compare it to state of the art on the chest x-ray classification. Experiments are clearly described and results are significantly better compared to state of the art. The only issue with this paper is, that their proposed method, in practice is not tractable for inference on estimating probability of a single output, a task which would be critical in medical domain. Considering that their paper is titled as a work to use "dependencies" among labels, not being able to evaluate their network*s, and lack of interpretable evaluation results on this model in the experiment section is a major limitation. On the other hand, there are many alternative models where one could simply use multi-task learning and shared parameter, to predict multiple outcomes extremely efficiently. To be able to claim that this paper improved the prediction by better modeling of *dependencies* among labels, I would need to see how the (much simpler) multi-task setting works as well. That said, the paper has several positive aspects in all areas: Originality - the paper presents first combination of DenseNets with LSTM-based output factorization, Writing clarity - the paper is very well written and clear. Quality - (apart from the missing multi-task baseline), the results are significantly better than state of the art, and experiments are well done, Significance - Apart from the issue of intractable inference which is arguably a large limitation of this work, the application in medical field is significant.