Learning to diagnose from scratch by exploiting dependencies among labels
The field of medical diagnostics contains a wealth of challenges which closely resemble classical machine learning problems; practical constraints, however, complicate the translation of these endpoints naively into classical architectures. Many tasks in radiology, for example, are largely problems of multi-label classification wherein medical images are interpreted to indicate multiple present or suspected pathologies. Clinical settings drive the necessity for high accuracy simultaneously across a multitude of pathological outcomes and greatly limit the utility of tools which consider only a subset. This issue is exacerbated by a general scarcity of training data and maximizes the need to extract clinically relevant features from available samples -- ideally without the use of pre-trained models which may carry forward undesirable biases from tangentially related tasks. We present and evaluate a partial solution to these constraints in using LSTMs to leverage interdependencies among target labels in predicting 14 pathologic patterns from chest x-rays and establish state of the art results on the largest publicly available chest x-ray dataset from the NIH without pre-training. Furthermore, we propose and discuss alternative evaluation metrics and their relevance in clinical practice.
 Well written and appropriately structured. Well within the remit of the conference. Not much technical novelty to be found, but the original contributions are adequately identified and they are interesting on their own. My main concern (and complaint) is not technical, but application-based. This study is (unfortunately) typical in that it focuses on and provides detail of the technical modeling issues, but ignores the medical applicability of the model and results. This is exemplified by the fact that the data set is hardly described at all and the 14 abnormalities/pathologies, the rationale behind their choice and the possible interrelations and dependencies are never described from a medical viewpoint. If I were a medical expert, I would not have a clue about how these results and models could be applied in practice, or about what medical insight I could achieve. The bottom line seems to be: "my model and approach works better than the other guys* model and approach", but one is left with the impression that these experiments could have been made with other data, other problems, other fields of application and they would not have not changed much
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Learning to diagnose from scratch by exploiting dependencies among labels
The field of medical diagnostics contains a wealth of challenges which closely resemble classical machine learning problems; practical constraints, however, complicate the translation of these endpoints naively into classical architectures. Many tasks in radiology, for example, are largely problems of multi-label classification wherein medical images are interpreted to indicate multiple present or suspected pathologies. Clinical settings drive the necessity for high accuracy simultaneously across a multitude of pathological outcomes and greatly limit the utility of tools which consider only a subset. This issue is exacerbated by a general scarcity of training data and maximizes the need to extract clinically relevant features from available samples -- ideally without the use of pre-trained models which may carry forward undesirable biases from tangentially related tasks. We present and evaluate a partial solution to these constraints in using LSTMs to leverage interdependencies among target labels in predicting 14 pathologic patterns from chest x-rays and establish state of the art results on the largest publicly available chest x-ray dataset from the NIH without pre-training. Furthermore, we propose and discuss alternative evaluation metrics and their relevance in clinical practice.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 Well written and appropriately structured. Well within the remit of the conference. Not much technical novelty to be found, but the original contributions are adequately identified and they are interesting on their own. My main concern (and complaint) is not technical, but application-based. This study is (unfortunately) typical in that it focuses on and provides detail of the technical modeling issues, but ignores the medical applicability of the model and results. This is exemplified by the fact that the data set is hardly described at all and the 14 abnormalities/pathologies, the rationale behind their choice and the possible interrelations and dependencies are never described from a medical viewpoint. If I were a medical expert, I would not have a clue about how these results and models could be applied in practice, or about what medical insight I could achieve. The bottom line seems to be: "my model and approach works better than the other guys* model and approach", but one is left with the impression that these experiments could have been made with other data, other problems, other fields of application and they would not have not changed much
assistant
 Well written and appropriately structured. Well within the remit of the conference. Not much technical novelty to be found, but the original contributions are adequately identified and they are interesting on their own. My main concern (and complaint) is not technical, but application-based. This study is (unfortunately) typical in that it focuses on and provides detail of the technical modeling issues, but ignores the medical applicability of the model and results. This is exemplified by the fact that the data set is hardly described at all and the 14 abnormalities/pathologies, the rationale behind their choice and the possible interrelations and dependencies are never described from a medical viewpoint. If I were a medical expert, I would not have a clue about how these results and models could be applied in practice, or about what medical insight I could achieve. The bottom line seems to be: "my model and approach works better than the other guys* model and approach", but one is left with the impression that these experiments could have been made with other data, other problems, other fields of application and they would not have not changed much