Making DenseNet Interpretable: A Case Study in Clinical Radiology
The monotonous routine of medical image analysis under tight time constraints has always led to work fatigue for many medical practitioners. Medical image interpretation can be error-prone and this can increase the risk of an incorrect procedure being recommended. While the advancement of complex deep learning models has achieved performance beyond human capability in some computer vision tasks, widespread adoption in the medical field has been held back, among other factors, by poor model interpretability and a lack of high-quality labelled data. This paper introduces a model interpretation and visualisation framework for the analysis of the feature extraction process in a deep convolutional neural network and applies it to abnormality detection using the musculoskeletal radio-graph dataset (MURA, Stanford). The proposed framework provides a mechanism for interpreting DenseNet deep learning architectures. It aims to provide a deeper insight about the paths of feature generation and reasoning within a DenseNet architecture. When evaluated on MURA at abnormality detection tasks, the model interpretation framework has been shown capable of identifying limitations from the reasoning of a DenseNet architecture applied to radiography, which can in turn be ameliorated through model interpretation and visualization.
Summary The authors present a framework for visualization and interpretation of medical images classified with a DenseNet: the framework is tested on MURA, for abnormality detection in musculoskeletal radiograph images, allowing to do feature extraction and visualize CAMs. The authors claim that the framework helps clinicians to build trustability in the model and identify hidden features extraction failures. Content Although the work is interesting, it doesn*t bring any novelties, the methods used (feature extraction and CAM) are well known and the paper does not offer any quantitative measurements to back-up its claims. The paper is well written, in my opinion, it is a good paper for a conference/journal at the interface of healthcare and Deep Learning in a more applied field. However, most of the content is too detailed for the conference it aims for: Whole paragraph on how radiography works, which is not important here to explain the results, too many details on the structure of a DenseNet which is a well-known model in the community, even most of the proposed framework could have been written more concisely. The results are exclusively qualitative, moreover, the resulting images are shown in the appendix which elongates the paper. With more concise writing and a better formating for the figures (not showing all the features for example), the authors could have fit the results with the explanations. As for quantitative experiments, I would encourage the authors to have a look at Evaluating Feature Importance Estimates (Hooker et al) and other experiments in Sanity Checks for Saliency Maps (Adebayo et al). The last paper even gives good explanations on the *edge detector* resemblance of CNN, which could help the authors to explain some of their results (the frame in fig 10 for example). It is unclear why CAM was used here and not GradCam or Grad-CAM++ (Chattopadhyay et al) or even more recently Smooth Grad-CAM++ (Omeiza et al.). Even saliency map approaches such as Guided backprop would have been interesting to see in this work. The paper brings results on a DenseNet trained on ImageNet, but it would also have been interesting to see the resulting visualizations with a pre-trained model on an X-ray dataset (for example ChestXray from NIH). The part of how this framework could help clinicians make their analysis faster could be detailed: The framework generates features for each blocks, which seems a lot to analyze in the end, and even if those features offer a good interpretation of where the network is looking and why it is wrong, the authors do not offer a solution as to change this in order to improve the accuracy. Typo - in Discussion, first paragraph, last line, *it will then zoomed* -> *it will then be zoomed*? Conclusion In the end, the paper is interesting and I believe the framework would be relevant in a hospital, with maybe more features such as different models, methods of interpretability, etc. However, it doesn*t offer any novelty in the Deep Learning field and does not fit for a conference such as ICLR.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Making DenseNet Interpretable: A Case Study in Clinical Radiology
The monotonous routine of medical image analysis under tight time constraints has always led to work fatigue for many medical practitioners. Medical image interpretation can be error-prone and this can increase the risk of an incorrect procedure being recommended. While the advancement of complex deep learning models has achieved performance beyond human capability in some computer vision tasks, widespread adoption in the medical field has been held back, among other factors, by poor model interpretability and a lack of high-quality labelled data. This paper introduces a model interpretation and visualisation framework for the analysis of the feature extraction process in a deep convolutional neural network and applies it to abnormality detection using the musculoskeletal radio-graph dataset (MURA, Stanford). The proposed framework provides a mechanism for interpreting DenseNet deep learning architectures. It aims to provide a deeper insight about the paths of feature generation and reasoning within a DenseNet architecture. When evaluated on MURA at abnormality detection tasks, the model interpretation framework has been shown capable of identifying limitations from the reasoning of a DenseNet architecture applied to radiography, which can in turn be ameliorated through model interpretation and visualization.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Summary The authors present a framework for visualization and interpretation of medical images classified with a DenseNet: the framework is tested on MURA, for abnormality detection in musculoskeletal radiograph images, allowing to do feature extraction and visualize CAMs. The authors claim that the framework helps clinicians to build trustability in the model and identify hidden features extraction failures. Content Although the work is interesting, it doesn*t bring any novelties, the methods used (feature extraction and CAM) are well known and the paper does not offer any quantitative measurements to back-up its claims. The paper is well written, in my opinion, it is a good paper for a conference/journal at the interface of healthcare and Deep Learning in a more applied field. However, most of the content is too detailed for the conference it aims for: Whole paragraph on how radiography works, which is not important here to explain the results, too many details on the structure of a DenseNet which is a well-known model in the community, even most of the proposed framework could have been written more concisely. The results are exclusively qualitative, moreover, the resulting images are shown in the appendix which elongates the paper. With more concise writing and a better formating for the figures (not showing all the features for example), the authors could have fit the results with the explanations. As for quantitative experiments, I would encourage the authors to have a look at Evaluating Feature Importance Estimates (Hooker et al) and other experiments in Sanity Checks for Saliency Maps (Adebayo et al). The last paper even gives good explanations on the *edge detector* resemblance of CNN, which could help the authors to explain some of their results (the frame in fig 10 for example). It is unclear why CAM was used here and not GradCam or Grad-CAM++ (Chattopadhyay et al) or even more recently Smooth Grad-CAM++ (Omeiza et al.). Even saliency map approaches such as Guided backprop would have been interesting to see in this work. The paper brings results on a DenseNet trained on ImageNet, but it would also have been interesting to see the resulting visualizations with a pre-trained model on an X-ray dataset (for example ChestXray from NIH). The part of how this framework could help clinicians make their analysis faster could be detailed: The framework generates features for each blocks, which seems a lot to analyze in the end, and even if those features offer a good interpretation of where the network is looking and why it is wrong, the authors do not offer a solution as to change this in order to improve the accuracy. Typo - in Discussion, first paragraph, last line, *it will then zoomed* -> *it will then be zoomed*? Conclusion In the end, the paper is interesting and I believe the framework would be relevant in a hospital, with maybe more features such as different models, methods of interpretability, etc. However, it doesn*t offer any novelty in the Deep Learning field and does not fit for a conference such as ICLR.
assistant
Summary The authors present a framework for visualization and interpretation of medical images classified with a DenseNet: the framework is tested on MURA, for abnormality detection in musculoskeletal radiograph images, allowing to do feature extraction and visualize CAMs. The authors claim that the framework helps clinicians to build trustability in the model and identify hidden features extraction failures. Content Although the work is interesting, it doesn*t bring any novelties, the methods used (feature extraction and CAM) are well known and the paper does not offer any quantitative measurements to back-up its claims. The paper is well written, in my opinion, it is a good paper for a conference/journal at the interface of healthcare and Deep Learning in a more applied field. However, most of the content is too detailed for the conference it aims for: Whole paragraph on how radiography works, which is not important here to explain the results, too many details on the structure of a DenseNet which is a well-known model in the community, even most of the proposed framework could have been written more concisely. The results are exclusively qualitative, moreover, the resulting images are shown in the appendix which elongates the paper. With more concise writing and a better formating for the figures (not showing all the features for example), the authors could have fit the results with the explanations. As for quantitative experiments, I would encourage the authors to have a look at Evaluating Feature Importance Estimates (Hooker et al) and other experiments in Sanity Checks for Saliency Maps (Adebayo et al). The last paper even gives good explanations on the *edge detector* resemblance of CNN, which could help the authors to explain some of their results (the frame in fig 10 for example). It is unclear why CAM was used here and not GradCam or Grad-CAM++ (Chattopadhyay et al) or even more recently Smooth Grad-CAM++ (Omeiza et al.). Even saliency map approaches such as Guided backprop would have been interesting to see in this work. The paper brings results on a DenseNet trained on ImageNet, but it would also have been interesting to see the resulting visualizations with a pre-trained model on an X-ray dataset (for example ChestXray from NIH). The part of how this framework could help clinicians make their analysis faster could be detailed: The framework generates features for each blocks, which seems a lot to analyze in the end, and even if those features offer a good interpretation of where the network is looking and why it is wrong, the authors do not offer a solution as to change this in order to improve the accuracy. Typo - in Discussion, first paragraph, last line, *it will then zoomed* -> *it will then be zoomed*? Conclusion In the end, the paper is interesting and I believe the framework would be relevant in a hospital, with maybe more features such as different models, methods of interpretability, etc. However, it doesn*t offer any novelty in the Deep Learning field and does not fit for a conference such as ICLR.