Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation
Generating long and coherent reports to describe medical images poses challenges to bridging visual patterns with informative human linguistic descriptions. We propose a novel Hybrid Retrieval-Generation Reinforced Agent (HRGR-Agent) which reconciles traditional retrieval-based approaches populated with human prior knowledge, with modern learning-based approaches to achieve structured, robust, and diverse report generation. HRGR-Agent employs a hierarchical decisionmaking procedure. For each sentence, a high-level retrieval policy module chooses to either retrieve a template sentence from an off-the-shelf template database, or invoke a low-level generation module to generate a new sentence. HRGR-Agent is updated via reinforcement learning, guided by sentence-level and word-level rewards. Experiments show that our approach achieves the state-of-the-art results on two medical report datasets, generating well-balanced structured sentences with robust coverage of heterogeneous medical report contents. In addition, our model achieves the highest detection precision of medical abnormality terminologies, and improved human evaluation performance.
Summary The paper presents an approach using hierarchical reinforcement learning to address the problem of automatically generating medical reports using diagnostics images. The approach first predicts a sequence of hidden states for each sentence, and deicdes when to stop, and a low level model takes the hidden state and either retrieves a sentence and uses it as an output or passes control to a generator which generates a sentence. The overall system is trained with rewards at both sentence level as well as word-level for generation. The proposed approach shows promise over ablations of the proposed model as well as some sensible baseline CNN-RNN based approaches for image captioning.  Strengths + Paper provides the experimental details of the setup quite thoroughly. + Paper clearly mentions the hyperparameters used for training.  Weaknesses  Motivation 1. It would be nice to explain the motivation for the model more clearly. Is the idea that one wants to be able to retrieve the templated sentences and generate the less templated ones? How is mixing generation and retrieval helping with more common sentences vs less common sentences problem?  2. It would be good to perform evaluation with actual patient outcomes instead of evaluation using metrics like CIDEr or even human evaluation on Mechanical Turk. In a lot of ways, it seems important to get validation in terms of patient outcomes instead of M-Turk since that is what matters at the end of the day. It would also be good to take different risk functions (for saying different things) into account in the model.    Notation 3. Eqn. 14 and 15, the notation for conditioning inside the indicator variable is somewhat confusing, and there does not seem an obvious need for it to be there. Also, it would be good to make the bracket after I(z_i > 1 / 2) larger to indicate that it is over both the terms in the gradient.   Formulation  4. Choice of delta-CIDEr, especially for providing the reward at each timestep of the generation seems very odd and unjustified. Ideally the reward that is optimized for should be the reward provided at the end of an entire sequence CIDEr(w_1, …, w_T, gt), instead of that we are optimizing a proxy, which is the described delta CIDEr score, which might have a problem, that the reward for a word might be low at the given timestep, but might be a part of say, a 4-gram which would get us really high reward (at the sequence level), yet it would not be given importance because the marginal improvement at the world level is low. A more principled approach might be to get intermediate rewards by training a critic using an approach similar to [A].  5. How long do the reports tend to be? What is the cost for summarizing things incorrectly? It woud be good to do attention on the image and indeed verify that the model is looking at the image when providing the caption. How often does the model retrieve vs generate? (*)  *Preliminary Evaluation* The paper is quite thorough in its execution, has a number of sensible baselines and proposes a sensible model for a real-world task. Although the model itself is not very novel, it seems like a decent application paper that is well executed and clear.  References: [A]: Bahdanau, Dzmitry, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016. “An Actor-Critic Algorithm for Sequence Prediction.” arXiv [cs.LG]. arXiv. http://arxiv.org/abs/1607.07086.  Final Comments -------------------------------------- After having read the other reviews as well as the author rebuttal, I am unchanged in my opinion about this paper. I think it would be a good application paper at NIPS -- the idea of mixing retrieval with generation in the context of the application makes a lot of sense (although the approach itself is not very novel as pointed out by R4 -- copy mechanisms have been used for sequence to sequence models). In light of the thoroughness of the execution, i still lean towards accept, but it would not be too bad to reject the paper either.