Abstract Diagrammatic Reasoning with Multiplex Graph Networks
Abstract reasoning, particularly in the visual domain, is a complex human ability, but it remains a challenging problem for artificial neural learning systems. In this work we propose MXGNet, a multilayer graph neural network for multi-panel diagrammatic reasoning tasks. MXGNet combines three powerful concepts, namely, object-level representation, graph neural networks and multiplex graphs, for solving visual reasoning tasks. MXGNet first extracts object-level representations for each element in all panels of the diagrams, and then forms a multi-layer multiplex graph capturing multiple relations between objects across different diagram panels. MXGNet summarises the multiple graphs extracted from the diagrams of the task, and uses this summarisation to pick the most probable answer from the given candidates. We have tested MXGNet on two types of diagrammatic reasoning tasks, namely Diagram Syllogisms and Raven Progressive Matrices (RPM). For an Euler Diagram Syllogism task MXGNet achieves state-of-the-art accuracy of 99.8%. For PGM and RAVEN, two comprehensive datasets for RPM reasoning, MXGNet outperforms the state-of-the-art models by a considerable margin.
This paper proposes using a new version of graph networks – multiplex graph networks – which do object representation followed by some form of graph processing and reasoning to answer *IQ test* style diagrammatic reasoning, in particular including Raven Progressive Matrices that have been previously studied (a little). The paper shows very strong results on multiple datasets, much stronger than previous results (from strong groups) on these datasets. On these grounds, I believe the paper should be accepted. However, the structure and writing of the paper was very frustrating to me. The paper just didn*t make much of an attempt to explain and then motivate/analyze the model used. I mean, if I were writing the paper, I would have considered and done many things, such as: - shortening the introduction - shortening the related work - making the presentation of the datasets more succinct - having only one figure that covers most of what is currently in figures 1 and 2 - putting details of what seem more ancillary details like the treatment of background lines objects in an appendix - remove Figure 3, which didn*t convey much to me in the absence of more careful explanation of the model. so that I could motivate, carefully explain, and evaluate the main model in the paper. But here, all these things fill the main text, and we*re told that we have to read the appendices to understand the model.... And the presentation in the appendix is more a dump-all-the-facts presentation than a careful development of the design. Nevertheless, the general direction of the architecture seems sound, and the results look very strong, and there are even some useful ablations in the appendix.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Abstract Diagrammatic Reasoning with Multiplex Graph Networks
Abstract reasoning, particularly in the visual domain, is a complex human ability, but it remains a challenging problem for artificial neural learning systems. In this work we propose MXGNet, a multilayer graph neural network for multi-panel diagrammatic reasoning tasks. MXGNet combines three powerful concepts, namely, object-level representation, graph neural networks and multiplex graphs, for solving visual reasoning tasks. MXGNet first extracts object-level representations for each element in all panels of the diagrams, and then forms a multi-layer multiplex graph capturing multiple relations between objects across different diagram panels. MXGNet summarises the multiple graphs extracted from the diagrams of the task, and uses this summarisation to pick the most probable answer from the given candidates. We have tested MXGNet on two types of diagrammatic reasoning tasks, namely Diagram Syllogisms and Raven Progressive Matrices (RPM). For an Euler Diagram Syllogism task MXGNet achieves state-of-the-art accuracy of 99.8%. For PGM and RAVEN, two comprehensive datasets for RPM reasoning, MXGNet outperforms the state-of-the-art models by a considerable margin.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes using a new version of graph networks – multiplex graph networks – which do object representation followed by some form of graph processing and reasoning to answer *IQ test* style diagrammatic reasoning, in particular including Raven Progressive Matrices that have been previously studied (a little). The paper shows very strong results on multiple datasets, much stronger than previous results (from strong groups) on these datasets. On these grounds, I believe the paper should be accepted. However, the structure and writing of the paper was very frustrating to me. The paper just didn*t make much of an attempt to explain and then motivate/analyze the model used. I mean, if I were writing the paper, I would have considered and done many things, such as: - shortening the introduction - shortening the related work - making the presentation of the datasets more succinct - having only one figure that covers most of what is currently in figures 1 and 2 - putting details of what seem more ancillary details like the treatment of background lines objects in an appendix - remove Figure 3, which didn*t convey much to me in the absence of more careful explanation of the model. so that I could motivate, carefully explain, and evaluate the main model in the paper. But here, all these things fill the main text, and we*re told that we have to read the appendices to understand the model.... And the presentation in the appendix is more a dump-all-the-facts presentation than a careful development of the design. Nevertheless, the general direction of the architecture seems sound, and the results look very strong, and there are even some useful ablations in the appendix.
assistant
This paper proposes using a new version of graph networks – multiplex graph networks – which do object representation followed by some form of graph processing and reasoning to answer *IQ test* style diagrammatic reasoning, in particular including Raven Progressive Matrices that have been previously studied (a little). The paper shows very strong results on multiple datasets, much stronger than previous results (from strong groups) on these datasets. On these grounds, I believe the paper should be accepted. However, the structure and writing of the paper was very frustrating to me. The paper just didn*t make much of an attempt to explain and then motivate/analyze the model used. I mean, if I were writing the paper, I would have considered and done many things, such as: - shortening the introduction - shortening the related work - making the presentation of the datasets more succinct - having only one figure that covers most of what is currently in figures 1 and 2 - putting details of what seem more ancillary details like the treatment of background lines objects in an appendix - remove Figure 3, which didn*t convey much to me in the absence of more careful explanation of the model. so that I could motivate, carefully explain, and evaluate the main model in the paper. But here, all these things fill the main text, and we*re told that we have to read the appendices to understand the model.... And the presentation in the appendix is more a dump-all-the-facts presentation than a careful development of the design. Nevertheless, the general direction of the architecture seems sound, and the results look very strong, and there are even some useful ablations in the appendix.