Neural Reverse Engineering of Stripped Binaries
We address the problem of reverse engineering of stripped executables which contain no debug information. This is a challenging problem because of the low amount of syntactic information available in stripped executables, and due to the diverse assembly code patterns arising from compiler optimizations. We present a novel approach for predicting procedure names in stripped executables. Our approach combines static analysis with encoder-decoder-based models. The main idea is to use static analysis to obtain enriched representations of API call sites; encode a set of sequences of these call sites by traversing the Control-Flow Graph; and finally, attend to the encoded sequences while decoding the target name. Our evaluation shows that our model performs predictions that are difficult and time consuming for humans, while improving on the state-of-the-art by 20%.
= Summary Nero, a method for naming procedures when decompiling binary executables is presented. It consists of a static analysis-based approach to extract a set of sequences of calls to (named) library methods present in the procedure under analysis, and a (mostly standard) seq2seq architecture to predict the procedure name as a sequence of subtokens. Experiments on real-world data show that the method performs well and outperforms some baselines. = Strong/Weak Points + The method seems to work in practice + The writing is reasonably clear, and it feels sufficiently precise to reproduce the results (given enough time) - Little novelty on modeling side (straightforward extension of seq2seq) and data extraction (straightforward application of control flow graphs) - The central novelty in the analysis (extracting *kinds*) seems to contribute very little to results (cf. Tab. 2) - Bad fit for ICLR, as the paper is written with a focus on the PL analysis side = Recommendation Overall, I believe that this is an application with little novelty, and few lessons to be learned for the ICLR audience - a PL or Software Engineering conference (such as VMCAI or FSE) may be a better fit for this work. Hence, I would tend to reject this in favour of more ML-heavy papers. = Minor Comments * It may make sense to compare to https://2019.ase-conferences.org/details/ase-2019-papers/49/DIRE-A-Neural-Approach-to-Decompiled-Identifier-Renaming, which was very recently published (at least I only saw this after the ICLR submission deadline closed, so this is no way meant as a negative) * page 3, second to last paragraph: *basic blocks in ,* but is undefined? * page 4, *Combined Example*: I found the writing slightly misleading here: the extracted path does not show that *(ii) getaddrinfo ... stores the results _in the previously initialized memory_* or *(iii) configure _it_ to be a TCP socket.* The extracted information provides no information about the flow of information or values, which seems to be a weakness of the method. * page 4, last par: - this should be up to ? * page 5, footnote 1: *each call site may participates* -> participate * page 8: *In our approach, there is no need to manually define rules for building the graph as in Allamanis et al. (2018)* - this is confusing to me, as I believe the rules for building the call graph (and abstracting kinds!) to be manually defined?
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Neural Reverse Engineering of Stripped Binaries
We address the problem of reverse engineering of stripped executables which contain no debug information. This is a challenging problem because of the low amount of syntactic information available in stripped executables, and due to the diverse assembly code patterns arising from compiler optimizations. We present a novel approach for predicting procedure names in stripped executables. Our approach combines static analysis with encoder-decoder-based models. The main idea is to use static analysis to obtain enriched representations of API call sites; encode a set of sequences of these call sites by traversing the Control-Flow Graph; and finally, attend to the encoded sequences while decoding the target name. Our evaluation shows that our model performs predictions that are difficult and time consuming for humans, while improving on the state-of-the-art by 20%.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



= Summary Nero, a method for naming procedures when decompiling binary executables is presented. It consists of a static analysis-based approach to extract a set of sequences of calls to (named) library methods present in the procedure under analysis, and a (mostly standard) seq2seq architecture to predict the procedure name as a sequence of subtokens. Experiments on real-world data show that the method performs well and outperforms some baselines. = Strong/Weak Points + The method seems to work in practice + The writing is reasonably clear, and it feels sufficiently precise to reproduce the results (given enough time) - Little novelty on modeling side (straightforward extension of seq2seq) and data extraction (straightforward application of control flow graphs) - The central novelty in the analysis (extracting *kinds*) seems to contribute very little to results (cf. Tab. 2) - Bad fit for ICLR, as the paper is written with a focus on the PL analysis side = Recommendation Overall, I believe that this is an application with little novelty, and few lessons to be learned for the ICLR audience - a PL or Software Engineering conference (such as VMCAI or FSE) may be a better fit for this work. Hence, I would tend to reject this in favour of more ML-heavy papers. = Minor Comments * It may make sense to compare to https://2019.ase-conferences.org/details/ase-2019-papers/49/DIRE-A-Neural-Approach-to-Decompiled-Identifier-Renaming, which was very recently published (at least I only saw this after the ICLR submission deadline closed, so this is no way meant as a negative) * page 3, second to last paragraph: *basic blocks in ,* but is undefined? * page 4, *Combined Example*: I found the writing slightly misleading here: the extracted path does not show that *(ii) getaddrinfo ... stores the results _in the previously initialized memory_* or *(iii) configure _it_ to be a TCP socket.* The extracted information provides no information about the flow of information or values, which seems to be a weakness of the method. * page 4, last par: - this should be up to ? * page 5, footnote 1: *each call site may participates* -> participate * page 8: *In our approach, there is no need to manually define rules for building the graph as in Allamanis et al. (2018)* - this is confusing to me, as I believe the rules for building the call graph (and abstracting kinds!) to be manually defined?
assistant
= Summary Nero, a method for naming procedures when decompiling binary executables is presented. It consists of a static analysis-based approach to extract a set of sequences of calls to (named) library methods present in the procedure under analysis, and a (mostly standard) seq2seq architecture to predict the procedure name as a sequence of subtokens. Experiments on real-world data show that the method performs well and outperforms some baselines. = Strong/Weak Points + The method seems to work in practice + The writing is reasonably clear, and it feels sufficiently precise to reproduce the results (given enough time) - Little novelty on modeling side (straightforward extension of seq2seq) and data extraction (straightforward application of control flow graphs) - The central novelty in the analysis (extracting *kinds*) seems to contribute very little to results (cf. Tab. 2) - Bad fit for ICLR, as the paper is written with a focus on the PL analysis side = Recommendation Overall, I believe that this is an application with little novelty, and few lessons to be learned for the ICLR audience - a PL or Software Engineering conference (such as VMCAI or FSE) may be a better fit for this work. Hence, I would tend to reject this in favour of more ML-heavy papers. = Minor Comments * It may make sense to compare to https://2019.ase-conferences.org/details/ase-2019-papers/49/DIRE-A-Neural-Approach-to-Decompiled-Identifier-Renaming, which was very recently published (at least I only saw this after the ICLR submission deadline closed, so this is no way meant as a negative) * page 3, second to last paragraph: *basic blocks in ,* but is undefined? * page 4, *Combined Example*: I found the writing slightly misleading here: the extracted path does not show that *(ii) getaddrinfo ... stores the results _in the previously initialized memory_* or *(iii) configure _it_ to be a TCP socket. * The extracted information provides no information about the flow of information or values, which seems to be a weakness of the method. * page 4, last par: - this should be up to ? * page 5, footnote 1: *each call site may participates* -> participate * page 8: *In our approach, there is no need to manually define rules for building the graph as in Allamanis et al. (2018)* - this is confusing to me, as I believe the rules for building the call graph (and abstracting kinds!) to be manually defined?