Towards Reverse-Engineering Black-Box Neural Networks
Many deployed learned models are black boxes: given input, returns output. Internal information about the model, such as the architecture, optimisation procedure, or training data, is not disclosed explicitly as it might contain proprietary information or make the system more vulnerable. This work shows that such attributes of neural networks can be exposed from a sequence of queries. This has multiple implications. On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -- we show that the revealed internal information helps generate more effective adversarial examples against the black box model. On the other hand, this technique can be used for better protection of private content from automatic recognition models using adversarial examples. Our paper suggests that it is actually hard to draw a line between white box and black box models.
 The basic idea is to train a neural network to predict various hyperparameters of a classifier from input-output pairs for that classifier (kennen-o approach). It is surprising that some of these hyperparameters can even be predicted with more than chance accuracy. As a simple example, it*s possible that there are values of batch size for which the classifiers may become indistinguishable, yet Table 2 shows that batch size can be predicted with much higher accuracy than chance. It would be good to provide insights into under what conditions and why hyperparameters can be predicted accurately. That would make the results much more interesting, and may even turn out to be useful for other problems, such as hyperparameter optimization. The selection of the queries for kennen-o is not explained. What is the procedure for selecting the queries? How sensitive is the performance of kennen-o to the choice of the queries? One would expect that there is significant sensitivity, in which case it may even make sense to consider learning to select a sequence of queries to maximize accuracy. In table 3, it would be useful to show the results for kennen-o as well, because Split-E seems to be the more realistic problem setting and kennen-o seems to be a more realistic attack than kennen-i or kennen-io. In the ImageNet classifier family prediction, how different are the various families from each other? Without going through all the references, it is difficult to get a sense of the difficulty of the prediction task for a non-computer-vision reader. Overall the results seem interesting, but without more insights it*s difficult to judge how generally useful they are.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Towards Reverse-Engineering Black-Box Neural Networks
Many deployed learned models are black boxes: given input, returns output. Internal information about the model, such as the architecture, optimisation procedure, or training data, is not disclosed explicitly as it might contain proprietary information or make the system more vulnerable. This work shows that such attributes of neural networks can be exposed from a sequence of queries. This has multiple implications. On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -- we show that the revealed internal information helps generate more effective adversarial examples against the black box model. On the other hand, this technique can be used for better protection of private content from automatic recognition models using adversarial examples. Our paper suggests that it is actually hard to draw a line between white box and black box models.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The basic idea is to train a neural network to predict various hyperparameters of a classifier from input-output pairs for that classifier (kennen-o approach). It is surprising that some of these hyperparameters can even be predicted with more than chance accuracy. As a simple example, it*s possible that there are values of batch size for which the classifiers may become indistinguishable, yet Table 2 shows that batch size can be predicted with much higher accuracy than chance. It would be good to provide insights into under what conditions and why hyperparameters can be predicted accurately. That would make the results much more interesting, and may even turn out to be useful for other problems, such as hyperparameter optimization. The selection of the queries for kennen-o is not explained. What is the procedure for selecting the queries? How sensitive is the performance of kennen-o to the choice of the queries? One would expect that there is significant sensitivity, in which case it may even make sense to consider learning to select a sequence of queries to maximize accuracy. In table 3, it would be useful to show the results for kennen-o as well, because Split-E seems to be the more realistic problem setting and kennen-o seems to be a more realistic attack than kennen-i or kennen-io. In the ImageNet classifier family prediction, how different are the various families from each other? Without going through all the references, it is difficult to get a sense of the difficulty of the prediction task for a non-computer-vision reader. Overall the results seem interesting, but without more insights it*s difficult to judge how generally useful they are.
assistant
 The basic idea is to train a neural network to predict various hyperparameters of a classifier from input-output pairs for that classifier (kennen-o approach). It is surprising that some of these hyperparameters can even be predicted with more than chance accuracy. As a simple example, it*s possible that there are values of batch size for which the classifiers may become indistinguishable, yet Table 2 shows that batch size can be predicted with much higher accuracy than chance. It would be good to provide insights into under what conditions and why hyperparameters can be predicted accurately. That would make the results much more interesting, and may even turn out to be useful for other problems, such as hyperparameter optimization. The selection of the queries for kennen-o is not explained. What is the procedure for selecting the queries? How sensitive is the performance of kennen-o to the choice of the queries? One would expect that there is significant sensitivity, in which case it may even make sense to consider learning to select a sequence of queries to maximize accuracy. In table 3, it would be useful to show the results for kennen-o as well, because Split-E seems to be the more realistic problem setting and kennen-o seems to be a more realistic attack than kennen-i or kennen-io. In the ImageNet classifier family prediction, how different are the various families from each other? Without going through all the references, it is difficult to get a sense of the difficulty of the prediction task for a non-computer-vision reader. Overall the results seem interesting, but without more insights it*s difficult to judge how generally useful they are.