Search-Guided, Lightly-supervised Training of Structured Prediction Energy Networks
In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching through the entire output space to find the best output with respect to this reward function is typically intractable. In this paper, we instead use efficient truncated randomized search in this reward function to train structured prediction energy networks (SPENs), which provide efficient test-time inference using gradient-based search on a smooth, learned representation of the score landscape, and have previously yielded state-of-the-art results in structured prediction. In particular, this truncated randomized search in the reward function yields previously unknown local improvements, providing effective supervision to SPENs, avoiding their traditional need for labeled training data.
 The paper proposes to use a reward function to guide the learning of energy-based models for structured prediction. The idea is to update the energy function based on a random search algorithm guided by a reward function. At each iteration, the SPEN proposes a solution, then a better one is found by the search algorithm, and the energy function is updated accordingly. Experiments are made on three use-cases and show that this method is able to outperform other training algorithms for SPENs. In term of model, the proposed algorithm is interesting since it can allow us to learn from weakly supervised datasets (i.e a reward function is enough). Note that in Section 3, the reward function R is never properly defined which would be nice. The algorithm is quite simple and well presented in the paper. The fact that it is based on a margin could be discussed a little bit more since the effect of the margin is not clear in the paper (the value of alpha). Moreover, the structured prediction problem has already been handled as the maximization of a reward function using RL techniques (see works by H. Daume, and works by F. Maes) and the interest of this approach w.r.t these papers is not clear to me. A clear discussion on that point (and experimental comparison) would be nice. The experimental section could be improved. First, the experiments on multi-label classification do not provide any comparison with SoTA methods while the two other use-cases provide some comparisons. Moreover, as far as I understand, the different use-cases could be fully supervised, and different reward functions could be defined. So investigating more deeply the consequences of the nature of the supervision/reward on these use-cases could be interesting and strengthen the paper. Moreover, training sets are very small and it is difficult to know if this method can work on large-scale problems. Pro: * interesting algorithm for structured prediction (base on reward) * interesting results on some (toy) use-cases Cons: * Lack of discussion on the positive/negative point of the approach w.r.t SoTA, and on the influence of the reward function * Lack of experimental comparisons * Only toy (but complicated) problems with limited training sets
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Search-Guided, Lightly-supervised Training of Structured Prediction Energy Networks
In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching through the entire output space to find the best output with respect to this reward function is typically intractable. In this paper, we instead use efficient truncated randomized search in this reward function to train structured prediction energy networks (SPENs), which provide efficient test-time inference using gradient-based search on a smooth, learned representation of the score landscape, and have previously yielded state-of-the-art results in structured prediction. In particular, this truncated randomized search in the reward function yields previously unknown local improvements, providing effective supervision to SPENs, avoiding their traditional need for labeled training data.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper proposes to use a reward function to guide the learning of energy-based models for structured prediction. The idea is to update the energy function based on a random search algorithm guided by a reward function. At each iteration, the SPEN proposes a solution, then a better one is found by the search algorithm, and the energy function is updated accordingly. Experiments are made on three use-cases and show that this method is able to outperform other training algorithms for SPENs. In term of model, the proposed algorithm is interesting since it can allow us to learn from weakly supervised datasets (i.e a reward function is enough). Note that in Section 3, the reward function R is never properly defined which would be nice. The algorithm is quite simple and well presented in the paper. The fact that it is based on a margin could be discussed a little bit more since the effect of the margin is not clear in the paper (the value of alpha). Moreover, the structured prediction problem has already been handled as the maximization of a reward function using RL techniques (see works by H. Daume, and works by F. Maes) and the interest of this approach w.r.t these papers is not clear to me. A clear discussion on that point (and experimental comparison) would be nice. The experimental section could be improved. First, the experiments on multi-label classification do not provide any comparison with SoTA methods while the two other use-cases provide some comparisons. Moreover, as far as I understand, the different use-cases could be fully supervised, and different reward functions could be defined. So investigating more deeply the consequences of the nature of the supervision/reward on these use-cases could be interesting and strengthen the paper. Moreover, training sets are very small and it is difficult to know if this method can work on large-scale problems. Pro: * interesting algorithm for structured prediction (base on reward) * interesting results on some (toy) use-cases Cons: * Lack of discussion on the positive/negative point of the approach w.r.t SoTA, and on the influence of the reward function * Lack of experimental comparisons * Only toy (but complicated) problems with limited training sets
assistant
 The paper proposes to use a reward function to guide the learning of energy-based models for structured prediction. The idea is to update the energy function based on a random search algorithm guided by a reward function. At each iteration, the SPEN proposes a solution, then a better one is found by the search algorithm, and the energy function is updated accordingly. Experiments are made on three use-cases and show that this method is able to outperform other training algorithms for SPENs. In term of model, the proposed algorithm is interesting since it can allow us to learn from weakly supervised datasets (i.e a reward function is enough). Note that in Section 3, the reward function R is never properly defined which would be nice. The algorithm is quite simple and well presented in the paper. The fact that it is based on a margin could be discussed a little bit more since the effect of the margin is not clear in the paper (the value of alpha). Moreover, the structured prediction problem has already been handled as the maximization of a reward function using RL techniques (see works by H. Daume, and works by F. Maes) and the interest of this approach w.r.t these papers is not clear to me. A clear discussion on that point (and experimental comparison) would be nice. The experimental section could be improved. First, the experiments on multi-label classification do not provide any comparison with SoTA methods while the two other use-cases provide some comparisons. Moreover, as far as I understand, the different use-cases could be fully supervised, and different reward functions could be defined. So investigating more deeply the consequences of the nature of the supervision/reward on these use-cases could be interesting and strengthen the paper. Moreover, training sets are very small and it is difficult to know if this method can work on large-scale problems. Pro: * interesting algorithm for structured prediction (base on reward) * interesting results on some (toy) use-cases Cons: * Lack of discussion on the positive/negative point of the approach w.r.t SoTA, and on the influence of the reward function * Lack of experimental comparisons * Only toy (but complicated) problems with limited training sets