Solving Packing Problems by Conditional Query Learning
Neural Combinatorial Optimization (NCO) has shown the potential to solve traditional NP-hard problems recently. Previous studies have shown that NCO outperforms heuristic algorithms in many combinatorial optimization problems such as the routing problems. However, it is less efficient for more complicated problems such as packing, one type of optimization problem that faces mutual conditioned action space. In this paper, we propose a Conditional Query Learning (CQL) method to handle the packing problem for both 2D and 3D settings. By embedding previous actions as a conditional query to the attention model, we design a fully end-to-end model and train it for 2D and 3D packing via reinforcement learning respectively. Through extensive experiments, the results show that our method could achieve lower bin gap ratio and variance for both 2D and 3D packing. Our model improves 7.2% space utilization ratio compared with genetic algorithm for 3D packing (30 boxes case), and reduces more than 10% bin gap ratio in almost every case compared with extant learning approaches. In addition, our model shows great scalability to packing box number. Furthermore, we provide a general test environment of 2D and 3D packing for learning algorithms. All source code of the model and the test environment is released.
This paper proposes an end-to-end deep reinforcement learning-based algorithm for the 2D and 3D bin packing problems. Its main contribution is conditional query learning (CQL) which allows effective decision over mutually conditioned action spaces through policy expressed as a sequence of conditional distributions. Efficient neural architectures for modeling of such a policy is proposed. Experiments validate the effectiveness of the algorithm through comparisons with genetic algorithm and vanilla RL baselines. Overall, the paper provides a solid contribution by proposing a new RL-based algorithm for the bin packing problem. In particular, the dense reward design for the MDP is quite interesting. I also think solving the 2D and 3D bin packing problem via RL is already quite valuable as an application. However, I do not think the proposed method is novel enough. Most importantly, I perceive the concept of conditional query learning indifferent from the autoregressive modeling of the policy (with conditionally masked inputs). Although the authors divide the processing of each bin into three steps of MDP (not strictly), the steps can be merged into one without any change in the environment. Since other parts of the algorithm also come from the existing literature (except the reward design), I think this point is crucial for the paper to be published at the conference. Minor comment: - I think the genetic algorithm described in the experiments should be equipped with more details, i.e., the type of processor used and elapsed time. - In table 2, I think bold numbers for the lowest variance of performance is very confusing since it does not necessarily mean a better algorithm.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Solving Packing Problems by Conditional Query Learning
Neural Combinatorial Optimization (NCO) has shown the potential to solve traditional NP-hard problems recently. Previous studies have shown that NCO outperforms heuristic algorithms in many combinatorial optimization problems such as the routing problems. However, it is less efficient for more complicated problems such as packing, one type of optimization problem that faces mutual conditioned action space. In this paper, we propose a Conditional Query Learning (CQL) method to handle the packing problem for both 2D and 3D settings. By embedding previous actions as a conditional query to the attention model, we design a fully end-to-end model and train it for 2D and 3D packing via reinforcement learning respectively. Through extensive experiments, the results show that our method could achieve lower bin gap ratio and variance for both 2D and 3D packing. Our model improves 7.2% space utilization ratio compared with genetic algorithm for 3D packing (30 boxes case), and reduces more than 10% bin gap ratio in almost every case compared with extant learning approaches. In addition, our model shows great scalability to packing box number. Furthermore, we provide a general test environment of 2D and 3D packing for learning algorithms. All source code of the model and the test environment is released.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes an end-to-end deep reinforcement learning-based algorithm for the 2D and 3D bin packing problems. Its main contribution is conditional query learning (CQL) which allows effective decision over mutually conditioned action spaces through policy expressed as a sequence of conditional distributions. Efficient neural architectures for modeling of such a policy is proposed. Experiments validate the effectiveness of the algorithm through comparisons with genetic algorithm and vanilla RL baselines. Overall, the paper provides a solid contribution by proposing a new RL-based algorithm for the bin packing problem. In particular, the dense reward design for the MDP is quite interesting. I also think solving the 2D and 3D bin packing problem via RL is already quite valuable as an application. However, I do not think the proposed method is novel enough. Most importantly, I perceive the concept of conditional query learning indifferent from the autoregressive modeling of the policy (with conditionally masked inputs). Although the authors divide the processing of each bin into three steps of MDP (not strictly), the steps can be merged into one without any change in the environment. Since other parts of the algorithm also come from the existing literature (except the reward design), I think this point is crucial for the paper to be published at the conference. Minor comment: - I think the genetic algorithm described in the experiments should be equipped with more details, i.e., the type of processor used and elapsed time. - In table 2, I think bold numbers for the lowest variance of performance is very confusing since it does not necessarily mean a better algorithm.
assistant
This paper proposes an end-to-end deep reinforcement learning-based algorithm for the 2D and 3D bin packing problems. Its main contribution is conditional query learning (CQL) which allows effective decision over mutually conditioned action spaces through policy expressed as a sequence of conditional distributions. Efficient neural architectures for modeling of such a policy is proposed. Experiments validate the effectiveness of the algorithm through comparisons with genetic algorithm and vanilla RL baselines. Overall, the paper provides a solid contribution by proposing a new RL-based algorithm for the bin packing problem. In particular, the dense reward design for the MDP is quite interesting. I also think solving the 2D and 3D bin packing problem via RL is already quite valuable as an application. However, I do not think the proposed method is novel enough. Most importantly, I perceive the concept of conditional query learning indifferent from the autoregressive modeling of the policy (with conditionally masked inputs). Although the authors divide the processing of each bin into three steps of MDP (not strictly), the steps can be merged into one without any change in the environment. Since other parts of the algorithm also come from the existing literature (except the reward design), I think this point is crucial for the paper to be published at the conference. Minor comment: - I think the genetic algorithm described in the experiments should be equipped with more details, i.e., the type of processor used and elapsed time. - In table 2, I think bold numbers for the lowest variance of performance is very confusing since it does not necessarily mean a better algorithm.