Body: <p><strong>Problem Statement:</strong> We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained in the following picture. <a href="https://i.sstatic.net/ibLMb.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ibLMb.png" alt="Problem Statement" /></a></p>
  <p><strong>Objective function 1:</strong>
  Minimise costs = inventory costs + transportation costs + penalty costs + loading/unloading costs</p>
  <ol>
  <li><p><em>Inventory costs</em> = inventory cost at source airport + inventory costs at distribution centres</p>
  </li>
  <li><p><em>Transportation costs</em> = cost of transporting cargo from production centre to source airport (via trucks) + cost of transporting cargo through itineraries (via flight) + cost of transporting cargo from distribution centre to transfer points (via trucks) + cost of transporting cargo from transfer point to customers (via drones)</p>
  </li>
  <li><p><em>Penalty costs</em> = cost of operating flight routes and delay penalty costs</p>
  </li>
  <li><p><em>Loading/unloading costs</em> = cost of loading cargo on trucks at production centres + cost of unloading cargo from trucks at the transfer point</p>
  </li>
  </ol>
  <p><strong>Mathematical Solution (Using IBM CPLEX solver / Docplex):</strong> The complete python code (.ipynb file) with the formulation is present in this <a href="https://drive.google.com/drive/folders/1Vpz7Kpnowkz53Ay6TdK38nEMkFIe-t8W?usp=sharing" rel="nofollow noreferrer">Google Drive Link</a>. This gives an optimal solution.</p>
  <p><strong>Query:</strong> Is there any non-mathematical, non-formulation based method to solve this problem statement? Something on the lines of Reinforcement Learning? If any implementation is also provided, it will be icing on the cake.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-14T12:33:47.927Z
Comments.Text: if it can be formulated as a sequential decision making process then yes RL can be applied to a problem of this nature. The reward would have to be suitably chosen so that maximising the rewards is equivalent to minimising your cost. To me, it is not clear what the actions would be. What do you have control over in this setting? Do you control any of the scheduling aspects? Maybe you can add this detail to your question.
Comments.UserDisplayName: David
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-14T13:05:36.890Z
Comments.Text: @DavidIreland so we are given a fixed number of itineraries (ie. number of flights), production centres, destination airports transfer points and also a fixed amount of demand of each customer. And also the parameters such as transportation and storage costs are fixed. By scheduling aspects, if you mean which flight or truck is allocated to which customer then yes it can be chosen optimally.
Comments.UserDisplayName: Alpha
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-28T10:57:58.890Z
Comments.Text: I am wondering if you are trying to solve the optimization problem for a specific horizon. Is RL taking into consideration the horizon cost as a vector or is it working value by value (realizing the cost at that step)? It is important because I want to be sure the agent has the necessary knowledge in terms of future horizon costs and maybe expected customer load and so on.
Comments.UserDisplayName: Ayska
Tags: <reinforcement-learning><deep-learning><dqn>
Title: Reinforcement Learning applied to Optimisation Problem
UserDisplayName: Alpha
ThreadText: # Reinforcement Learning applied to Optimisation Problem
  
  (Asked by: username_0 on 2021-12-14)
  
  **Problem Statement:** We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained in the following picture. [![Problem Statement](https://i.sstatic.net/ibLMb.png)](https://i.sstatic.net/ibLMb.png)
  
  **Objective function 1:**
  Minimise costs = inventory costs + transportation costs + penalty costs + loading/unloading costs
  
  1. *Inventory costs* = inventory cost at source airport + inventory costs at distribution centres
  2. *Transportation costs* = cost of transporting cargo from production centre to source airport (via trucks) + cost of transporting cargo through itineraries (via flight) + cost of transporting cargo from distribution centre to transfer points (via trucks) + cost of transporting cargo from transfer point to customers (via drones)
  3. *Penalty costs* = cost of operating flight routes and delay penalty costs
  4. *Loading/unloading costs* = cost of loading cargo on trucks at production centres + cost of unloading cargo from trucks at the transfer point
  
  **Mathematical Solution (Using IBM CPLEX solver / Docplex):** The complete python code (.ipynb file) with the formulation is present in this [Google Drive Link](https://drive.google.com/drive/folders/1Vpz7Kpnowkz53Ay6TdK38nEMkFIe-t8W?usp=sharing). This gives an optimal solution.
  
  **Query:** Is there any non-mathematical, non-formulation based method to solve this problem statement? Something on the lines of Reinforcement Learning? If any implementation is also provided, it will be icing on the cake.
  
  **Tags:** reinforcement-learning, deep-learning, dqn
  
  > **Comments:**
  > **username_1**: if it can be formulated as a sequential decision making process then yes RL can be applied to a problem of this nature. The reward would have to be suitably chosen so that maximising the rewards is equivalent to minimising your cost. To me, it is not clear what the actions would be. What do you have control over in this setting? Do you control any of the scheduling aspects? Maybe you can add this detail to your question.  
  > **username_0**: @DavidIreland so we are given a fixed number of itineraries (ie. number of flights), production centres, destination airports transfer points and also a fixed amount of demand of each customer. And also the parameters such as transportation and storage costs are fixed. By scheduling aspects, if you mean which flight or truck is allocated to which customer then yes it can be chosen optimally.  
  > **username_2**: I am wondering if you are trying to solve the optimization problem for a specific horizon. Is RL taking into consideration the horizon cost as a vector or is it working value by value (realizing the cost at that step)? It is important because I want to be sure the agent has the necessary knowledge in terms of future horizon costs and maybe expected customer load and so on.  
  
  ---
  
  ## 0 answers
  
  