Body: <p>I am trying to define the number of states and possible actions for a reinforcement learning problem that I want to solve with Q-learning, but I am a bit confused, as I'm totally new to reinforcement learning.</p>
  <p>The problem I'm trying to solve is to assign different groups with people in the sample group having sequential numbers. Let's say there are three in each group.</p>
  <p>Group 1, Group2, Group3.</p>
  <pre><code>1:{&quot;group: Group1, &quot;number&quot;: 1},
  2:{&quot;group: Group2, &quot;number&quot;: 2},
  3:{&quot;group: Group3, &quot;number&quot;: 3},
  4:{&quot;group: Group2, &quot;number&quot;: 4},
  5:{&quot;group: Group1, &quot;number&quot;: 5},
  6:{&quot;group: Group3, &quot;number&quot;: 6},
  7:{&quot;group: Group3, &quot;number&quot;: 7},
  8:{&quot;group: Group2, &quot;number&quot;: 8},
  9:{&quot;group: Group1, &quot;number&quot;: 9},
  </code></pre>
  <p>An optimal output will be a case where the numbers are sequential with respect to the group. For example, all in group1 should have number 1, 2, 3, or 4,5,6 or 7, 8, 9. and not 1, 5, 9 as in the dictionary above.</p>
  <p>In other words,  group1, group2, group3 represent the group ids, which means I have 3 groups. The number represents seat numbers. All in each of the groups need to sit close to each other e.g seat numbers 1,2,3 or 4,5,6, or 7,8,9.</p>
  <p>I am wondering if the possible state will be all possible combinations of the group and numbers, in which case it will be 1680 and the possible action is the number of numbers to swap to get the desired output which is 9.</p>
  <p>Any useful information will be very much appreciated.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-14T19:27:39.133Z
Comments.Text: Hello. Welcome to AI SE. I don't fully understand your problem even with your example. Particularly, I don't understand this description "assign different groups with people in the sample group having sequential numbers.". What is a "sample group" here? It seems that you have 3 groups. How are these groups different from the "sample group"? What do these sequences of numbers 1, 2, 3 or 7, 8, 9 represent in your problem? Please, edit your post to clarify all these points.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-15T19:14:51.757Z
Comments.Text: @nbro thank you for your comment. I have added more information. Hope it is clearer now. The same group is Group1 or Group2, or Group3. The numbers can be 1,2,3 or 4,5,6 or 5,7,6 for any of the groups.
Comments.UserDisplayName: SAB
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-15T19:17:30.510Z
Comments.Text: Thank you for the edit. It is easier to understand your goal state here, but not easy to understand the rules of your environment. It seems trivial to sort by group id and then assign "seat numbers" sequentially, and that appears to be a perfect solution. As you are only asking about the state space, that may not be a problem, but I think you need to give a little bit more background so that readers understand why you want a state representation at all
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-15T19:25:06.690Z
Comments.Text: Also I think you need to be clearer on whether the goal is simply that a group must consist of sequential number assignments, or whether that sequence relates to the row (person?) id. E.g. if it is the former, then your starting state could be `123213321` and an acceptable end state might be `333111222` (there are 6 such end states, and yes I count 1280 states overall using this representation)
Comments.UserDisplayName: Neil Slater
Tags: <reinforcement-learning><q-learning><state-spaces>
Title: Defining states and possible actions in Q learning
UserDisplayName: SAB
ThreadText: # Defining states and possible actions in Q learning
  
  (Asked by: username_0 on 2021-07-14)
  
  I am trying to define the number of states and possible actions for a reinforcement learning problem that I want to solve with Q-learning, but I am a bit confused, as I'm totally new to reinforcement learning.
  
  The problem I'm trying to solve is to assign different groups with people in the sample group having sequential numbers. Let's say there are three in each group.
  
  Group 1, Group2, Group3.
  
  ```
  1:{"group: Group1, "number": 1},
  2:{"group: Group2, "number": 2},
  3:{"group: Group3, "number": 3},
  4:{"group: Group2, "number": 4},
  5:{"group: Group1, "number": 5},
  6:{"group: Group3, "number": 6},
  7:{"group: Group3, "number": 7},
  8:{"group: Group2, "number": 8},
  9:{"group: Group1, "number": 9},
  
  ```
  
  An optimal output will be a case where the numbers are sequential with respect to the group. For example, all in group1 should have number 1, 2, 3, or 4,5,6 or 7, 8, 9. and not 1, 5, 9 as in the dictionary above.
  
  In other words, group1, group2, group3 represent the group ids, which means I have 3 groups. The number represents seat numbers. All in each of the groups need to sit close to each other e.g seat numbers 1,2,3 or 4,5,6, or 7,8,9.
  
  I am wondering if the possible state will be all possible combinations of the group and numbers, in which case it will be 1680 and the possible action is the number of numbers to swap to get the desired output which is 9.
  
  Any useful information will be very much appreciated.
  
  **Tags:** reinforcement-learning, q-learning, state-spaces
  
  > **Comments:**
  > **username_1**: Hello. Welcome to AI SE. I don't fully understand your problem even with your example. Particularly, I don't understand this description "assign different groups with people in the sample group having sequential numbers.". What is a "sample group" here? It seems that you have 3 groups. How are these groups different from the "sample group"? What do these sequences of numbers 1, 2, 3 or 7, 8, 9 represent in your problem? Please, edit your post to clarify all these points.  
  > **username_0**: @username_1 thank you for your comment. I have added more information. Hope it is clearer now. The same group is Group1 or Group2, or Group3. The numbers can be 1,2,3 or 4,5,6 or 5,7,6 for any of the groups.  
  > **username_2**: Thank you for the edit. It is easier to understand your goal state here, but not easy to understand the rules of your environment. It seems trivial to sort by group id and then assign "seat numbers" sequentially, and that appears to be a perfect solution. As you are only asking about the state space, that may not be a problem, but I think you need to give a little bit more background so that readers understand why you want a state representation at all  
  > **username_2**: Also I think you need to be clearer on whether the goal is simply that a group must consist of sequential number assignments, or whether that sequence relates to the row (person?) id. E.g. if it is the former, then your starting state could be `123213321` and an acceptable end state might be `333111222` (there are 6 such end states, and yes I count 1280 states overall using this representation)  
  
  ---
  
  ## 0 answers
  
  