Body: <p>I have a problem writing a deep q-learning model for a multi-agent game.
  The game has very few states. I mean it only has 10 states at most and around 71 actions which some of them have some correlation.<br />
  My problem is I have to write the whole model from scratch.
  Some identical functions need to be considered twice because the model is confusing and I don't have much time.
  If there has been a supervised learning or a backward temporal difference algorithm I would be rescued.
  I don't want to waste my time rewriting everything from scratch.
  Edit: The set of possible actions in each round is a sampled subset of 71 actions.</p>
  <p>Edit 2: I tried to use multi-agent reinforcement learning but the loss function doesn't decrease at all. Using the network for all players isn't helpful. What should I do? The terminal reward depends on the players and It changes based on their behavior. What should I do?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-10T10:48:40.387Z
Comments.Text: 10 states and 70 actions can be handled with tabular RL
Comments.UserDisplayName: Alberto
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-10T11:04:02.610Z
Comments.Text: A player doesn't know whether the rest of players play which actions, so it basically isn't converging.
Comments.UserDisplayName: Mojtaba Razavi
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-11T07:34:38.017Z
Comments.Text: It's not clear, but you might not have the problem/need for supervised learning algorithm that you think you have. It may be better to explain your project, what you have tried, what happened and why you are stuck on what to do next.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-11T07:37:24.833Z
Comments.Text: It's worth explaining why you don't model or account for other player choices when considering each action. That's more of a game theory thing. But how the game works has a huge impact on which RL approaches work
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-11T07:41:02.037Z
Comments.Text: For clarification, a supervised learning approach is possible if you have some way other than random exploration to set the correct action choice. E.g. a full tree search using minimax. Then you could train a model to predict the policy
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-10-12T13:39:21.780Z
Comments.Text: Terminal reward changes based on the numbers guessed by the players and their behavior in the game. Using a single network for all players isn't useful. The loss only increases. What am I doing wrong?
Comments.UserDisplayName: Mojtaba Razavi
Tags: <reinforcement-learning><supervised-learning>
Title: Is there a suitable supervised learning algorithm to use instead of deep Q-learning?
UserDisplayName: Mojtaba Razavi
ThreadText: # Is there a suitable supervised learning algorithm to use instead of deep Q-learning?
  
  (Asked by: username_0 on 2024-10-10)
  
  I have a problem writing a deep q-learning model for a multi-agent game.
  The game has very few states. I mean it only has 10 states at most and around 71 actions which some of them have some correlation.  
  My problem is I have to write the whole model from scratch.
  Some identical functions need to be considered twice because the model is confusing and I don't have much time.
  If there has been a supervised learning or a backward temporal difference algorithm I would be rescued.
  I don't want to waste my time rewriting everything from scratch.
  Edit: The set of possible actions in each round is a sampled subset of 71 actions.
  
  Edit 2: I tried to use multi-agent reinforcement learning but the loss function doesn't decrease at all. Using the network for all players isn't helpful. What should I do? The terminal reward depends on the players and It changes based on their behavior. What should I do?
  
  **Tags:** reinforcement-learning, supervised-learning
  > **Comments:**
  > **username_1**: 10 states and 70 actions can be handled with tabular RL  
  > **username_0**: A player doesn't know whether the rest of players play which actions, so it basically isn't converging.  
  > **username_2**: It's not clear, but you might not have the problem/need for supervised learning algorithm that you think you have. It may be better to explain your project, what you have tried, what happened and why you are stuck on what to do next.  
  > **username_2**: It's worth explaining why you don't model or account for other player choices when considering each action. That's more of a game theory thing. But how the game works has a huge impact on which RL approaches work  
  > **username_2**: For clarification, a supervised learning approach is possible if you have some way other than random exploration to set the correct action choice. E.g. a full tree search using minimax. Then you could train a model to predict the policy  
  > **username_0**: Terminal reward changes based on the numbers guessed by the players and their behavior in the game. Using a single network for all players isn't useful. The loss only increases. What am I doing wrong?  
  
  ---
  
  ## 0 answers
  
  