Answers.Body: <p>If what you mean by a generic reinforcement-learning application is an application that can learn any game (or some games), then you can't do it. Why? Because the goal of each game isn't the same, so you have to adapt the rewards depending on the game. If you just want to make an AI for Connect Four, I suggest you to make a specific RL application for that game.</p>
  
  <p>I want to mention another thing: you shouldn't give a reward based on token's position, because it's hard to know what token's position is the best. Instead, just assign a big reward to the winner. That way, you're generalizing your algorithm, and you're avoiding your AI to focus on the wrong goal.</p>
  
  <p>You have to be careful, you don't know for sure what token's position is the best. Try to give a reward ONLY when a player wins the game, because you can know for sure that winning is the best thing to do. It may sound idiot said like that, but that way, you're ensuring you AI to learn by itself what token's position is the best.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T01:49:07.717Z
Answers.Text: Hey! Thanks for trying to contribute to our site :) Btw, I think you should also mention the case of DQN, if you are familiar with it. Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T03:14:08.727Z
Answers.Text: @nbro♦ Hi! Thanks for your reply! Sadly, I'm not familiar with Deep Q-Networks, but after reading you comment, I read some articles about DQNs. It's really interesting, but I found nothing about how DQNs are able to play many games. But the reward function definitely changes from a game to another, or maybe the goal of each game was the same. For example, the common goal to each game could simply be: surviving.
Answers.UserDisplayName: Dara Kong
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T12:07:59.550Z
Answers.Text: Hi Dara, thanks for your reply. Regarding your second paragraph – what's the difference between "you shouldn't give a reward based on token's position" and "giving a score for tokens that are aligned"? Aren't you saying we should give greater rewards when the tokens are in certain positions, this is the same, no?
Answers.UserDisplayName: mason7663
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T12:09:31.607Z
Answers.Text: nbro, thank you for your continued help (for me and the rest of the community, your assistance is invaluable). Regarding your reply and to clarify: is the only thing that changes from game to game the rewards assigned in the reward function?
Answers.UserDisplayName: mason7663
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T12:11:46.307Z
Answers.Text: I guess a follow-up question to the above main question is: should *I* assign the reward values, or let the AI work out what token positions are the best?
Answers.UserDisplayName: mason7663
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T13:50:51.757Z
Answers.Text: @mason7663 I think the question I asked above would be a good new post on the site. If you want, you can ask this question(s) in a new post: "_Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game?_".
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T13:52:45.983Z
Answers.Text: @mason7663 Regarding your follow-up question, are you asking if you should allow the agent to learn the actual reward function? This is also an interesting question that you can ask separately. In some cases, you are given only rewards and not necessarily a definition of the reward function. So this may be interesting to ask.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T13:53:21.960Z
Answers.Text: Asked here: 
  https://ai.stackexchange.com/questions/20004/in-rl-if-i-assign-the-rewards-for-better-positional-play-the-algorithm-is-lear
Answers.UserDisplayName: mason7663
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T14:00:04.410Z
Answers.Text: @mason7663 Actually, I think this is a different question and it's not completely clear to me what you are actually asking :P
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T18:16:17.747Z
Answers.Text: @mason7663 Hi! I updated my answer, because I haven't understood what you meant by "assigning greater rewards to certain token positions". Anyway, please read my answer again, I'm suggesting something else than "giving a score for tokens that are aligned".
Answers.UserDisplayName: Dara Kong
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-04T01:39:32.197Z
Answers.LastActivityDate: 2020-04-04T18:11:02.480Z
Answers.LastEditDate: 2020-04-04T18:11:02.480Z
Answers.UserDisplayName: Dara Kong
Body: <p>I'm creating an RL application for the game Connect Four. </p>
  
  <p>In general, should I be aiming to create an application that's more generic, which would 'learn' different games, or specific to a particular game (e.g. Connect Four, by assigning greater rewards to certain token positions in the C4 grid)? </p>
  
  <p>Does the difference between the two approaches just come down to adapting their respective reward functions to reward specific achievements or positions (in a board game setting), or something else?</p>
  
Tags: <reinforcement-learning><ai-design><game-ai>
Title: Should I be trying to create a generic or specific (to particular game) reinforcement learning agent?
UserDisplayName: mason7663
ThreadText: # Should I be trying to create a generic or specific (to particular game) reinforcement learning agent?
  
  (Asked by: username_0 on 2020-04-03)
  
  I'm creating an RL application for the game Connect Four.
  
  In general, should I be aiming to create an application that's more generic, which would 'learn' different games, or specific to a particular game (e.g. Connect Four, by assigning greater rewards to certain token positions in the C4 grid)?
  
  Does the difference between the two approaches just come down to adapting their respective reward functions to reward specific achievements or positions (in a board game setting), or something else?
  
  **Tags:** reinforcement-learning, ai-design, game-ai
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-04. Score: 1
  
  If what you mean by a generic reinforcement-learning application is an application that can learn any game (or some games), then you can't do it. Why? Because the goal of each game isn't the same, so you have to adapt the rewards depending on the game. If you just want to make an AI for Connect Four, I suggest you to make a specific RL application for that game.
  
  I want to mention another thing: you shouldn't give a reward based on token's position, because it's hard to know what token's position is the best. Instead, just assign a big reward to the winner. That way, you're generalizing your algorithm, and you're avoiding your AI to focus on the wrong goal.
  
  You have to be careful, you don't know for sure what token's position is the best. Try to give a reward ONLY when a player wins the game, because you can know for sure that winning is the best thing to do. It may sound idiot said like that, but that way, you're ensuring you AI to learn by itself what token's position is the best.
  
  > **Comments:**
  > **username_2**: Hey! Thanks for trying to contribute to our site :) Btw, I think you should also mention the case of DQN, if you are familiar with it. Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game?  
  > **username_1**: @username_2♦ Hi! Thanks for your reply! Sadly, I'm not familiar with Deep Q-Networks, but after reading you comment, I read some articles about DQNs. It's really interesting, but I found nothing about how DQNs are able to play many games. But the reward function definitely changes from a game to another, or maybe the goal of each game was the same. For example, the common goal to each game could simply be: surviving.  
  > **username_0**: Hi username_1, thanks for your reply. Regarding your second paragraph – what's the difference between "you shouldn't give a reward based on token's position" and "giving a score for tokens that are aligned"? Aren't you saying we should give greater rewards when the tokens are in certain positions, this is the same, no?  
  > **username_0**: username_2, thank you for your continued help (for me and the rest of the community, your assistance is invaluable). Regarding your reply and to clarify: is the only thing that changes from game to game the rewards assigned in the reward function?  
  > **username_0**: I guess a follow-up question to the above main question is: should *I* assign the reward values, or let the AI work out what token positions are the best?  
  > **username_2**: @username_0 I think the question I asked above would be a good new post on the site. If you want, you can ask this question(s) in a new post: "_Some people claim that DQN was used to play many games. But what actually happened? Was DQN trained only once (with some data from all games) or was it trained separately for each game? What was common to all those games? Only the architecture? Did the reward function change for each game?_".  
  > **username_2**: @username_0 Regarding your follow-up question, are you asking if you should allow the agent to learn the actual reward function? This is also an interesting question that you can ask separately. In some cases, you are given only rewards and not necessarily a definition of the reward function. So this may be interesting to ask.  
  > **username_0**: Asked here:  https://ai.stackexchange.com/questions/20004/in-rl-if-i-assign-the-rewards-for-better-positional-play-the-algorithm-is-lear  
  > **username_2**: @username_0 Actually, I think this is a different question and it's not completely clear to me what you are actually asking :P  
  > **username_1**: @username_0 Hi! I updated my answer, because I haven't understood what you meant by "assigning greater rewards to certain token positions". Anyway, please read my answer again, I'm suggesting something else than "giving a score for tokens that are aligned".  
  
  ---
  
  