Answers.Body: <p>Two things in this post need to be corrected.  Q-learning is not married to an estimator.  For instance, DQN stands for Deep Q network, a type of Q-Learning that utilizes neural networks.  On the other hand, tabular utilizes a table for estimation.  Second, tabular does not necessarily mean only discrete state spaces.  An algorithm can discretize the space and put it into a table.</p>
  <p>A neural network doesn't necessarily have an issue taking in a discrete input.  When the input is discrete, a common form of input pre-processing is min-max scaling, which often improves network performance.  This fact is especially true when some input features are on different scales.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-01T08:10:08.323Z
Answers.Text: For your anwser, can i conclude that effectively all RL algorithms can handle discrete state spaces ?
Answers.UserDisplayName: Vitor Martins
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-01T13:09:55.383Z
Answers.Text: Yes.  Discrete shouldn’t be a problem for any algorithm.
Answers.UserDisplayName: foreverska
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-01T17:31:35.203Z
Answers.Text: Absolutely, if my post answers your question, please remember to up vote and accept.
Answers.UserDisplayName: foreverska
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-07-02T09:38:50.533Z
Answers.Text: Sorry i can't upvote your anwser. "Thanks for the feedback! You need at least 15 reputation to cast a vote, but your feedback has been recorded."
Answers.UserDisplayName: Vitor Martins
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-06-29T18:35:58.733Z
Answers.LastActivityDate: 2024-06-29T18:35:58.733Z
Answers.UserDisplayName: foreverska
Body: <p>This question come to mind when i was planing to do a benchmark of RL algorithms to my Environment.
  In fact, Q-Learning, SARSA actually only handles with discrete state spaces because they are tabular methods, but Deep RL algorithms like PPO, DDPG and other algorithms that can handle continuous state spaces, can they actually handle discrete state spaces by discretizing the continuous state space into bins and then input them to the Neural Networks ?</p>
  
Tags: <reinforcement-learning><deep-rl><dqn><proximal-policy-optimization><discrete-state-spaces>
Title: Can all RL algorithms learn with discrete state spaces?
UserDisplayName: Vitor Martins
ThreadText: # Can all RL algorithms learn with discrete state spaces?
  
  (Asked by: username_0 on 2024-06-29)
  
  This question come to mind when i was planing to do a benchmark of RL algorithms to my Environment.
  In fact, Q-Learning, SARSA actually only handles with discrete state spaces because they are tabular methods, but Deep RL algorithms like PPO, DDPG and other algorithms that can handle continuous state spaces, can they actually handle discrete state spaces by discretizing the continuous state space into bins and then input them to the Neural Networks ?
  
  **Tags:** reinforcement-learning, deep-rl, dqn, proximal-policy-optimization, discrete-state-spaces
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-06-29. Score: 0 | Accepted answer
  
  Two things in this post need to be corrected. Q-learning is not married to an estimator. For instance, DQN stands for Deep Q network, a type of Q-Learning that utilizes neural networks. On the other hand, tabular utilizes a table for estimation. Second, tabular does not necessarily mean only discrete state spaces. An algorithm can discretize the space and put it into a table.
  
  A neural network doesn't necessarily have an issue taking in a discrete input. When the input is discrete, a common form of input pre-processing is min-max scaling, which often improves network performance. This fact is especially true when some input features are on different scales.
  
  > **Comments:**
  > **username_0**: For your anwser, can i conclude that effectively all RL algorithms can handle discrete state spaces ?  
  > **username_1**: Yes.  Discrete shouldn’t be a problem for any algorithm.  
  > **username_1**: Absolutely, if my post answers your question, please remember to up vote and accept.  
  > **username_0**: Sorry i can't upvote your anwser. "Thanks for the feedback! You need at least 15 reputation to cast a vote, but your feedback has been recorded."  
  
  ---
  
  