Body: <p>I designed a DQN architecture for some problem. The problem has a parameter <span class="math-container">$m$</span> as the number of clients. In my situation, <span class="math-container">$m$</span> is large, <span class="math-container">$m\in\{100,200,\ldots,1000\}$</span>. For this situation, the number of input ports of the DQN is some few thousand, <span class="math-container">$\{1000, 2000, \ldots, 10000\}$</span>. For some fixed <span class="math-container">$m$</span>, I would like to see the performance of deep Q learning on the performance. So I have to train the DQN for every change that occurs on <span class="math-container">$m$</span> and this should handle thousands of inputs ports for each training. Is this situation familiar in DQN and if not how to solve this issue?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-31T17:56:39.573Z
Comments.Text: By input ports are you talking about the state space being an $m$ dimensional vector, or the action space?
Comments.UserDisplayName: David
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-31T18:47:01.823Z
Comments.Text: Yes it is the state space, and it is a $O(m^2)$-dimensional vector.
Comments.UserDisplayName: zdm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-08-31T18:50:34.190Z
Comments.Text: that is...large. I've never experimented with a state space close to that size. I would imagine you'd first need to have a network capable of handling such large dimensional inputs and even then I'm not sure how feasible it would be to train an agent given the amount of exploration that would need to be done for a state space that large.
Comments.UserDisplayName: David
Tags: <q-learning><dqn><deep-rl><curse-of-dimensionality>
Title: Is it feasible to train a DQN with thousands of input ports?
UserDisplayName: zdm
ThreadText: # Is it feasible to train a DQN with thousands of input ports?
  
  (Asked by: username_0 on 2020-08-31)
  
  I designed a DQN architecture for some problem. The problem has a parameter $m$ as the number of clients. In my situation, $m$ is large, $m\in\{100,200,\ldots,1000\}$. For this situation, the number of input ports of the DQN is some few thousand, $\{1000, 2000, \ldots, 10000\}$. For some fixed $m$, I would like to see the performance of deep Q learning on the performance. So I have to train the DQN for every change that occurs on $m$ and this should handle thousands of inputs ports for each training. Is this situation familiar in DQN and if not how to solve this issue?
  
  **Tags:** q-learning, dqn, deep-rl, curse-of-dimensionality
  > **Comments:**
  > **username_1**: By input ports are you talking about the state space being an $m$ dimensional vector, or the action space?  
  > **username_0**: Yes it is the state space, and it is a $O(m^2)$-dimensional vector.  
  > **username_1**: that is...large. I've never experimented with a state space close to that size. I would imagine you'd first need to have a network capable of handling such large dimensional inputs and even then I'm not sure how feasible it would be to train an agent given the amount of exploration that would need to be done for a state space that large.  
  
  ---
  
  ## 0 answers
  
  