Answers.Body: <p>Compression will be lossy, some detailed features in the state will be off calculation.</p>
  <p>A common technique might be using max-pool function or layer (before feeding to policy network if RL here is deep RL).</p>
  <p>Max-pooling is very lossy, use some other classic compression algos such as Zip, Rar but using these classic no-loss compression is weird and extremely slow in the model pipeline.</p>
  <p>Possible solution if allowing lossy data, commonly: Use max-pool (giving out high contrast data), average-pool (giving out blurred data).</p>
  <p>To keep data intact, TensorFlow can compress tensors: <em>&quot;only sacrificing a tiny fraction of model performance. It can compress any floating point tensor to a much smaller sequence of bits.&quot;</em><br/>
  See: <a href="https://github.com/tensorflow/compression" rel="nofollow noreferrer">https://github.com/tensorflow/compression</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-12T07:48:23.907Z
Answers.Text: I think the OP is looking for lossy compression that works with state representations *in use* as input features, in which case zip etc will not work. However, it is not 100% clear from the question. In addition, pooling algorithms only work with dimensional patterns of identical signals (images, audio, financial series etc) - you should make that clear and it looks like the OP wants to exclude those solutions.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-12T13:22:38.283Z
Answers.Text: Exactly, the format of input data is in the form of tabular data, so I'm not sure max pooling or other signal processing solutions will work here.
Answers.UserDisplayName: Saeid Ghafouri
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-12T14:00:13.423Z
Answers.Text: this may be relevant then: https://github.com/tensorflow/compression
Answers.UserDisplayName: Dan D.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-12T04:10:34.977Z
Answers.LastActivityDate: 2021-03-12T13:59:28.890Z
Answers.LastEditDate: 2021-03-12T13:59:28.890Z
Answers.UserDisplayName: Dan D.
Body: <p>I'm working on a problem that involves an RL agent with very large states. These states consist of several pieces of information about the agent. The states are not images, so techniques like convolutional neural networks will not work here.</p>
  <p>Is there some general solutions to reduce/compress the size of the states for reinforcement learning algorithms?</p>
  
Tags: <reinforcement-learning><deep-rl><state-spaces><data-compression>
Title: How can I compress the states of a reinforcement learning agent?
UserDisplayName: Saeid Ghafouri
ThreadText: # How can I compress the states of a reinforcement learning agent?
  
  (Asked by: username_0 on 2021-03-11)
  
  I'm working on a problem that involves an RL agent with very large states. These states consist of several pieces of information about the agent. The states are not images, so techniques like convolutional neural networks will not work here.
  
  Is there some general solutions to reduce/compress the size of the states for reinforcement learning algorithms?
  
  **Tags:** reinforcement-learning, deep-rl, state-spaces, data-compression
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-03-12. Score: 1 | Accepted answer
  
  Compression will be lossy, some detailed features in the state will be off calculation.
  
  A common technique might be using max-pool function or layer (before feeding to policy network if RL here is deep RL).
  
  Max-pooling is very lossy, use some other classic compression algos such as Zip, Rar but using these classic no-loss compression is weird and extremely slow in the model pipeline.
  
  Possible solution if allowing lossy data, commonly: Use max-pool (giving out high contrast data), average-pool (giving out blurred data).
  
  To keep data intact, TensorFlow can compress tensors: *"only sacrificing a tiny fraction of model performance. It can compress any floating point tensor to a much smaller sequence of bits."*  
  See: <https://github.com/tensorflow/compression>
  
  > **Comments:**
  > **username_2**: I think the OP is looking for lossy compression that works with state representations *in use* as input features, in which case zip etc will not work. However, it is not 100% clear from the question. In addition, pooling algorithms only work with dimensional patterns of identical signals (images, audio, financial series etc) - you should make that clear and it looks like the OP wants to exclude those solutions.  
  > **username_0**: Exactly, the format of input data is in the form of tabular data, so I'm not sure max pooling or other signal processing solutions will work here.  
  > **username_1**: this may be relevant then: https://github.com/tensorflow/compression  
  
  ---
  
  