Body: <p>In the book &quot;Reinforcement Learning: An Introduction&quot; (2018) Sutton and Barto explain, <a href="http://incompleteideas.net/book/RLbook2020.pdf#page=243" rel="nofollow noreferrer">on page 221</a>, a form of tile coding using <em>hashing</em>, to reduce memory consumption.</p>
  <p>I have two questions about that:</p>
  <ol>
  <li><p>How can this approach reduce memory consumption? Doesn't it just depend on the number of tiles (you have to store one weight for each tile)?</p>
  </li>
  <li><p>They state that there is only a &quot;little loss of performance&quot;. In my understanding, the sense of tile coding (and coarse coding) is, that near-by states have many tiles in common and far-away states have only few tilings in common. With tilings &quot;randomly spread throughout the state space&quot; this isn't the case. How does this <strong>not</strong> influence performance?</p>
  </li>
  </ol>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-22T08:47:56.877Z
Comments.Text: you can look at the implementation [here](http://incompleteideas.net/tiles/tiles3.html)
Comments.UserDisplayName: Brale
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-22T13:20:30.187Z
Comments.Text: I already saw the implementation and understand the concept, but not why it works!
Comments.UserDisplayName: F.M.F.
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-10T21:32:46.523Z
Comments.Text: @ F.M.F.  did you find an answer to this posted question?
Comments.UserDisplayName: user3489173
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-04-11T08:58:44.780Z
Comments.Text: @user3489173 In my understanding, the memory consumption is reduced, by assigning weights to tiles lazily and therefore only storing weights for parts of the state-action space that actually are reached.
Comments.UserDisplayName: F.M.F.
Tags: <reinforcement-learning><function-approximation><sutton-barto><computational-complexity><tile-coding>
Title: When using hashing in tile coding, why are memory requirements reduced and there is only a little loss of performance?
UserDisplayName: F.M.F.
ThreadText: # When using hashing in tile coding, why are memory requirements reduced and there is only a little loss of performance?
  
  (Asked by: username_0 on 2019-05-22)
  
  In the book "Reinforcement Learning: An Introduction" (2018) Sutton and Barto explain, [on page 221](http://incompleteideas.net/book/RLbook2020.pdf#page=243), a form of tile coding using *hashing*, to reduce memory consumption.
  
  I have two questions about that:
  
  1. How can this approach reduce memory consumption? Doesn't it just depend on the number of tiles (you have to store one weight for each tile)?
  2. They state that there is only a "little loss of performance". In my understanding, the sense of tile coding (and coarse coding) is, that near-by states have many tiles in common and far-away states have only few tilings in common. With tilings "randomly spread throughout the state space" this isn't the case. How does this **not** influence performance?
  
  **Tags:** reinforcement-learning, function-approximation, sutton-barto, computational-complexity, tile-coding
  > **Comments:**
  > **username_1**: you can look at the implementation [here](http://incompleteideas.net/tiles/tiles3.html)  
  > **username_0**: I already saw the implementation and understand the concept, but not why it works!  
  > **username_2**: @ F.M.F.  did you find an answer to this posted question?  
  > **username_0**: @user3489173 In my understanding, the memory consumption is reduced, by assigning weights to tiles lazily and therefore only storing weights for parts of the state-action space that actually are reached.  
  
  ---
  
  ## 0 answers
  
  