Answers.Body: <p>This is sort of a tricky question to answer, by results you could mean 2 possible things. History, which is a culmination of actions, observations and rewards. Or the value func/policy which is the thing you are trying to improve. </p>
  
  <p>With that being said, you for most instances of RL will only be dealing with the last few events in the history, or sampling uniform randomly from it according to your route of exploration. So I don't think this is what you are talking about. </p>
  
  <p>As far as storing your value func/policy OpenAI offers a way to post and to a certain extent store your result. This <a href="https://gym.openai.com/read-only.html" rel="nofollow noreferrer">Readme</a> will show you how to do so. </p>
  
  <p>Edit: OpenAI does not offer anything to store experience out of the box, as I noted in my comment you could pretty easily whip up some code that writes JSON to some file and saving that would be a store of experience. Although, it looks like you already found something that does this in tensorflow. </p>
  
  <p>OpenAI likely hasn't included it up to this point because they don't put a ton into this framework. I would encourage you check out SerpentAI on Github, as it has a lot of features missing in OpenAI.</p>
  
  <p>Hope I answered your question!</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-28T03:09:26.397Z
Answers.Text: By results I mean the "experience" or "progress" during learning. I followed the link you posted but I don't see where it explain how to store that experience. I replaced "results" for "experience", I hope its clear now.
Answers.UserDisplayName: lepe
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-28T06:18:20.163Z
Answers.Text: Ah ok, I do not believe OpenAI offers anything to do this out of the box. You could pretty easily whip up code to write JSON and save the file which would have as much data as you told it to save.
Answers.UserDisplayName: hisairnessag3
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-29T00:47:19.923Z
Answers.Text: Thank you. Do you know why it is not included? Can you edit your answer, so it address the question (as you did with your comment), so I can accept it? (otherwise, it will not be clear why I accepted it).
Answers.UserDisplayName: lepe
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-30T20:51:49.843Z
Answers.Text: It has been done:)
Answers.UserDisplayName: hisairnessag3
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-27T20:37:32.760Z
Answers.LastActivityDate: 2017-09-30T20:51:26.233Z
Answers.LastEditDate: 2017-09-30T20:51:26.233Z
Answers.UserDisplayName: hisairnessag3
Body: <p>I have been playing around with the Pong-v0 game. After few days, the AI was able to beat it. I stopped the script for a moment and ran it again. For my surprise it seems it started all over again. </p>
  
  <p>Where and how does OpenAI store the experience?<br>
  How can I continue the learning procedure in another computer?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-28T04:00:29.333Z
Comments.Text: I think what I'm looking is to learn about [tensorflow' saver function](https://www.tensorflow.org/programmers_guide/saved_model)...
Comments.UserDisplayName: lepe
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-28T04:03:27.023Z
Comments.Text: Related: https://discuss.openai.com/t/how-do-i-save-and-load-a-trained-agent/570
Comments.UserDisplayName: lepe
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-10T22:09:05.993Z
Comments.Text: @Bobs I think we have a winner
Comments.UserDisplayName: hisairnessag3
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-05-10T22:24:25.710Z
Comments.Text: @Bobs Perception!! :)
Comments.UserDisplayName: hisairnessag3
Tags: <open-ai>
Title: OpenAI Gym: How is "experience" stored?
UserDisplayName: lepe
ThreadText: # OpenAI Gym: How is "experience" stored?
  
  (Asked by: username_0 on 2017-09-27)
  
  I have been playing around with the Pong-v0 game. After few days, the AI was able to beat it. I stopped the script for a moment and ran it again. For my surprise it seems it started all over again.
  
  Where and how does OpenAI store the experience?  
  How can I continue the learning procedure in another computer?
  
  **Tags:** open-ai
  > **Comments:**
  > **username_0**: I think what I'm looking is to learn about [tensorflow' saver function](https://www.tensorflow.org/programmers_guide/saved_model)...  
  > **username_0**: Related: https://discuss.openai.com/t/how-do-i-save-and-load-a-trained-agent/570  
  > **username_1**: @Bobs I think we have a winner  
  > **username_1**: @Bobs Perception!! :)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2017-09-27. Score: 4 | Accepted answer
  
  This is sort of a tricky question to answer, by results you could mean 2 possible things. History, which is a culmination of actions, observations and rewards. Or the value func/policy which is the thing you are trying to improve.
  
  With that being said, you for most instances of RL will only be dealing with the last few events in the history, or sampling uniform randomly from it according to your route of exploration. So I don't think this is what you are talking about.
  
  As far as storing your value func/policy OpenAI offers a way to post and to a certain extent store your result. This [Readme](https://gym.openai.com/read-only.html) will show you how to do so.
  
  Edit: OpenAI does not offer anything to store experience out of the box, as I noted in my comment you could pretty easily whip up some code that writes JSON to some file and saving that would be a store of experience. Although, it looks like you already found something that does this in tensorflow.
  
  OpenAI likely hasn't included it up to this point because they don't put a ton into this framework. I would encourage you check out SerpentAI on Github, as it has a lot of features missing in OpenAI.
  
  Hope I answered your question!
  
  > **Comments:**
  > **username_0**: By results I mean the "experience" or "progress" during learning. I followed the link you posted but I don't see where it explain how to store that experience. I replaced "results" for "experience", I hope its clear now.  
  > **username_1**: Ah ok, I do not believe OpenAI offers anything to do this out of the box. You could pretty easily whip up code to write JSON and save the file which would have as much data as you told it to save.  
  > **username_0**: Thank you. Do you know why it is not included? Can you edit your answer, so it address the question (as you did with your comment), so I can accept it? (otherwise, it will not be clear why I accepted it).  
  > **username_1**: It has been done:)  
  
  ---
  
  