You must make your own replay buffer by hand (because the quality of RL sometimes strongly depends on the quality of the buffer, you can’t impose a generic method). However, the most standard approach (not the most efficient) is to save the last 1e6 examples of transitions and to uniformly sample from them. You have one simple example here:

  
      github.com
  
  
    pranz24/pytorch-soft-actor-critic/blob/master/replay_memory.py 7
import random
import numpy as np

class ReplayMemory:
    def __init__(self, capacity):
        self.capacity = capacity
        self.buffer = []
        self.position = 0

    def push(self, state, action, reward, next_state, done):
        if len(self.buffer) < self.capacity:
            self.buffer.append(None)
        self.buffer[self.position] = (state, action, reward, next_state, done)
        self.position = (self.position + 1) % self.capacity

    def sample(self, batch_size):
        batch = random.sample(self.buffer, batch_size)
        state, action, reward, next_state, done = map(np.stack, zip(*batch))
        return state, action, reward, next_state, done



  This file has been truncated. show original