raharth:

I’m sorry, I didn’t mean it as an insult! 


Lol… non taken man! It must be my dark sense of humor 
As for the recurrent model; I’ve experimented yesterday with OpenAI game ‘CartPole-v0’, and a basic version of Policy Gradient to keep things simple until I understand the dynamics of recurrent networks in RL and they’re trained!
I’m going to  benchmark it against this linear model, which solves the environment in 7 seconds.
class PGNet(nn.Module):
    r"""Plain vanilla policy gradient network."""

    def __init__(self, obs_size, act_size, hid_size=128):
        super().__init__()
        self.fc1 = nn.Linear(obs_size, hid_size)
        self.output = nn.Linear(hid_size, act_size, bias=True)

    def forward(self, x):
        """Feed forward."""
        y = F.relu(self.fc1(x))
        return self.output(y)


Benchmark code on :DeepRL/05_PolicyGradient at main · ayjabri/DeepRL · GitHub
I’ll post the logic and code later today…tbc