Hello,
I’m trying to experiment different configuration with the A3C code posted on GitHub under the following link:
https://github.com/MorvanZhou/pytorch-A3C 1
All my tested are concerned with the script “discrete_A3C.py”
https://github.com/MorvanZhou/pytorch-A3C/blob/master/discrete_A3C.py
Python Version: 3.6.9
Torch Version: 1.4.0
If I keep everything as in the original code the memory usage is consistent and does not increase over time. Which can be shown by plotting the memory usage graph using the command:

mprof run --multiprocess discrete_A3C.py

However, I tried to change the NN architecture to be as follows:
    def __init__(self, s_dim, a_dim):
        super(Net, self).__init__()
        self.s_dim = s_dim
        self.a_dim = a_dim

        self.fc1 = nn.Linear(s_dim, 128)
        self.fc2 = nn.Linear(128, 128)

        self.pi1 = nn.Linear(128, 64)
        self.pi2 = nn.Linear(64, a_dim)  # Q values for each action are output

        self.v1 = nn.Linear(128, 128)
        self.v2 = nn.Linear(128, 1)
        set_init([self.pi1, self.pi2, self.fc1, self.fc2, self.v1, self.v2])
        self.distribution = torch.distributions.Categorical

    def forward(self, x):
        x = F.relu6(self.fc1(x))
        x = F.relu6(self.fc2(x))

        pi1 = F.relu6(self.pi1(x))
        logits = self.pi2(pi1)

        v1 = F.relu6(self.v1(x))
        values = self.v2(v1)
        return logits, values

re-running the same command as before however it can be noticed a linear increase of memory usage overtime.
Please check the image below where the figure on the left is from running the original code, and the figure on the right is from the modified architecture.

image3706×991 292 KB

I tried to debug this myself during the past week but can not come up with any clues, if anyone can provide any insights would be of valuable help.