Hello!
I try to build a distributed RL with PyTorch RPC, but I got a problem that I can’t use my GPU power fully. Here is my code structure. They are two parts, learner and actor.
class Learner:
    
    def __init__(self):
        self.policy = Policy()
        self.buffer = container()
        run_multiple_actor()
    
    def learning_loop(self):
        "backward & optimize"
        while True:
            get_trans_from_buffer()
            computing_loss()
            backward()
            optimizer.step()
    
    def get_action(self, state):
        action = self.policy(state)
        insert_to_buffer(state,action)
        return action

class Actor:
    
    def __init__(self):
        self.env = env
        self.learner_rref = rref
    
    def act_loop(self):
        while True:
            state = self.env.step(action)
            action = self.learner_rref.run_sync().get_action(state)

For learner, after initating, it runs in learning_loop. For actor, after initating, it runs in act_loop and call Learner’s get_action from remote.
The question is that could get_action threads run simultaneously on GPU? If they can, tt seems that as long as I run enough actors, I fully use my GPU power. However, after adding serveal actors, the volatile of GPU stop inscreasing and stays in a low level(eg. 20% or 30%). And I don’t think it’s a problem of my CPU cores. I have enough CPU to make all actors run simultaneously.
Could anyone point out what’s the problem of my code. I am new to PyTorch PRC, help me please.