Hi. I’m trying to implement DQN with prioritized experience replay. (This paper: https://arxiv.org/pdf/1511.05952.pdf 24). I need to multiply the gradients of the parameters by importance sampling weights before I update the neural network parameters. For my loss function I’ve been using huber loss.
Here is a snippet of my code:
    self.optimizer.zero_grad()
    loss.backward()
    for param, weight in zip(self.qnet.parameters(), sampling_weights_batch):
            param.grad.data *= weight

Instead of redefining the smooth_l1_loss, I go through each parameter gradient and multiply it by the corresponding sampling weight which I stored in my replay memory. Is there a faster way to do this? Thanks!