Hi alexis,
Thank you for your reply. The following is something I think about it.
As neural style gets its loss in hidden layer. In the original implementation of Torch, take ContentLoss for example:
function ContentLoss:updateGradInput(input, gradOutput)
  if self.mode == 'loss' then
    if input:nElement() == self.target:nElement() then
      self.gradInput = self.crit:backward(input, self.target)
    end
    if self.normalize then
      self.gradInput:div(torch.norm(self.gradInput, 1) + 1e-8)
    end
    self.gradInput:mul(self.strength)
-- This is where the layer loss has taken its effects.
    self.gradInput:add(gradOutput)
  else
    self.gradInput:resizeAs(gradOutput):copy(gradOutput)
  end
  return self.gradInput
end

We can see self.gradInput:add(gradOutput) means the “hidden layer gradient” can be added to the main gradient flow.
However, in pytorch, we don’t bother to add it ourselves. As pytorch’s autograd mechanics,  the gradient of the node will be accumulated automatically. So for the back-propagation of hidden gradient, we just need to overwritten the backward function:
    def backward(self, retain_variables=True):
        self.loss.backward(retain_variables=retain_variables)
        return self.loss

Then the node adds the gradient of self.loss and grad_output and finally back-propagate it.
If any misreading, please tell me!