Hi everyone,
I’m working on a project that requires me to have access to each step of backward propagation during the training process. Say I have a 10 layer fully connected neural net (input->fc1->fc2->…->fc10->output), and during the backward process I want something like output.backward()->fc10.backward()->fc9->backward()->…->fc1.backward() in separate steps so that I can get gradient of each layer and check how much time it costs for computing gradient of each layer.
However, for now, when I call loss.backward() using this backward function (https://github.com/pytorch/pytorch/blob/master/torch/autograd/ 31init.py#L46 ), then only the loss variable is contained in the variables, and pushed into the execution engine.
How can I get access to gradient computation process of each parameters (layers) in the network? I really want something like
for layers in reversed(network):
    layer_grad=layers.backward()
    # and at here I can check the time cost of gradient computing of a single layer
   layers.update(layer_grad) # based on certain optimizer

Any information will be appreciate.