Hi, @smth  thanks a lot for your response. Based on your suggestions, I tried the following things bellow:

I tried to remove every x.detach_() in the foregoing code and simply called loss.backward(), then the model works as the normal condition and my customized backward() function are not called.

def backward(self, grad_output):
    	for m in reversed(self.module_list_1):
    		grad_output = m.backward(grad_output)
    	grad_output.view(-1, 50, 4, 4)
    	for n in reversed(self.module_list_0):
    		grad_output = n.backward(grad_output)
    	return grad_output


When I tried to call the customized backward() function in this manner, self.network.backward(grad_output=${RANDOM_VARIABLE}), then I got this error:

    grad_output = m.backward(grad_output)
  File "/home/usr/anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 262, in __getattr__
    type(self).__name__, name))
AttributeError: 'Linear' object has no attribute 'backward'

Dose this mean I need to overwrite the corresponding nn.Module by myself with a backward function, or there is a way that the current Module will do that for me? So, in general, I only want to call the backward process layer by layer, after getting the gradient of each layer, I just want to do a simple check (e.g. the value of a certain variable) and to determine if I need to do or just skip the remaining backward process.