Solved by KFrank in post #2 
              
              
                Hi Fangyu! 

I’m not sure that I understand your exact use case, but the beta-version 
torch.autograd.functional.jacobian() might be what you want: 
>>> torch.__version__
'1.7.1'
>>> def my_model (x):
...     return torch.arange (2 * x.numel()).reshape ((2, -1)) * x * x
...
>>> x = torch.tensor ([2.…