Hey guys, these lines incur the error in jit mode. Do you have any idea?
 matching_scores = torch.matmul(bike_key_out.permute(0, 3, 4, 2, 1), taxi_key_out.permute(0, 3, 4, 1, 2))

bt_t_x = torch.matmul(matching_scores, taxi_x.permute(0, 3, 4, 2, 1)).permute(0, 4, 3, 1, 2)

Traceback (most recent call last):
  File "multi_expt.py", line 329, in <module>
    main()
  File "multi_expt.py", line 298, in main
    (bike_loss + taxi_loss).backward()
  File "/home/jindeng/anaconda3/envs/myenv/lib/python3.7/site-packages/torch/tensor.py", line 118, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/jindeng/anaconda3/envs/myenv/lib/python3.7/site-packages/torch/autograd/__init__.py", line 93, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: No grad accumulator for a saved leaf!