Nope, loss is not exploding. I was monitoring it, and the values were decreasing. I looks like that (few last batch losses):
2020-08-13 00:28:22 UTC -- tensor(3.1548, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(3.5346, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(3.6171, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(3.0142, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(2.5788, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(2.6942, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(2.9119, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(3.5253, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(3.5954, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(1.8311, device='cuda:0', grad_fn=<AddBackward0>)

2020-08-13 00:28:22 UTC -- tensor(nan, device='cuda:0', grad_fn=<AddBackward0>)