What is d here? A python dictionary?
If it is, then this should be fine and is not considered an inplace operation.
A good way to track this is to enable anomaly mode: torch.autograd.set_detect_anomaly(True) at the beginning of your code to get a more helpful stack trace.
Can you report here what it points to and the code surrounding the faulty bit?