This question wraps together several question I have about how data_ptr is expected to work.
I discovered today that moving the same tensor to the same device twice leads to different .data_ptrs. This makes sense, since could be copying the tensors over separately.
x = torch.tensor([1]).cuda()
x.cpu().data_ptr() == x.cpu().data_ptr()
# False

My issue is that this complicates matters when we expect the data_ptr to be the same. For instance, for models that share weights in different modules, such as word embeddings in input and output layers. In these cases, the input and output embeddings would have two entries in the model state_dict, but both would have the same data_ptr. However, if we use the typical recipe for moving state_dicts across devices:
new_state_dict = {k: v.to(device) for k, v in state_dict.items()}

The input and output embeddings will now have different data_ptrs (in addition to taking up twice the memory).
My question is: is there a built-in method that handles this correctly, such that the moved dictionary of tensors correctly retains tensors with the same data_ptrs?