I traced it back to _utils.pin_memory.pin_memory, which uses an if/else statement to outline a bunch of options for the pinning. One of these options checks if the object is a collections.abc.Mapping, and if so, returns a dict with each item pinned. Since my custom object is a dict subclass, this is triggering instead of my class’s custom pin_memory.
I’ve monkey-patched the function on my own PyTorch build, so I’m good, but it might be worth fixing for the library. Maybe you could modify that function so that it exclusively triggers if the object in question is a dict.