I have a fixed tensor that needs to be moved to GPU according to DataParallel’s whims, but I don’t want to bloat the state_dict with this tensor (since it never changes).  I can’t seem to figure out a good solution.  [for a little more context, this tensor is the weight in F.conv_transpose2d]  The things I’ve considered:

override _apply - for single GPU, this seems to work fine, but for multi GPU, DataParallel seems to rely on replicate… which doesn’t use _apply and instead iterates over all buffers and parameters [the same things that goes into the state_dict]
register_buffer (currently using) - code runs, but state_dict is bloated with unnecesary crap; also if this tensor is ever modified (accidentally or otherwise), entire network is silently corrupted
custom code for saving checkpoints by manually removing this tensor from state_dict every single time - ugly
move tensor to appropriate GPU during every forward pass - probably slow
construct tensor during every forward pass - probably even slower
any other ideas?