With recent pytorch versions, you can use register_buffer(persistent=False) to prevent the buffer from appearing in the state_dct. This is present in 1.7 and not present in 1.4, not 100% sure when it appeared.