I read some posts and think maybe the problem is that those tensors are not defined as parameters or buffers. For example, maybe they are just defined to do some computation in forward function. Does that make sense to you? And is there a way to solve this problem?
The best way I can find is to use register_buffer. However, it seems that it has to be used when you defined the model.