Hi,
If you only read it, you will be fine.
If you have to write, you should make sure no reads happen at the same time but that should be much simpler to enforce.

For forward and step I have a simple solution: overriding the toch.nn.Parameter class.

I am curious how you do that?