oh ok. so its my understanding that only simple atomic operations can be done without locks with cuda but many parameter updating operations requires some locking structure. So cuda tensors don’t require in general but for your needs they would. Sorry should not have wrote in absolute terms. I would see if you can ask someone from Nvidia on more information on this as they may have more info or may have a way to do it. And also may have some docs for you.