Do you think a similar gain achievable with optimizing the CUDA version?

No idea, I didn’t expect such an improvement with avx.
I think there is no point in messing around with avx, you should probably focus on CUDA since you’re going to train on GPU anyway (I presume).