I am pushing all the data onto the GPU.
I am using indexing (views) on the matrix.
I cannot get rid of the for-loops because I have different sized one-hot categoricals. (I explain in another post why I need one-hot and not Longs.) One might have 10 categories, one might have 20, so they each have different embedding matrices.
I am concerned that with indexes so small, that data locality will be lost and that I should pre-splice the matrix by categorical so they each are in their own matrix.