I updated the code snippet to replicate the OOM issue during model training.
Also args.jbatch should be equal to one so that the matrix is 4000 x 1 x 35000.
That’s my mistake, I forgot to fix the default value (I was using 32x32 tiling before, instead of the 1xvocab_size tiling I’m using now).