Hi,
You were right thank you very much ! I do not understand however why the kernel synchronization takes so long though ? Both embeddings take only 0.008 ms to be computed and I call loss.backward() right after. Is there any copy from shared to global memory in the GPU or whatever that takes so much time (approx 0.8ms) ? How could I speed up the training in such a case ?
Thank you