Yes, that is because it switches between native and cudnn. If you use long tensors (do use torch.tensor(..., dtype=torch.long), not the constructor…), you’ll always get the native implementation.
We are looking to improve this situation 41.
Best regards
Thomas