But so there are bits of advice in various places, and it might depend on what you want to achieve where the best place to look is.
NVidia’s dev blog (to pick a random article) 22, the PyTorch article on LSTMs 17, I tried to blog a bit about optimizations 30 and a common reduction code pattern 23…
In the end, you probably want to benchmark optimizations and gather some experience around what works and does not work.
Also, you can always look here at the forums or try to ask with concrete code. We have some cuda questions here every once in a while and I certainly have learnt a lot about GPU programming from people around PyTorch.
Best regards
Thomas