PEP8 (PyTorch uses flake8 for coding style itself) is a good idea, as is the general Python Zen. Don’t write Python like people who don’t like Python.
However, a couple of the PyTorch-specific items I disagree with (in the top two below). My list would be something like:

If you need to use torch.no_grad() somewhere where it isn’t because you’re evaluating something that’s written for training, you should ask yourself if you’re doing it wrong.
Be mindful of loops over tensor dimensions slowing that down. It’s conventional wisdom to avoid these, but there are quite a few legitimate cases for them in PyTorch. I have a half-written section “For loop or not for loop” discussing them somewhere.
Using item and detach for things to keep around longer than the next backward is generally a good idea (e.g. when you record loss history, statistics, …), but be careful to not ruin your graph. (Targeted detach is good in nn.Module subclass code, with torch.no_grad() should be needed very rarely.)
If you write for re-use, the functional / Module split of PyTorch has turned out to be a good idea.
Use functional for stuff without state (unless you have a quick and dirty Sequential).
Don’t use deprecated stuff .data, Tensor(...) and friends, .type (might be me), t.new_.... It’s bad!
Use the documented PyTorch interfaces if you can (e.g. when something from torch.nn.functional shows up in torch for internal reasons.
Benchmark your stuff if you think it’s performance critical. Don’t forget CUDA needs synchronising for valid results.
The JIT will speed up chains of pointwise ops a lot.
C++ will be a bit faster than plain Python but for many cases only ca 10%.
Keep your modules clean.

Best regards
Thomas