I don’t remember exactly what I read but I believe it stated that the pytorch way to use weight decay was different from the original paper. I don’t think it was in relation to tensorflow.
You probably know much more than me on this subject 
blog post L2 != weight decay 30