Sorry, I am not familiar with the a3c algorithm you mentioned, and I cannot provide you with more help.But I found a newbie guide, hope it helps you.
https://pytorch.org/tutorials/beginner/transformer_tutorial.html 3