Well, anything you can find “oficially implemented” is not SOTA.
Think that by the time they port it to pytorch researchers relese new transformers.
A different thing is whether it is worth to spend time on a coarser code or not.
Pytorch transformers are straight-forward.