I’ve already tried SGD - it was 3-4 times faster, but convergence was much slower.

Sounds good!

But what I am wondering - why the same model architecture on the same data with same parameters, but on TF2 trains much faster (3-4 times) than my on PyTorch.

Do you mean the overall runtime?
Or is the model more or less the same and the optimizer is mostly responsible for this?
It is possible that some perf improvements can be done on our Adam implementation.