I cannot comment of Tensorflow and I think their board or stackoverflow would be a better place to dig into this framework.
However, PyTorch uses C/C++/CUDA as the underlying functions.
Python is quite popular in the scientific community as it’s really easy to create your scripts using this language.
The heavy workload is performed by highly optimized code, so that you usually don’t have to worry about it.
Some core devs can give you definitely more insight on this topic.
Also. have a look at their paper 10.