PyTorch uses a library called pybind11 to create Python bindings. You can find the documentation in this https://pybind11.readthedocs.io/en/stable/index.html 46. I am not sure if releasing GIL would break anything.