Recently, I updated my PyTorch from 0.3 to 0.4 so I also rewrote my C extension using the new C++ APIs in 0.4. Previously, if I wrapped my C extension with DataParallel, the CPU usage of my script could go above 100% (170%). However, if I wrapped the new C++ extension with DataParallel, the CPU usage could not go above 100% and the training was slowed down. After doing some research, I suspected that is related to GIL and I also found a way to release GIL in my C++ source files.
PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
    m.def("forward", &foo_forward, "Foo Forward", py::call_guard<py::gil_scoped_release>());
    m.def("backward", &foo_backward, "Foo Backward", py::call_guard<py::gil_scoped_release>());
}

After I added py::call_guard<py::gil_scoped_release>(), the CPU usage could go above 100% and the training speed returned to normal. I just wonder if this will have some potential problems. Thanks in advance.