I have only tried on CPU so I’m not sure if it will help,
but have you checked if openmp is disabled?
In my case it used up all CPU resources when openmp is enabled, but less than 1 percent when disabled.