Zinuo:

I wonder what global memory it is and its functions.


These maintain state of the device and also work areas for various libraries I think.
You can poke around in the relevant PyTorch source directories and read up on context 1 in the CUDA docs and the libraries like cuDNN, cuBLAS etc. (Seems that NVidia+cloudfare helpfully decided I should not link to the NVidia docs more, so you’ll have to find those yourself. Typically the workflow is that you get a global handle).