I found every sub-process will initGlobalState in c10, even they use the same one underlying device. Thus we cannot invoke the same CUDA stream between different processes. Is there any to share CUDA context across many processes in python-level l. Thus, we can manage CUDA streams by ourselves across multiple processes. Thanks.