zero-gpu-explorers/README · Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

ZeroGPU Explorers org Aug 25, 2024

for some reason, im getting:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/gradio/queueing.py", line 536, in process_events
    response = await route_utils.call_process_api(
  File "/usr/local/lib/python3.10/site-packages/gradio/route_utils.py", line 321, in call_process_api
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1935, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1520, in call_function
    prediction = await anyio.to_thread.run_sync(  # type: ignore
  File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 859, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 826, in wrapper
    response = f(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/spaces/zero/wrappers.py", line 214, in gradio_handler
    raise res.value
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method```

bluuebunny

Sep 3, 2024

Try:

#...
import torch.multiprocessing as mp
#...
# Can also use 'forkserver' on linux
mp.set_start_method('spawn')
#...

ameyv6

2 days ago

I have faced the same issue. I have tried the approach suggested by bluuebunny but it doesn't work. The issue is something deeper.

John6666

2 days ago

In a zero GPU space, functions with the @spaces.GPU decorator are executed in a separate process, so writing to global scope variables from within those functions is usually difficult.
In addition, if any library calls CUDA before the spaces library initializes CUDA, it will basically crash.
optimum-quanto is an easy example.