RuntimeError: MPS backend out of memory.
#8
by
TahaZk
- opened
Hello,
I get this error when running on my mac m3:
RuntimeError: MPS backend out of memory (MPS allocated: 45.50 GB, other allocations: 12.22 GB, max allowed: 61.20 GB). Tried to allocate 12.20 GB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
Is there no way to run this model on a mac ?
I've run it successfully on apple silicon as below
https://gist.github.com/cavit99/811919b3e7753c925ab603b1929dbd99
Having said that I think you may be out of memory , not sure which m3 you have but mine is 128GB and can handle it at this full precision. Smaller URAM models will need to wait for quantized model like Q8 or smaller.