Question
#1
by
x-polyglot-x
- opened
Hello there!
I want to try your model on my Mac Studio (128gb ram).
You mentioned running this command to make it run comfortably: "sudo sysctl iogpu.wired_limit_mb=121000"
Can you describe this in a step-by-step method? I am currently using LM-Studio to run models.
Thanks!
P.S. I tried the command in the terminal and understand it works to set VRAM explicitly. However, the model itself doesn't run by default in LM Studio. I receive an error about the jinja template. I used Llama 3 template just to see if it would run. It did run but seemed unstable at 40k context. I also had odd memory usage graphs in the activity monitor - slow spikes and then big drops in usage.
I'd like to get this model working. Any advice is appreciated.