Suggested runtime for small laptops?
#1
by
XsongyangX
- opened
Thanks for updating your 1b models! I'm on a MacBook Air so these are the only possible models within my hardware range. What runtimes do you recommend for low resource people? LM Studio?
Yes, LM, because you can easily switch to CPU based generation. (and it is updated regularly).
Also - easy of use too.
LM can also operate as an API service as well.
What do you think about Jan? https://github.com/menloresearch/jan
I tried Jan when it first appeared ; due to time constraints have not used/downloaded its later versions.