As a GPU poor, I found a nice open source project.
text
As a GPU poor, I found a nice open source project.
new_line
new_line
dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)
text
dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)
new_line
new_line
Provision VM instances in 3 different use cases. These are the essential for any ML projects.
text
Provision VM instances in 3 different use cases. These are the essential for any ML projects.
new_line
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
text
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
new_line
- Task: run experiments (training, fine-tuning, ...) via SSH 
text
- Task: run experiments (training, fine-tuning, ...) via SSH 
new_line
- Service: run your model in production via HTTPS
text
- Service: run your model in production via HTTPS
new_line
new_line
dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.
text
dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.
new_line
new_line
The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.
text
The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.
new_line
new_line
If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important. 
text
If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important. 
new_line
: 
text
: 
https://dstack.ai/
https://dstack.ai/
link
new_line
new_line
If you are looking for an alternative, there is SkyPilot project as well
text
If you are looking for an alternative, there is SkyPilot project as well
new_line
: 
text
: 
https://github.com/skypilot-org/skypilot
https://github.com/skypilot-org/skypilot
link
As a GPU poor, I found a nice open source project.

dstackai is the perfect open source project for GPU poor. Simply specify resource requirements (GPU RAM, spot, ...), then will suggest the cheapest options among the popular GPU cloud providers (AWS, GCP, Azure, Lambda Labs, TensorDock, and vast.ai)

Provision VM instances in 3 different use cases. These are the essential for any ML projects.
- Dev: connect provisioned VM instance to your fav IDE (including Jupyter)
- Task: run experiments (training, fine-tuning, ...) via SSH 
- Service: run your model in production via HTTPS

dstack is 100% open source, but you need to have your own accounts for each GPU cloud provider, enough GPU quota, configure credentials, etc., all by yourself. Luckily, dstack will announce dstack cloud which let you not worried about all the hassles. The price is almost as same as you directly connect to each cloud with your account.

The attached code snippet shows you how to provision a Mistral-7B model in Text Generation Inference(TGI) on the cheapest VM instance (of having 24GB of VRAM). Then you get the HTTPS connection to it, and play with it as usual with TGI client library as attached in the second code snippet.

If you want to learn more about dstack, check out the official website. Without GPU sponsors, as an individual open source contributor in ML, this kind of project is pretty important. 
: https://dstack.ai/

If you are looking for an alternative, there is SkyPilot project as well
: https://github.com/skypilot-org/skypilot
❤️
peterschmidt85 chansung osanseviero lysandre merve victor sayakpaul linoyts abhishek abidlabs KnutJaegersberg auditor johannhartmann julien-c radames bitdeep samusenps mishig gblazex juyongjiang greencookies akhaliq Citaman sbrandeis
🤗
peterschmidt85 lysandre merve KnutJaegersberg julien-c pierrci samusenps juyongjiang akhaliq
👍
bitdeep juyongjiang akhaliq
5f7fbd813e94f16a85448745
https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
Sayak Paul
sayakpaul
user
60d3b57ad7b174177faabd6e
https://cdn-avatars.huggingface.co/v1/production/uploads/1659971187637-60d3b57ad7b174177faabd6e.jpeg
chansung park
chansung
user
5dd96eb166059660ed1ee413
https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg
Julien Chaumond
julien-c
user