Hardware recomendations help
#100
by
MLSDev
- opened
Hi I want to deploy the model for use test it in a Rag.
what is the gpu ram recommended for inference with 1024 chunk token size?
is cpu usage possible? with my Hetzner dedicated VPS doesn't appear enought
I try to use the api of jina ai, but tokens of my api key seems disapear without my use.
I find xinference like a method to integrate the use, any other recomendation?