Hi community,
I am new to transformer models and particularly interested by a multilingual sentence transformer (stsb-xlm-r-multilingual, 1.1 Go).
I have passed multiple times to search on how to deploy this model on cloud with the highest throughput (up to 1000 requests/sec) and lowest latency (<1 sec), while trying to saving costs. It seems to be painful.
Would anyone have advice on deployment requirements (CPU, GPU, RAM,…), clouds (AWS, GCP,…),  and frameworks (torchserve, triton,…) to meet my needs?
Thanks!