Groq, Language Processing Units, Fastest Llm Inference, Gpu, AI.
https://github.com/ray-project/llmperf-leaderboard Latency vs Throughput Latency: Time to first tokens chunk received, in seconds, after API request sent Throughput: Token per second received while the model is generating tokens (ie after the first chunk has been received from the API) Lower and to