FP8 LLMs for vLLM
Collection
Accurate FP8 quantized models by Neural Magic, ready for use with vLLM!
•
44 items
•
Updated
•
71
H.N Huy Nguyen
nhathuy25-orange
·
AI & ML interests
Efficient Energy inference for LLMs
Recent Activity
upvoted
a
collection
21 days ago
FP8 LLMs for vLLM
Organizations
