Great work!
Joonhyung Lee
joonhyung-lee-naver
AI & ML interests
None yet
Recent Activity
commented on
an
article
26 days ago
Accelerating LLM Inference with TGI on Intel Gaudi
upvoted
an
article
26 days ago
Accelerating LLM Inference with TGI on Intel Gaudi
liked
a Space
2 months ago
nanotron/ultrascale-playbook
Organizations
joonhyung-lee-naver's activity
commented on
Accelerating LLM Inference with TGI on Intel Gaudi
26 days ago
upvoted
an
article
26 days ago
Article
Accelerating LLM Inference with TGI on Intel Gaudi
By
and 4 others
•
•
13
liked
a
Space
2 months ago
reacted to
regisss's
post with 🔥
2 months ago
Post
1714
Nice paper comparing the fp8 inference efficiency of Nvidia H100 and Intel Gaudi2:
An Investigation of FP8 Across Accelerators for LLM Inference (2502.01070)
The conclusion is interesting: "Our findings highlight that the Gaudi 2, by leveraging FP8, achieves higher throughput-to-power efficiency during LLM inference"
One aspect of AI hardware accelerators that is often overlooked is how they consume less energy than GPUs. It's nice to see researchers starting carrying out experiments to measure this!
Gaudi3 results soon...
The conclusion is interesting: "Our findings highlight that the Gaudi 2, by leveraging FP8, achieves higher throughput-to-power efficiency during LLM inference"
One aspect of AI hardware accelerators that is often overlooked is how they consume less energy than GPUs. It's nice to see researchers starting carrying out experiments to measure this!
Gaudi3 results soon...
authored
2
papers
9 months ago