After giving GPU Programming a hands-on try, I have come to appreciate the level of complexity in AI compute:
text
After giving GPU Programming a hands-on try, I have come to appreciate the level of complexity in AI compute:
new_line
new_line
- Existing/leading frameworks (CUDA, OpenCL, DSLs, even Triton), still fall at the mercy of low-level compute that requires deeper understanding and experience.
text
- Existing/leading frameworks (CUDA, OpenCL, DSLs, even Triton), still fall at the mercy of low-level compute that requires deeper understanding and experience.
new_line
- Ambiguous optimizations methods that will literally drive you mad 🤯
text
- Ambiguous optimizations methods that will literally drive you mad 🤯
new_line
- Triton is cool but not cool enough (high level abstractions that fall back to low level compute issues as you build more specialized kernels)
text
- Triton is cool but not cool enough (high level abstractions that fall back to low level compute issues as you build more specialized kernels)
new_line
- As for CUDA, optimization requires considering all major components of the GPU (DRAM, SRAM, ALUs) 🤕
text
- As for CUDA, optimization requires considering all major components of the GPU (DRAM, SRAM, ALUs) 🤕
new_line
- Models today require stallion written GPU kernels to reduce storage and compute cost.
text
- Models today require stallion written GPU kernels to reduce storage and compute cost.
new_line
- GPTQ was a big save 👍🏼
text
- GPTQ was a big save 👍🏼
new_line
new_line
@karpathy
mention
karpathy
  is right expertise in this area is scarce and the reason is quite obvious - uncertainties: we are still struggling to get peak performance from multi-connected GPUs while maintaining precision and reducing cost. 
text
  is right expertise in this area is scarce and the reason is quite obvious - uncertainties: we are still struggling to get peak performance from multi-connected GPUs while maintaining precision and reducing cost. 
new_line
new_line
May the Scaling Laws favor us lol.
text
May the Scaling Laws favor us lol.
After giving GPU Programming a hands-on try, I have come to appreciate the level of complexity in AI compute:

- Existing/leading frameworks (CUDA, OpenCL, DSLs, even Triton), still fall at the mercy of low-level compute that requires deeper understanding and experience.
- Ambiguous optimizations methods that will literally drive you mad 🤯
- Triton is cool but not cool enough (high level abstractions that fall back to low level compute issues as you build more specialized kernels)
- As for CUDA, optimization requires considering all major components of the GPU (DRAM, SRAM, ALUs) 🤕
- Models today require stallion written GPU kernels to reduce storage and compute cost.
- GPTQ was a big save 👍🏼

@karpathy  is right expertise in this area is scarce and the reason is quite obvious - uncertainties: we are still struggling to get peak performance from multi-connected GPUs while maintaining precision and reducing cost. 

May the Scaling Laws favor us lol.
🤗
alielfilali01 nebiyu29 victor lunarflu larryvrh nikgr str4007 Abecid lixin67
❤️
samusenps lunarflu badayvedat Abecid axwinters
🧠
samusenps lunarflu
631392ea3ef0644e2568446f
/avatars/e4eea78b457e9f00b2ccfe17efeaace2.svg
Aaron Lisman
pwf
user
621d539f962e5162b2f7d9bd
/avatars/acff4702ebdeb696a185a6b9e58c0558.svg
Matt St. Hilaire
msthil2
user
644a8b60ccc39135fb30c82b
/avatars/613ea7a2b4f8acc06c1def85da8f1640.svg
Clyde Wright
clydeiii
user
6438a9027de34e8ea7e4b257
https://cdn-avatars.huggingface.co/v1/production/uploads/6438a9027de34e8ea7e4b257/vib8QSd1AWMr_bR9ig_xJ.jpeg
Jaward Sesay
Jaward
user
628127502a08e33da867e6a4
/avatars/223d964e993126fac680a8dfb71cc291.svg
James K J
james92
user