Integrating helium-1-preview-2b with Ditto for TensorRT-LLM Support
#6
by
HyungjunKim
- opened
Hello kyutai team,
Thanks for sharing a great model and we're looking forward to future releases, such as an instruct model or a non-preview version.
In the meantime, we built and tested the Helium model as a TensorRT-LLM engine. Since TensorRT-LLM does not currently support the Helium model natively, we used our recently open-sourced Ditto library. We also conducted quality and throughput evaluations—detailed results can be found in our README and blog article. If you need to build engine with TensorRT-LLM, we highly recommend giving Ditto a try.
Looking forward to even better models!