ONNX
#1
by
cnmoro
- opened
Any chance we can get an ONNX version with examples?
Would be awesome.
For context:
Spinning up a container with the "transformers" lib eats a lot of disk. Onnx and onnxruntime are very lightweight
Also, kudos, the models are awesome.
Well, I've managed to do it:
https://gist.github.com/cnmoro/3c66de4f92716e8cf044e550a23ee9d2
This gist contains code for converting to onnx, quantizing to 8bit and performing inference.
I will be uploading some of these in my profile.
cnmoro
changed discussion status to
closed
cnmoro
changed discussion status to
open
cnmoro
changed discussion status to
closed