Will quantization be supported for GPUs anytime soon? I have a project where evaluation speed is a very major concern and would love to use quantization to speed it up.
I see the CPU quantization tutorial on the docs was written about 6 months ago, so I am really just curious if this is on the developers’ radar at all and if we can expect this eventually or in the near future.