kimishpatel:


Is your end goal maximizing resource utilization in such a way that it reduces latency?



Yes that’s exactly what we are trying to do. In our case of TTS models, the user is quite sensitive to latency.
I understand that this highly depends on the mobile phone type used. On some of the phones, we can utilize the GPU or even a Tensor as in the Pixel6 case. On others, we’d need the CPU’s. We probably even need to provide different models depending on the phone.
I wanted to know, how fast our models would be on CPU’s, when running on an up-to-date current phone. Using the GPU or Tensor on that phone would probably dramatically decrease the latency.
I will open an issue and assign it to you.