Serving improvements
This PR brings some improvements over the CLI serving command.

Changes:
- Expose the possibility to change the number of underlying FastAPI workers.
- Make forward() async so it doesn't timeout in the middle a requests.
- Fixed USE_TF, USE_TORCH env vars fighting each other.
> This PR brings some improvements over the CLI serving command.
> 
> Changes:
> 
> * Expose the possibility to change the number of underlying FastAPI workers.
> * Make forward() async so it doesn't timeout in the middle a requests.
> * Fixed USE_TF, USE_TORCH env vars fighting each other.

Hi @mfuntowicz 
I have tested multi workers on my PC with localhost, and I observe that getting more workers does not make Query request per sec any faster.  

I suppose the bottleneck somewhere, maybe the model file ?
<|||||>Hi @zhoudoufu,

There is a high probability if all the workers are running on the same GPU there still no to get sequential access to the hardware.

One possible improvement would be to specify env variable / GPU ordinal for each worker instance. I may try to have a look in the near future.

Morgan<|||||>Hi @mfuntowicz 
For my test, I use only CPUs. And I do observe CPUs' usage goes up when having more workers. For CPU usage I think there might be other causes.