I'm working on moving data from Postgres DB to AWS Redshift using Python3.7. I created an SQL query that retrieves a query set when executed (I'm using where clause to query. So for every query execution, I'll be changing the ID that I'm passing).
I'm going to run this script in Flask docker container which will be ultimately run on Kubernetes.
I have exposed an POST method enpoint in dockerized Flask app on which I'll receiving a list of IDs that needs to be queried on DB and data to be moved to Redshift using Python.
I want to do multithreading for executing multiple queries at once and moving data as there could be lot of IDs in the POST request that I receive. 
But, as I'm using Python3.7, I came to know that GIL is going to be a bottleneck and it doesn't matter if how many threads you are running, and there will be only one thread executing at any time.
How do I overcome this problem and make the parallel execution of SQL queries on DB possible and that finally works on Kubernetes.
Can I go with multiprocessing or is there any other better way to achieve this?
If you're using Flask app, you can host your app in Gunicorn with multiple workers and threads which allows parallelism by running multiple workers.
Gunicorn launches a master process that can create multiple (configured workers) where each worker can handle an HTTP request independently. The kernel handles the load balancing of request among the workers. 
If you add Gevent to the mix, that also provides concurrency per worker i.e. each worker would be able to handle multiple requests concurrently (not in parallel).
Both of these are available as pip packages, Gevent requires the installation of libev. 
The support for flask by Gunicorn is ensured by the fact that flask is a WSGI framework and Gunicorn is a WSGI server, they're simply plugged into each other.
How to do parallel PostgreSQL query exections in Python3.7 docker container?