I am currently overhauling a project here at work and need some advice. We currently have a morning checklist that runs daily and executes roughly 30 SQL files with 1 select statement each. This is being done in an excel macro which is very unreliable. These statements will be executed against an oracle database.
Basically, if you were re-implementing this project, how would you do it? I have been researching concurrency in python, but have not had any luck. We will need to capture the results and display them, so please keep that in mind.If more information is needed, please feel free to ask.
Thank you.
There are lots of ways depending on how long the queries run, how much data is output, are there input parameters and what is done to the data output.
Consider:
1. Don't worry about concurrency up front
2. Write a small python app to read in every *.sql file in a directory and execute each one.
3. Modify the python app to summarize the data output in the format that it is needed
4. Modify the python app to save the summary back into the database into a daily check table with the date / time the SQL queries were run.  Delete all rows from the daily check table before inserting new rows
5. Have the Excel spreadsheet load it's data from that daily check table including the date / time the data was put in the table 
6. If run time is slows, optimize the PL/SQL for the longer running queries
7. If it's still slow, split the SQL files into 2 directories and run 2 copies of the python app, one against each directory.
8. Schedule the python app to run at 6 AM in the Windows task manager.
Most efficient way to execute 20+ SQL Files?