Question: I need to manage a large workflow of ETL tasks, which execution depends on time, data availability or an external event. Some jobs may fail during execution of the workflow and the system should have the ability to restart a failed workflow branch without waiting for whole workflow to finish execution.

Are there any frameworks in python that can handle this?

I see several core functions:

* DAG bulding
* Execution of nodes (run shell cmd with wait,logging etc.)
* Ability to rebuild sub-graph in parent DAG during execution
* Ability to manual execute nodes or sub-graph while parent graph is running
* Suspend graph execution while waiting for external event
* List job queue and job details

Something like [Oozie](http://oozie.apache.org), but more general purpose and in python.

Answer: 
There are a ton of these; everyone seems to write their own. There is a good list at <https://github.com/common-workflow-language/common-workflow-language/wiki/Existing-Workflow-systems>. Which includes systems that originate in both industry and academia.
Have you looked at [Ruffus](http://www.ruffus.org.uk/index.html)?

I have no experience with it but it appears to do some of the items on your list. It also looks quite hackable so you might be able to implement your other requirements yourself.