Data Enginnering, Data Pipeline, Etl, Workflow Management, Airflow.
current job has to be retried when upstream pipeline has successfully retried or when the issue has been resolved. Retry and reprocessing of upstream pipeline jobs are not automatically coordinated with the downstream pipelines and requires either manual intervention. Technique #2 — Rely on time