Data Enginnering, Data Pipeline, Etl, Workflow Management, Airflow.
(Extract, Load, Transform) or ELETL pattern as referred by some architectures, the first set of data pipelines (Extract part) are usually responsible for collection and ingestion of data from source systems into the central data lake’s landing or raw zone/database. The ingestion pipelines are