Data Engineering, Elt Pipeline, Apache Airflow, Kinesis Firehose, Snowflake Data Cloud.
directing data from landing zone to processing task_customer_landing_to_processing = BashOperator( task_id="landing_to_processing", bash_command='aws s3 mv s3://landing/firehose/customers/bucket/ s3://processing/firehose/customers/bucket/{0}/ --recursive'.format(batch_id), dag=dag ) # for directing