Data Engineering, Elt Pipeline, Apache Airflow, Kinesis Firehose, Snowflake Data Cloud.
processing, processed and dags. We will keep the dag (Directed Acyclic Graph) file in the bucket. Configure the IAM role. We will create the IAM policies and role for EC2 for connecting to Firehose and sending logs to Cloudwatch. Add the following policies to the role, lets call it FirehoseAccess :