title: Sturdy serverless ETL flow in AWS
body: <p>I am trying to build a AWS sturdy serverless ETL flow to load (CSV) files from an S3 bucket to a Postgres instance (postgresql9.6). The files are rather big (10gb+), so, while testing I have ran into multiple limitations of Lambda ( /tmp space, memory space, time). </p>
  
  <p>I am looking into AWS Glue, but have the feeling that this might have limitations toward RDS, and that it is a bit overkill for what I am trying to do.</p>
  
  <p>Furthermore, the CSV file contains a multichar delimiter ("~|~") that Postgres COPY FROM cannot process (so, I need to do some preprocessing as well).</p>
  
  <p>Does anyone have experience with a similar situation and managed to do it serverless, or should I stick with a EC2 instance for know?</p>