title: How to Convert Many CSV files to Parquet using AWS Glue
body: <p>I'm using AWS S3, Glue, and Athena with the following setup:</p>
  
  <p>S3 --> Glue --> Athena</p>
  
  <p>My raw data is stored on S3 as CSV files. I'm using Glue for ETL, and I'm using Athena to query the data. </p>
  
  <p>Since I'm using Athena, I'd like to convert the CSV files to Parquet. I'm using AWS Glue to do this right now. This is the current process I'm using:</p>
  
  <ol>
  <li>Run Crawler to read CSV files and populate Data Catalog.</li>
  <li>Run ETL job to create Parquet file from Data Catalog.</li>
  <li>Run a Crawler to populate Data Catalog using Parquet file.</li>
  </ol>
  
  <p>The Glue job only allows me to convert one table at a time. If I have many CSV files, this process quickly becomes unmanageable. Is there a better way, perhaps a "correct" way, of converting <strong>many</strong> CSV files to Parquet using AWS Glue or some other AWS service?</p>