Adding only one of s3 partitioned files to AWS Glue
When configuring the Crawler (or editing an existing one), under the Output section, expand Grouping behavior for S3 data (optional) and select Create a single schema for each S3 path
I'm having slight issues when it comes to running a crawler through my s3 buckets. My folders have data that was dumped from redshift that was sliced into many different files. These files naming convention go as the following:
dump_0000_part_00.gz,   dump_0001_part_01.gz ....
However when my crawler fetches the metadata in this folder, it instead makes a few hundred tables, thinking each one of these sliced files is its own table. Is there a way to tell the crawler to group all these sliced files into ONE catalog table?