I am using the AWS EMR Cluster service.
It is a situation in which machine learning tasks such as spark-build are being performed by referring to the model file with the S3 Bucket between EMR Cluster use.
I request a lot of head and list requests from S3, but I am wondering if it is normal for AWS EMR to send a lot of list and head requests to the S3 model file.
Symptom: AWS EMR is about 2.7 million head and list requests per day to S3.
A lot of list/head requests get sent.
This is related to how directories are emulated on the hadoop/spark/hive S3 clients; every time a progress looks to see if there's a directory on a path it will issue a LIST request, maybe a HEAD request first (to see if its a file).
Then there's the listing of the contents, more LIST requests, and finally reading the files. There'll be one HEAD request on every open() call to verify the file exists and to determine how long it is.
Files are read with GET Requests. Every time there's a seek()/buffer read on the input stream and the data isn't in a buffer the client has to do one of

read to the end of the current ranged get (assuming its a ranged GET), discarding the data, issue a new ranged GET
abort() the HTTPS connection, negotiate a new one. Slow.

Overall then, a lot of IO, especially if the application is inefficient about caching the output of directory listings, whether files exist, doing needless checks before operations (if fs.exists(path) fs.delete(path, false)) and the like.
If this is your code, try not to do that
(disclaimer: this is all guesses based on the experience of tuning the open source hive/spark apps to work through the S3A connector. I'm assuming the same for EMR)
I am wondering if it is normal for AWS EMR to send a lot of list and head requests for S3 model files