I have packages stored in s3 bucket. I need to read metadata file of each package and pass the metadata to program. I used boto3.resource('s3') to read these files in python. The code took few minutes to run. While if I use aws cli sync, it downloads these metafiles much faster than boto. My guess was that if I do not download and just read the meta files, it should be faster. But it isn't the case. Is it safe to say that aws cli is faster than using boto?
It's true that the AWS CLI uses boto, but the cli is not a thin wrapper, as you might expect.  When it comes to copying a tree of S3 data (which includes the multipart chunks behind a single large file), it is quite a lot of logic to make a wrapper that is as thorough and fast, and that does things like seamlessly pick up where a partial download has left off, or efficiently sync down only the changed data on the server.
The implementation in the awscli code is more thorough than anything in the Python or Java SDKs, as far as I have seen.  I've seen several developers who were too proud to call the CLI from their code, but zero thus far all such attempts I have seen have failed to measure up.  Love to see a counter example, though.
Is aws CLI faster than using boto3?