What's the Fastest way to get a large number of files (relatively small 10-50kB) from Amazon S3 from Python? (In the order of 200,000 - million files). 
At the moment I am using boto to generate Signed URLs, and using PyCURL to get the files one by one. 
Would some type of concurrency help? PyCurl.CurlMulti object? 
I am open to all suggestions. Thanks!
I don't know anything about python, but in general you would want to break the task down into smaller chunks so that they can be run concurrently.  You could break it down by file type, or alphabetical or something, and then run a separate script for each portion of the break down.
Downloading a Large Number of Files from S3