I have a S3 bucket and some Python code, the code read all the available files for the current day and download them to s3 (it reads the files from FTP in an ascending order, based on the datetime in the filename when the file gets uploaded to FTP), so for example I have downloaded file 1 and file 2 in the last run and uploaded them to S3, now I know FTP has a new file file 3 available, then a new run will download files in the following order: file1 file2 and file3 and upload all the files again in the same order to the same S3 path (file1 and file2 gets overwritten, and new file file 3 will also be uploaded to s3).
My question is what's the easiest way to identify the newly-uploaded file file3 in Python?
The easiest way I can think of to see the difference between 'updated' files and newly created files is simply doing a try/except GetObject before the PutObject. This is preferred over first doing the PutObject then trying to figure out what changed since S3 has no easy way of retrieing objects by 'Modified date' or simular.
So if your question was about checking which files were already present in S3 before uploading, try doing the GetObject first :).
What's the easieat way to get the latest uploaded file in S3 (when other existing files get overwritten) - Python