Python Boto3 - how to check if s3 file is completely written before process start copying to another bucket
I also do uploads of up to 40GB.  
Since I do multi-part uploads, I check if the file I am writing to is closed. An S3 file(object) is only closed when all operations are finished.
Another way is to use asynchronous task queue like Celery. You will get notifications when a task is done.
I now use Golang but both those methods have worked very well for me.
How to make sure that Process A has completely written large file (5+ GB) in AWS S3 Bucket A before Process B starts copying file to AWS S3 Bucket B using boto3?