I have a CSV file in S3. I want to run a python script using data present in S3. The S3 file will change once in a week. I need to pass an input argument to my python script which loads my S3 file into Pandas and do some calculation to return the result.
Currently I am loading this S3 file using Boto3 in my server for each input argument. This process takes more time to return the result, and my nginx returns with 504 Gateway timeout.
I am expecting some AWS service to do it in cloud. Can anyone point me in a right direction which AWS service is suitable to use here
You have several options:

Use AWS Lambda, but Lambda has limited local storage (500mb) and memory (3gb) with 15 run time.
Since you mentioned Pandas I recommend using AWS Glue which has ability:


Detect new file
Large Mem, CPU supported
Visual data flow
Support Spark DF
Ability to query data from your CSV files
Connect to different database engines.


We currently use AWS Glue for our data parser processes
How to run python script using S3 data in AWS