Why am I getting different http responses from different locations?
There are public sites, and public api's that specifically block calls from ec2 instances (and probably other cloud providers). 
It's not impossible that some of the sites you are trying to scrape, simply blacklist ec2 instances ip ranges to cut down on the 'bots' that are eating up their resources ... I have come across this several times, for several sites.
The NBA stats api is one example I am familiar with, but I have come across others as well - the sites you are scraping may be some of them as well.
I am working on a task where I need to web scrape boxofficemojo site.
I coded everything and it is working perfectly fine in my local machine.
There are around 19000 urls that I need to scrape. As it is obviously a time consuming process, I don't want to run it on my local machine. Instead I want to run it on an aws ec2 instance.
The ec2 instance is Ubuntu 18.04. I have verified python versions and the libraries used in script are present or not and everything.
However, if I try
requests.get('http://www.boxofficemojo.com') ,
it is giving me 503 response. If I print the response text, it is saying We are in process of updating site right now.  But the same thing is working in my local machine.
Why am I getting this wierd behaviour in ec2 instance.
I tried testing internet connection inside the ec2 instance by issuing ping command . It's working fine.