I was trying to download some files from a sever but it came back with an error page saying only links from internal server is allowed. I was able to download the file with any browser by clicking the link and I have verified the link I captured in Python was correct.
Is there any way this can be done by using python? I tried urllib, urllib2 and requests, but none works. I could use selenium but the solution is not elegent
When you use your browser, it sends a header known as a User-Agent that identifies it. You need to 'spoof' the user agent from your python script to make it think a human is browsing the website. Set the User-Agent header to that of a common browser, this makes it difficult for the server to detect that you are using a script.
Download files that only allow access from internal server with Python