I am trying to download only files modified in the last 30 minutes from a URL. Can you please guide me how to proceed with this. Please let me know if I should use shell scripting or python scripting for this.
If the server supports if-modified-since, you could send the request with If-Modified-Since: (T-30 minutes) and ignore the 304 responses.
Download files modified in last 30 minutes from a URL