I am writing a program in python that will automatically download pdf files from a website once a day.
When trying to test I noticed that the files downloaded had the correct extension but they are very small (<1kB) compared to the normal size of about 100kB when downloaded manually.
Can a website block a program from automatically downloading files?
Is there anything that can be done about this?
Yes. Cloudflare can block bots from downloading files. Blocking is usually done by detecting the user-agent or including javascript in a webpage. I would examine the pdf file in notepad and see what it contains also try adding a user-agent option in your python code.
Can a website stop a program from downloading files automatically?