Python Script on Google App Engine, which scrapes only updates from a website
Doesn't the website have RSS or API or something?
Anyway, you could store the list of scraped news titles (might not be unique though) / IDs / URLs as entity IDs in the datastore right after you send them to your email & just before sending the email you would first check whether the news IDs exist in the datastore with simply not including the onces that do.
Or depending in what strucure the articles are being published and what data is available (Do they have an incrimental post ID? Do they have a date of when an article was posted?) you may simply need to remember the highest value of your previous scrapping and only send email to yourself with the articles where that value is higher than the one previously saved.
I am hosting a Python script on Google App Engine which uses bs4 and mechanize to scrap news section of a website, it runs every 2 hours and sends an email to me all the news.
The Problem is, I want only the Latest news to be sent as mail, As of now it sends me all the news present every time.
I am storing all the news in a list, is there a way to send only the latest news, which has not been mailed to me, not the complete list every time?