Python: Parsing out publication dates from html pages
I don't see why a collection of regexs wouldn't work. There are a variety of different formats, but there are really only a handful that are most common. With, say, a dozen easy regex, you could probably scrub 90% of the dates out there.
Another (partial) approach would be to scan for month names and abbreviations, and then scan the surrounding text for days and year.
For numeric-only, the hardest part would be figuring out whether it's month then date or date then month. So will be easy, if the date part is greater than 12, but otherwise there's not really anyway of knowing.
You could also look for <time> elements with the datetime attribute, which is supposed to follow an unambiguous format (though not necessarily consistent).
Bottom line, I don't think there is any one way to find all the dates in a document, unless you know they all follow the same format, which obviously isn't going to be the case in general. To have a good shot of finding them, you'll just need to employ several different strategies.
For html pages, and news-related pages especially, it would be very helpful and incredibly useful if there were a mechanism for parsing out the publication dates.
Unfortunately, there is not one set regex/pattern for dates on the internet. CNN may publish it like MONTH DD, YYYY and HuffingtonPost may publish as MM/DD/YY, and so on.
Does anyone have any strategies which are better than just pure regex parsing for extracting publication dates out of html pages?
Thank you.