How to Extract Data from Wikipedia and Wikidata
Extracting just raw text from the Wikipedia dumps is pretty hellish. Apparently the official Wikipedia parser itself is something like one 5000 line PHP function. There are about 30 or so alternate parsers that attempt to do this with limited success. I tried a lot of those options, but eventually had to hack together some terrible scripts of my own to do the job.https://www.mediawiki.org/wiki/Alternative_parsersIt was for a web app that I made as a way to learn web dev. The data extraction from Wikipedia was what really killed my motivation and eventually left the project in a sort of amateurish, half-finished state. But, if any language learners wants to practice reading difficult English texts it might be somewhat useful:https://lexical.io