  We report on the Wikidata vandalism detection task at the WSDM Cup 2017. The
task received five submissions for which this paper describes their evaluation
and a comparison to state of the art baselines. Unlike previous work, we recast
Wikidata vandalism detection as an online learning problem, requiring
participant software to predict vandalism in near real-time. The
best-performing approach achieves a ROC-AUC of 0.947 at a PR-AUC of 0.458. In
particular, this task was organized as a software submission task: to maximize
reproducibility as well as to foster future research and development on this
task, the participants were asked to submit their working software to the TIRA
experimentation platform along with the source code for open source release.

Information Retrieval