Reasoning about Goals, Steps, and Temporal Ordering with WikiHow
We propose a suite of reasoning tasks on two types of relations between procedural events: goal step relations "learn poses" is a step in the larger goal of "doing yoga" and step step temporal relations "buy a yoga mat" typically precedes "learn poses" We introduce a dataset targeting these two relations based on wikiHow, a website of instructional how to articles. Our human validated test set serves as a reliable benchmark for commonsense inference, with a gap of about 10% to 20% between the performance of state of the art transformer models and human performance. Our automatically generated training set allows models to effectively transfer to out of domain tasks requiring knowledge of procedural events, with greatly improved performances on SWAG, Snips, and the Story Cloze Test in zero and few shot settings.
EMNLP