example of Figure 4, this transformed text span
happens to be part of the second mutation, and
as per Table 2 forward entailment is the corre-
sponding NatOp given the veracity label, result-
ing in the selection of the first NatOp sequence.
In rare occasions (2.55% claims in FEVER), we
manually performed NatOp assignment, as the fil-
tering steps led to zero candidates in those cases.
As the heuristic annotation requires manual ef-
fort, we explore how it can be obtained using a
supervised classifier (see §5.5).
5 Experimental Methodology
5.1 Data
ProoFVer is trained using heuristically annotated
proofs (§4) obtained from FEVER (Thorne et al.,
2018a), which has a train/test/development split
of 145,449/19,998/19,998 claims. Further,
the
heuristic proof annotation involves the use of
additional information from the manual annota-
tion logs of FEVER, recently released by Thorne
ProoFVer- Natural Logic Theorem Proving for Fact Verification