3
2
0
2

 

b
e
F
0
2

 

 
 
]
L
C
.
s
c
[
 
 

1
v
6
8
0
0
1

.

2
0
3
2
:
v
i
X
r
a

Abstract

Van Miltenburg et al. (2021) suggest NLP re-
search should adopt preregistration to prevent
ﬁshing expeditions and to promote publication
of negative results. At face value, this is a
very reasonable suggestion, seemingly solving
many methodological problems with NLP re-
search. We discuss pros and cons—some old,
some new: a) Preregistration is challenged by
the practice of retrieving hypotheses after the
results are known; b) preregistration may bias
NLP toward conﬁrmatory research; c) prereg-
istration must allow for reclassiﬁcation of re-
search as exploratory; d) preregistration may
increase publication bias; e) preregistration
may increase ﬂag-planting; f) preregistration
may increase p-hacking; and ﬁnally, g) prereg-
istration may make us less risk tolerant. We
cast our discussion as a dialogue, presenting
both sides of the debate.

1 Preregistration
A Two-Sided Discussion of Preregistration of NLP Research