We see no good reason , however , why such text spans should necessarily be sentences , since the majority of tagging paradigms ( e.g. , Hidden Markov Model [ HMM ] [ Kupiec 1992 ] , Brill 's [ Brill 1995a ] , and MaxEnt [ Ratnaparkhi 1996 ] ) do not attempt to parse an entire sentence and operate only in the local window of two to three tokens .
transformationbased errordriven learning and natural language parsing a case study in partofspeech tagging
Eric Brill
This requires resolving sentence boundaries before tagging . We see no good reason , however , why such text spans should necessarily be sentences , since the majority of tagging paradigms ( e.g. , Hidden Markov Model [ HMM ] [ Kupiec 1992 ] , Brill 's [ Brill 1995a ] , and MaxEnt [ Ratnaparkhi 1996 ] ) do not attempt to parse an entire sentence and operate only in the local window of two to three tokens . The only reason why taggers traditionally operate on the sentence level is that a sentence naturally represents a text span in which POS information does not depend on the previous and following history .
We see no good reason , however , why such text spans should necessarily be sentences , since the majority of tagging paradigms ( e.g. , Hidden Markov Model [ HMM ] [ Kupiec 1992 ] , Brill 's [ @@CITATIONa ] , and MaxEnt [ Ratnaparkhi 1996 ] ) do not attempt to parse an entire sentence and operate only in the local window of two to three tokens .