Berger et al ( 1996 ) describe an efficient algorithm for accomplishing this in which approximations to Pst ( TIS ) are computed in parallel for all ( new ) features ft by holding all weights in the existing model fixed and optimizing only over a8t .
a maximum entropy approach to natural language processing
Adam L Berger Stephen A Della Pietra Vincent J Della Pietra
A powerful strategy for using gains is to build a model iteratively by adding at each step the feature which gives the highest gain with respect to those already added . Berger et al ( 1996 ) describe an efficient algorithm for accomplishing this in which approximations to Pst ( TIS ) are computed in parallel for all ( new ) features ft by holding all weights in the existing model fixed and optimizing only over a8t . However , this method requires many expensive passes over the corpus to optimize the weights for the set of features under consideration at each step , and it adds only one feature per step , so it is not practical for constructing models containing thousands of features or more .
@@CITATION describe an efficient algorithm for accomplishing this in which approximations to Pst ( TIS ) are computed in parallel for all ( new ) features ft by holding all weights in the existing model fixed and optimizing only over a8t .