  This paper discusses two existing approaches to the correlation analysis
between automatic evaluation metrics and human scores in the area of natural
language generation. Our experiments show that depending on the usage of a
system- or sentence-level correlation analysis, correlation results between
automatic scores and human judgments are inconsistent.

Computation and Language