• Bad: A summary that is extremely partial or grossly not on point due to severe information coverage issues
and does an extremely partial job at communicating the sources’s gist. Importantly, the summary does not add
information to what the source reported. Rather, it misses crucial details or focuses on marginal ones.

• Very Bad: An unreliable summary. It either distorts the meaning conveyed by the source by misleadingly
rearranging the details the source mentions (reasoning violations), reports information not supported by the
source (hallucination) or features noticeably awkward, repetitive or nonsensical language (incoherence).
AI21 SUMMARIZE API- TECHNICAL EVALUATION