Answers.Body: <p><a href="https://en.wikipedia.org/wiki/Division_by_zero" rel="nofollow noreferrer">Division by zero</a> is <strong>not</strong> mathematically defined. </p>
  
  <p>A usual or standard way of dealing with this issue is to raise an exception. For example, in Python, the exception <code>ZeroDivisionError</code> is raised at <em>runtime</em> if you happen to divide by zero. </p>
  
  <p>If you execute <a href="https://repl.it/repls/OddHeartyDatalogs" rel="nofollow noreferrer">the following program</a></p>
  
  <pre><code>zero = 0
  numerator = 10
  numerator / zero
  </code></pre>
  
  <p>You will get</p>
  
  <pre><code>Traceback (most recent call last):
    File "main.py", line 3, in &lt;module&gt;
      numerator / zero
  ZeroDivisionError: division by zero
  </code></pre>
  
  <p>However, if you want to avoid this runtime exception, you can check for division by zero and deal with this issue in a way that is appropriate for your program (without needing to terminate it).</p>
  
  <p>In the paper <a href="https://www.aclweb.org/anthology/P02-1040.pdf" rel="nofollow noreferrer">BLEU: a Method for Automatic Evaluation of Machine Translation</a> that introduced the BLEU (and brevity penalty) metric, the authors defined the brevity penalty as</p>
  
  <p><span class="math-container">\begin{align}
  BP =
  \begin{cases}
  1, &amp; \text{if } c &gt; r\\
  e^{(1- r/c)} &amp; \text{if } c \leq r\\
  \end{cases} \label{1} \tag{1}
  \end{align}</span></p>
  
  <p>This definition does not explicitly take into account the division by zero. </p>
  
  <p>The Python package <a href="https://www.nltk.org" rel="nofollow noreferrer"><code>nltk</code></a> does not raise an exception, but it (apparently, arbitrarily) returns zero when <code>c == 0</code>. Note that the BLEU metric ranges from 0 to 1. For example, if you execute the <a href="https://repl.it/repls/ShortWelltodoActivecontent" rel="nofollow noreferrer">following program</a></p>
  
  <pre><code>from nltk.translate.bleu_score import brevity_penalty, closest_ref_length
  
  reference1 = list("hello") # A human reference translation. 
  references = [reference1] # You could have more than one human reference translation.
  
  # references = [] Without a reference, you will get a ValueError.
  
  candidate = list() # The machine translation.
  c = len(candidate)
  
  r =  closest_ref_length(references, c)
  print("brevity_penalty =", brevity_penalty(r, c))
  </code></pre>
  
  <p>You will get</p>
  
  <pre><code>brevity_penalty = 0
  </code></pre>
  
  <p>In the example above, the only human reference (translation) is <code>reference1 = list("hello")</code> and the only candidate (the machine translation) is an empty list. However, if <code>references = []</code> (you have no references), then you will get the error <code>ValueError: min() arg is an empty sequence</code>, where <code>references</code> are used to look for the closest reference (the closest human translation) to the  <code>candidate</code> (the machine translation), given that there could be more than one human reference translation, and one needs to be chosen to compute the brevity penalty, with respect to your given candidate. </p>
  
  <p>In fact, in the <a href="https://www.nltk.org/_modules/nltk/translate/bleu_score.html#brevity_penalty" rel="nofollow noreferrer">documentation of the <code>brevity_penalty</code> function</a>, the following comment is written </p>
  
  <pre><code># If hypothesis is empty, brevity penalty = 0 should result in BLEU = 0.0.
  </code></pre>
  
  <p>where <code>hypothesis</code> is a synonym for <em>candidate</em> (the machine translation) and the length of the candidate is <span class="math-container">$c$</span> in the formula \ref{1} (and <code>c</code> in the example above).</p>
  
  <p>To answer your second question more directly, I don't think there's a standard way of dealing with the issue, but I've not fully read the BLEU paper yet.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-08T16:40:58.307Z
Answers.LastActivityDate: 2019-11-08T20:00:38.727Z
Answers.LastEditDate: 2019-11-08T20:00:38.727Z
Answers.UserDisplayName: nbro
Body: <p>The brevity penalty is defined as</p>
  
  <p><span class="math-container">$$bp = e^{(1- r/c)},$$</span></p>
  
  <p>where <span class="math-container">$r$</span> is the reference length and <span class="math-container">$c$</span> is the output length.</p>
  
  <p>But what happens if the output length gets zero? Is there any standard way of coping with that issue?</p>
  
Tags: <natural-language-processing><machine-translation><bleu>
Title: What happens when the output length in the brevity penalty is zero?
UserDisplayName: ScientiaEtVeritas
ThreadText: # What happens when the output length in the brevity penalty is zero?
  
  (Asked by: username_0 on 2019-11-08)
  
  The brevity penalty is defined as
  
  $$bp = e^{(1- r/c)},$$
  
  where $r$ is the reference length and $c$ is the output length.
  
  But what happens if the output length gets zero? Is there any standard way of coping with that issue?
  
  **Tags:** natural-language-processing, machine-translation, bleu
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-08. Score: 1 | Accepted answer
  
  [Division by zero](https://en.wikipedia.org/wiki/Division_by_zero) is **not** mathematically defined.
  
  A usual or standard way of dealing with this issue is to raise an exception. For example, in Python, the exception `ZeroDivisionError` is raised at *runtime* if you happen to divide by zero.
  
  If you execute [the following program](https://repl.it/repls/OddHeartyDatalogs)
  
  ```
  zero = 0
  numerator = 10
  numerator / zero
  
  ```
  
  You will get
  
  ```
  Traceback (most recent call last):
    File "main.py", line 3, in <module>
      numerator / zero
  ZeroDivisionError: division by zero
  
  ```
  
  However, if you want to avoid this runtime exception, you can check for division by zero and deal with this issue in a way that is appropriate for your program (without needing to terminate it).
  
  In the paper [BLEU: a Method for Automatic Evaluation of Machine Translation](https://www.aclweb.org/anthology/P02-1040.pdf) that introduced the BLEU (and brevity penalty) metric, the authors defined the brevity penalty as
  
  \begin{align}
  BP =
  \begin{cases}
  1, & \text{if } c > r\\
  e^{(1- r/c)} & \text{if } c \leq r\\
  \end{cases} \label{1} \tag{1}
  \end{align}
  
  This definition does not explicitly take into account the division by zero.
  
  The Python package [`nltk`](https://www.nltk.org) does not raise an exception, but it (apparently, arbitrarily) returns zero when `c == 0`. Note that the BLEU metric ranges from 0 to 1. For example, if you execute the [following program](https://repl.it/repls/ShortWelltodoActivecontent)
  
  ```
  from nltk.translate.bleu_score import brevity_penalty, closest_ref_length
  
  reference1 = list("hello") # A human reference translation. 
  references = [reference1] # You could have more than one human reference translation.
  
  # references = [] Without a reference, you will get a ValueError.
  
  candidate = list() # The machine translation.
  c = len(candidate)
  
  r =  closest_ref_length(references, c)
  print("brevity_penalty =", brevity_penalty(r, c))
  
  ```
  
  You will get
  
  ```
  brevity_penalty = 0
  
  ```
  
  In the example above, the only human reference (translation) is `reference1 = list("hello")` and the only candidate (the machine translation) is an empty list. However, if `references = []` (you have no references), then you will get the error `ValueError: min() arg is an empty sequence`, where `references` are used to look for the closest reference (the closest human translation) to the `candidate` (the machine translation), given that there could be more than one human reference translation, and one needs to be chosen to compute the brevity penalty, with respect to your given candidate.
  
  In fact, in the [documentation of the `brevity_penalty` function](https://www.nltk.org/_modules/nltk/translate/bleu_score.html#brevity_penalty), the following comment is written
  
  ```
  # If hypothesis is empty, brevity penalty = 0 should result in BLEU = 0.0.
  
  ```
  
  where `hypothesis` is a synonym for *candidate* (the machine translation) and the length of the candidate is $c$ in the formula \ref{1} (and `c` in the example above).
  
  To answer your second question more directly, I don't think there's a standard way of dealing with the issue, but I've not fully read the BLEU paper yet.
  
  
  ---
  
  