Answers.Body: <p>It doesn't seem that it is a "proper" symbol. </p>
  
  <p>I guess that <span class="math-container">$\sup$</span> simply refers to the supremum, that is, you want to select actions that maximize the quantity that comes to the right of <span class="math-container">$\sup$</span>, while <span class="math-container">$\text{dist}$</span> is simply a proxy for any possible distance between distributions. For example, you can replace <span class="math-container">$\text{dist}$</span> with the Kullback-Leibler divergence or with the mutual information.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-01-06T20:25:33.337Z
Answers.LastActivityDate: 2020-01-06T20:25:33.337Z
Answers.UserDisplayName: Diego Gomez
Body: <p>I'm trying to understand distributional RL, based on this <a href="https://mtomassoli.github.io/2017/12/08/distributional_rl/#mjx-eqn-eqclassic_td_error" rel="nofollow noreferrer">article</a>. In one of the equations, there is a symbol <span class="math-container">$\operatorname{sup dist}$</span>. </p>
  
  <p><span class="math-container">\begin{align}
  \operatorname{sup dist}_{s, a} (R(s, a) + \gamma Z(s', a^*), Z(s, a)) \\
  s' \sim p(\cdot \mid s, a)
  \end{align}</span></p>
  
  <p>What does <span class="math-container">$\operatorname{sup dist}$</span> mean? </p>
  
Tags: <reinforcement-learning><math><notation>
Title: What does the notation sup dist mean in distributional RL?
UserDisplayName: Full.Of.Life
ThreadText: # What does the notation sup dist mean in distributional RL?
  
  (Asked by: username_0 on 2020-01-06)
  
  I'm trying to understand distributional RL, based on this [article](https://mtomassoli.github.io/2017/12/08/distributional_rl/#mjx-eqn-eqclassic_td_error). In one of the equations, there is a symbol $\operatorname{sup dist}$.
  
  \begin{align}
  \operatorname{sup dist}\_{s, a} (R(s, a) + \gamma Z(s', a^\*), Z(s, a)) \\
  s' \sim p(\cdot \mid s, a)
  \end{align}
  
  What does $\operatorname{sup dist}$ mean?
  
  **Tags:** reinforcement-learning, math, notation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-01-06. Score: 3 | Accepted answer
  
  It doesn't seem that it is a "proper" symbol.
  
  I guess that $\sup$ simply refers to the supremum, that is, you want to select actions that maximize the quantity that comes to the right of $\sup$, while $\text{dist}$ is simply a proxy for any possible distance between distributions. For example, you can replace $\text{dist}$ with the Kullback-Leibler divergence or with the mutual information.
  
  
  ---
  
  