Answers.Body: <p>One of the reasons a discount factor is used, is to make sure the reward maximization is a well-defined problem and to make the sum of all rewards convergent.</p>
  
  <p>In the MAB problem, the number of trials is typically finite owing to some sort of budget in the number of trials. Hence, this is less of problem. However, by all means discounts are still valid and helpful in cases where the analysis in asymptotic in the number of trials.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-05T06:26:21.897Z
Answers.Text: Thanks for your response. Except for convergence analysis, in what cases discounted MAB can be meaningful? Is it possible to have an optimal policy in the discounted case different from the undiscounted setting?
Answers.UserDisplayName: Amin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-05T03:57:42.837Z
Answers.LastActivityDate: 2020-06-05T03:57:42.837Z
Answers.UserDisplayName: nyumerics
Body: <p>Many of multi-armed bandit(MAB) algorithms are used when the total reward is the sum of all rewards. However, in RL, the discounted reward is mainly used. Why is the discounted reward not prevailing in MAB problem, and in what cases is this type of modeling valid and might be better?</p>
  
Tags: <reinforcement-learning><rewards><multi-armed-bandits><discount-factor>
Title: When discounted MAB is useful?
UserDisplayName: Amin
ThreadText: # When discounted MAB is useful?
  
  (Asked by: username_0 on 2020-06-04)
  
  Many of multi-armed bandit(MAB) algorithms are used when the total reward is the sum of all rewards. However, in RL, the discounted reward is mainly used. Why is the discounted reward not prevailing in MAB problem, and in what cases is this type of modeling valid and might be better?
  
  **Tags:** reinforcement-learning, rewards, multi-armed-bandits, discount-factor
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-06-05. Score: 1
  
  One of the reasons a discount factor is used, is to make sure the reward maximization is a well-defined problem and to make the sum of all rewards convergent.
  
  In the MAB problem, the number of trials is typically finite owing to some sort of budget in the number of trials. Hence, this is less of problem. However, by all means discounts are still valid and helpful in cases where the analysis in asymptotic in the number of trials.
  
  > **Comments:**
  > **username_0**: Thanks for your response. Except for convergence analysis, in what cases discounted MAB can be meaningful? Is it possible to have an optimal policy in the discounted case different from the undiscounted setting?  
  
  ---
  
  