Answers.Body: <p>Convergence analysis is about proving that your policy and/or value function converge to some desired value, which is usually the fixed-point of an operator or an extremum. So it essentially proves that theoretically the algorithm achieves the desired function. Without convergence, we have no guarantees that the value function will be accurate or the policy will be any good, so in other words the proposed RL algorithm can completely fail at serving its purpose even in simple cases.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-16T01:31:05.193Z
Answers.LastActivityDate: 2020-07-16T01:31:05.193Z
Answers.UserDisplayName: harwiltz
Body: <p>While reading a paper about Q-learning in network energy consumption, I came across the section on convergence analysis. Does anyone know what convergence analysis is, and why is convergence analysis needed in reinforcement learning?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-07-15T16:12:58.200Z
Comments.Text: What's the title of the paper you're reading?
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><q-learning><papers><convergence>
Title: What is convergence analysis, and why is it needed in reinforcement learning?
UserDisplayName: Daniel Koh
ThreadText: # What is convergence analysis, and why is it needed in reinforcement learning?
  
  (Asked by: username_0 on 2020-07-15)
  
  While reading a paper about Q-learning in network energy consumption, I came across the section on convergence analysis. Does anyone know what convergence analysis is, and why is convergence analysis needed in reinforcement learning?
  
  **Tags:** reinforcement-learning, q-learning, papers, convergence
  > **Comments:**
  > **username_1**: What's the title of the paper you're reading?  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-07-16. Score: 3
  
  Convergence analysis is about proving that your policy and/or value function converge to some desired value, which is usually the fixed-point of an operator or an extremum. So it essentially proves that theoretically the algorithm achieves the desired function. Without convergence, we have no guarantees that the value function will be accurate or the policy will be any good, so in other words the proposed RL algorithm can completely fail at serving its purpose even in simple cases.
  
  
  ---
  
  