Answers.Body: <p>Reinforcement Learning is a method for learning to perform beneficial actions in an environment. One way this is accomplished is by learning to predict useful actions as a function of the observed state of the environment. Another is by learning to predict the expected utility gain of doing an action in a particular observed state. Usually the fact that the agent executes a <em>sequence</em> of actions is exploited to learn more rapidly.</p>
  
  <p>In contrast, the problem you describe is to solve a linear system of equations, which is to say, to learn some hidden values <span class="math-container">$x$</span> such that <span class="math-container">$Ax = b$</span> for known <span class="math-container">$A$</span> and <span class="math-container">$b$</span>. Gradient decent is a natural way to solve this problem because it is easy to calculate the gradient, and, since the matrices and vectors have the same dimensionality, it is reasonable to expect that the optimization surface is smooth with a single global minima.</p>
  
  <p>While RL techniques <em>could</em> be applied to solve this problem (I guess by predicting values of X in response to inputs b?), none of the usual features that privilege the RL approach over a standard supervised learning approach are present (there's no sequential relationship between datapoints, there's just a list of constraints essentially). </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-10-13T20:09:19.973Z
Answers.LastActivityDate: 2018-10-13T20:09:19.973Z
Answers.UserDisplayName: John Doucette
Body: <p>Sparse linear systems are normally solved by using solvers like MINRES, Conjugate gradient, GMRES. </p>
  
  <p>Efficient preconditioning, i.e., finding a matrix P such that PAx = Pb is easier to solve than the original problem, can drastically reduce the computational effort to solve for x. However, preconditioning is normally problem-specific and there is not ONE preconditioner that works well for every problem.</p>
  
  <p>I thought this would be an interesting problem to apply RL since there are certain norms (e.g. condition number of matrix PA) to measure if P is a good preconditioner, but I could not find any research in this field.</p>
  
  <p>Is there a specific problem why RL could not be applied?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-13T08:48:25.417Z
Comments.Text: Why would you do that?
Comments.UserDisplayName: Martin Thoma
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-10-13T18:19:12.543Z
Comments.Text: You are right, I was wondering about a different question and went to far. I will restate my question.
Comments.UserDisplayName: timudk
Tags: <reinforcement-learning><linear-algebra>
Title: Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b
UserDisplayName: timudk
ThreadText: # Using reinforcement learning to find a preconditioner for linear systems of the form Ax = b
  
  (Asked by: username_0 on 2018-10-12)
  
  Sparse linear systems are normally solved by using solvers like MINRES, Conjugate gradient, GMRES.
  
  Efficient preconditioning, i.e., finding a matrix P such that PAx = Pb is easier to solve than the original problem, can drastically reduce the computational effort to solve for x. However, preconditioning is normally problem-specific and there is not ONE preconditioner that works well for every problem.
  
  I thought this would be an interesting problem to apply RL since there are certain norms (e.g. condition number of matrix PA) to measure if P is a good preconditioner, but I could not find any research in this field.
  
  Is there a specific problem why RL could not be applied?
  
  **Tags:** reinforcement-learning, linear-algebra
  
  > **Comments:**
  > **username_1**: Why would you do that?  
  > **username_0**: You are right, I was wondering about a different question and went to far. I will restate my question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2018-10-13. Score: 1
  
  Reinforcement Learning is a method for learning to perform beneficial actions in an environment. One way this is accomplished is by learning to predict useful actions as a function of the observed state of the environment. Another is by learning to predict the expected utility gain of doing an action in a particular observed state. Usually the fact that the agent executes a *sequence* of actions is exploited to learn more rapidly.
  
  In contrast, the problem you describe is to solve a linear system of equations, which is to say, to learn some hidden values $x$ such that $Ax = b$ for known $A$ and $b$. Gradient decent is a natural way to solve this problem because it is easy to calculate the gradient, and, since the matrices and vectors have the same dimensionality, it is reasonable to expect that the optimization surface is smooth with a single global minima.
  
  While RL techniques *could* be applied to solve this problem (I guess by predicting values of X in response to inputs b?), none of the usual features that privilege the RL approach over a standard supervised learning approach are present (there's no sequential relationship between datapoints, there's just a list of constraints essentially).
  
  
  ---
  
  