Answers.Body: <blockquote>
  <p>Is there an analogy between client/server in web development and agent/environment in reinforcement learning?</p>
  </blockquote>
  <p>The answer is &quot;not really&quot;. There is no useful analogy here that allows any insight into RL from web server knowledge or vice versa.</p>
  <p>However, you <em>could</em> set up an agent where the goal was to collect information, and the available actions were to make web requests. Clearly, in order to do this, you would need to make use of the client/server model for web servers, with the agent having control over the client web requests, and the environment being the network and servers of the world wide web.</p>
  <p>There are some very hard challenges to construct an open-ended &quot;web assistant&quot; agent. Here are a couple that I can think of:</p>
  <ul>
  <li><p>How to describe actions? Starting with raw web requests composed as strings would likely be very frustrating. Probably you would simplify and have a first action be a call to a search engine with some variation of the topic description, and then decisions about which links to follow, or perhaps whether to refine the search to better fetch sites related to the topic as it is being built.</p>
  </li>
  <li><p>How to create a model of reward for collecting information? The first major stumbling block would be to measure the amount of <em>useful</em> information that the agent had found on any request.</p>
  </li>
  </ul>
  <p>I think with current levels of Natural Language Processing, setting an agent free to discover information according to some goal from a text topic description is a very hard task, beyond cutting edge research. It would definitely be unreasonable to expect any such agent to end up with any resemblance to &quot;understanding&quot; subject matter from text. The agent would have very little ability to tell the difference between accurate facts or lies, or just grammatically correct gibberish.</p>
  <p>One interesting idea for agents trying to learn unsupervised from exploring an environment is creating a reward signal from data compression. An agent will have learned something useful about its environment, if on processing new information, it is able to compress its existing world model more efficiently. This is basic concept behind ideas of general learning agents from <a href="https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber" rel="nofollow noreferrer">Jürgen Schmidhuber</a>, <a href="https://en.wikipedia.org/wiki/Marcus_Hutter" rel="nofollow noreferrer">Marcus Hutter</a> and others. Research into this idea <em>could</em> be a driver towards creating more generic AI learning systems - however, it is one idea amongst many in AI, and so far is research-only, it has not yet led to anything as practical as an AI web-searching assistant.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-06-26T21:12:01.070Z
Answers.LastActivityDate: 2021-12-16T18:15:56.033Z
Answers.LastEditDate: 2021-12-16T18:15:56.033Z
Answers.UserDisplayName: Neil Slater
Body: <p>I've recently come across the client-server model. From my understanding, the client requests the server, to which the server responds with a response. In this case, both the request and responses are vectors.</p>
  <p>In reinforcement learning, the agent communicates with the environment via an &quot;action&quot;, to which the environment sends a scalar reward signal. The &quot;goal&quot; is to maximize this scalar reward signal in long run.</p>
  <p>Is there an analogy between client/server in web development and agent/environment in reinforcement learning?</p>
  
Tags: <reinforcement-learning>
Title: Is there an analogy between client/server in web development and agent/environment in reinforcement learning?
UserDisplayName: thecomplexitytheorist
ThreadText: # Is there an analogy between client/server in web development and agent/environment in reinforcement learning?
  
  (Asked by: username_0 on 2018-06-26)
  
  I've recently come across the client-server model. From my understanding, the client requests the server, to which the server responds with a response. In this case, both the request and responses are vectors.
  
  In reinforcement learning, the agent communicates with the environment via an "action", to which the environment sends a scalar reward signal. The "goal" is to maximize this scalar reward signal in long run.
  
  Is there an analogy between client/server in web development and agent/environment in reinforcement learning?
  
  **Tags:** reinforcement-learning
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-06-26. Score: 3 | Accepted answer
  
  > Is there an analogy between client/server in web development and agent/environment in reinforcement learning?
  
  The answer is "not really". There is no useful analogy here that allows any insight into RL from web server knowledge or vice versa.
  
  However, you *could* set up an agent where the goal was to collect information, and the available actions were to make web requests. Clearly, in order to do this, you would need to make use of the client/server model for web servers, with the agent having control over the client web requests, and the environment being the network and servers of the world wide web.
  
  There are some very hard challenges to construct an open-ended "web assistant" agent. Here are a couple that I can think of:
  
  * How to describe actions? Starting with raw web requests composed as strings would likely be very frustrating. Probably you would simplify and have a first action be a call to a search engine with some variation of the topic description, and then decisions about which links to follow, or perhaps whether to refine the search to better fetch sites related to the topic as it is being built.
  * How to create a model of reward for collecting information? The first major stumbling block would be to measure the amount of *useful* information that the agent had found on any request.
  
  I think with current levels of Natural Language Processing, setting an agent free to discover information according to some goal from a text topic description is a very hard task, beyond cutting edge research. It would definitely be unreasonable to expect any such agent to end up with any resemblance to "understanding" subject matter from text. The agent would have very little ability to tell the difference between accurate facts or lies, or just grammatically correct gibberish.
  
  One interesting idea for agents trying to learn unsupervised from exploring an environment is creating a reward signal from data compression. An agent will have learned something useful about its environment, if on processing new information, it is able to compress its existing world model more efficiently. This is basic concept behind ideas of general learning agents from [Jürgen Schmidhuber](https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber), [Marcus Hutter](https://en.wikipedia.org/wiki/Marcus_Hutter) and others. Research into this idea *could* be a driver towards creating more generic AI learning systems - however, it is one idea amongst many in AI, and so far is research-only, it has not yet led to anything as practical as an AI web-searching assistant.
  
  
  ---
  
  