Answers.Body: <p>Interesting question(s). I understand an 'agent' as an entity that perceives its environment, performs appropriate actions, and can to a certain degree adapt its behavior. If such an agent is also endowed with an understanding of the consequences of its actions the agent also has agency. Of course, the complexity of agents varies widely with respect to the complexity of their forward dynamics model, the amount of domains an agent operates in, and the degrees of freedom an agent posses in its actions. Both the idea of what an agent is and what agency constitutes should best be seen on a spectrum instead of as binary concepts (my dog likely has more agency than most RL models, but both show signs of agency).</p>
  <p>Video games provide a minimal (controllable) set up to test a range of capabilities. They provide a safe and reliable testing bed that one can quickly and cheaply iterate on.</p>
  <p>RL is a subdomain of AI, which as a field is interested in intelligent agents. Many domains, such as cognitive neuroscience, computational neuroscience, behavioral economics, and others are also interested in the study and modeling of intelligent agents. Building and applying those is however particularly relevant in RL and robotics.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T15:26:40.223Z
Answers.Text: I guess the form of agency I’m referring to is “performs appropriate actions”. Why pursue an “agent” that takes actions instead of a tool that e.g. describes actions to take? Why are we interested in agents basically? What advantages do they have over e.g. a transformer?
Answers.UserDisplayName: thesofakillers
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T20:21:34.170Z
Answers.Text: I'm not sure that I can follow. In the most general sense, you could consider every output the model produces as an action (e.g. a language model producing a text output). Such a language model could be based on a transformer architecture but still fall within the definition of being an agent if it its influences upon the environment are interactive. The term 'agent' can thus be used as an umbrella term for a broad range of models with different architectures.
Answers.UserDisplayName: Mariusmarten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T20:27:23.167Z
Answers.Text: I disagree. A vanilla LM is barely interacting with its environment - you could argue at most that it interacts with the environment only indirectly by influencing the actions of actual agents (humans) with its output. For me, an agent is something that _directly_ interacts with its environment (e.g. manipulate it, guide its own perception of it, etc.). LM’s are just tools at the mercy of agents
Answers.UserDisplayName: thesofakillers
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T20:37:59.397Z
Answers.Text: The colloquial definition of an agent (in an RL setting) is: "The learner and
  decision-maker is called the agent. The thing it interacts with, comprising
  everything outside the agent, is called the environment. These interact continually, the agent selecting actions and the environment responding to those
  actions and presenting new situations to the agent"(Sutton & Barto). Under this definition, one could consider an LM-based chatbot as an agent. If you have a different definition of an agent, you will of course arrive at a different conclusion.
Answers.UserDisplayName: Mariusmarten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T20:40:40.330Z
Answers.Text: But an LM-based chatbot is not an LM, it’s an agent that uses an LM as a tool. The definition you quote aligns with what I am saying and I don’t see a vanilla LM aligning with it without performing some minor mental gymnastics.
Answers.UserDisplayName: thesofakillers
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T20:56:17.150Z
Answers.Text: An LM that is prompted to predict the next token and does so is performing an action in reaction to an input, so one could consider it as an agent. But tbh this is on the 'minimal' end of the spectrum (it's like considering cells as an agent because they have sensitive and locomotive capacities).
Answers.UserDisplayName: Mariusmarten
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-31T12:43:13.560Z
Answers.LastActivityDate: 2022-07-31T12:52:54.080Z
Answers.LastEditDate: 2022-07-31T12:52:54.080Z
Answers.UserDisplayName: Mariusmarten
Body: <p>Why seek to develop artificially intelligent <em>agents</em>? Are there certain advantages and/or needs provided by such supposed intelligent agents that are preferred to simply using intelligent <em>tools</em> that are devoid of agency (e.g. language models)? If so, what are these needs and advantages that intelligent agents could serve?</p>
  <p>Basically, why would you want agency in an AI Model in the first place?</p>
  <p>Here, I define <strong>agency</strong> as the ability to <em>autonomously</em> perceive and interact with a given environment. Anything capable of agency is then an <strong>agent</strong>. Furthermore, I define</p>
  <ul>
  <li><strong><em>autonomous</em> perception</strong>: the ability to perceive a given environment without the need of an external agent</li>
  <li><strong>interaction</strong>: the ability to change the state of the environment</li>
  </ul>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-07-31T21:11:10.833Z
Comments.Text: Please move your follow-up questions to actual separate follow-up questions, to keep the focus on your main question. You can link back to this one in the body text if you feel it is important.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-01T07:06:44.067Z
Comments.Text: I removed my follow up questions elsewhere for focus, and now my question is closed for lack of focus - How can I make it more focused?
Comments.UserDisplayName: thesofakillers
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-01T07:20:03.080Z
Comments.Text: The close/reopen is a bit laggy because it relies on people voting. I have voted to reopen.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-03T18:28:09.257Z
Comments.Text: When dealing with AI on a *mathematical* level we do not use soft terms, we define them rigorously. I recommend researching mathematical definitions of agents. Everything can be mathematically defined as an agent within some environment (eg a neural network has a subset of a computer's hardware/software as an environment). Though, not everything is an interesting agent. I also argue that a LM is not devoid of agency by definition: "action or intervention, especially such as to produce a particular effect." Agent: "a person or thing that takes an active role or produces a specified effect."
Comments.UserDisplayName: respectful
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-03T19:54:33.573Z
Comments.Text: Many interesting applications of AI/ML do involve agents. And not all AI/ML is about agents. If you want an AI/ML model to control a robot or a video game character then naturally you are trying to train an agent.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-03T21:23:08.063Z
Comments.Text: @thesofakillers not "somehow" but *by definition* of the term "agent." Furthermore, it is not the AI *model* that is the agent it is the *real* hardware and software implementation of that model that becomes the agent. Because then you have "a person or thing that takes an active role or *produces a specified effect*." I think the issue here is that you have an implied level of "complexity" or some other constraints that constitute your own personal definition of "agent" from a more theoretical definition of "agent." Checkout: https://en.wikipedia.org/wiki/Intelligent_agent
Comments.UserDisplayName: respectful
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-03T21:24:54.550Z
Comments.Text: Might be something interesting to discuss in the chat: https://chat.stackexchange.com/rooms/43371/the-singularity
Comments.UserDisplayName: respectful
Tags: <reinforcement-learning><agi><intelligent-agent>
Title: What is the need for _agency_ in AI?
UserDisplayName: thesofakillers
ThreadText: # What is the need for _agency_ in AI?
  
  (Asked by: username_0 on 2022-07-31)
  
  Why seek to develop artificially intelligent *agents*? Are there certain advantages and/or needs provided by such supposed intelligent agents that are preferred to simply using intelligent *tools* that are devoid of agency (e.g. language models)? If so, what are these needs and advantages that intelligent agents could serve?
  
  Basically, why would you want agency in an AI Model in the first place?
  
  Here, I define **agency** as the ability to *autonomously* perceive and interact with a given environment. Anything capable of agency is then an **agent**. Furthermore, I define
  
  * ***autonomous* perception**: the ability to perceive a given environment without the need of an external agent
  * **interaction**: the ability to change the state of the environment
  
  **Tags:** reinforcement-learning, agi, intelligent-agent
  > **Comments:**
  > **username_1**: Please move your follow-up questions to actual separate follow-up questions, to keep the focus on your main question. You can link back to this one in the body text if you feel it is important.  
  > **username_0**: I removed my follow up questions elsewhere for focus, and now my question is closed for lack of focus - How can I make it more focused?  
  > **username_1**: The close/reopen is a bit laggy because it relies on people voting. I have voted to reopen.  
  > **username_2**: When dealing with AI on a *mathematical* level we do not use soft terms, we define them rigorously. I recommend researching mathematical definitions of agents. Everything can be mathematically defined as an agent within some environment (eg a neural network has a subset of a computer's hardware/software as an environment). Though, not everything is an interesting agent. I also argue that a LM is not devoid of agency by definition: "action or intervention, especially such as to produce a particular effect." Agent: "a person or thing that takes an active role or produces a specified effect."  
  > **username_3**: Many interesting applications of AI/ML do involve agents. And not all AI/ML is about agents. If you want an AI/ML model to control a robot or a video game character then naturally you are trying to train an agent.  
  > **username_2**: @thesofakillers not "somehow" but *by definition* of the term "agent." Furthermore, it is not the AI *model* that is the agent it is the *real* hardware and software implementation of that model that becomes the agent. Because then you have "a person or thing that takes an active role or *produces a specified effect*." I think the issue here is that you have an implied level of "complexity" or some other constraints that constitute your own personal definition of "agent" from a more theoretical definition of "agent." Checkout: https://en.wikipedia.org/wiki/Intelligent_agent  
  > **username_2**: Might be something interesting to discuss in the chat: https://chat.stackexchange.com/rooms/43371/the-singularity  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_4 on 2022-07-31. Score: 1
  
  Interesting question(s). I understand an 'agent' as an entity that perceives its environment, performs appropriate actions, and can to a certain degree adapt its behavior. If such an agent is also endowed with an understanding of the consequences of its actions the agent also has agency. Of course, the complexity of agents varies widely with respect to the complexity of their forward dynamics model, the amount of domains an agent operates in, and the degrees of freedom an agent posses in its actions. Both the idea of what an agent is and what agency constitutes should best be seen on a spectrum instead of as binary concepts (my dog likely has more agency than most RL models, but both show signs of agency).
  
  Video games provide a minimal (controllable) set up to test a range of capabilities. They provide a safe and reliable testing bed that one can quickly and cheaply iterate on.
  
  RL is a subdomain of AI, which as a field is interested in intelligent agents. Many domains, such as cognitive neuroscience, computational neuroscience, behavioral economics, and others are also interested in the study and modeling of intelligent agents. Building and applying those is however particularly relevant in RL and robotics.
  
  > **Comments:**
  > **username_0**: I guess the form of agency I’m referring to is “performs appropriate actions”. Why pursue an “agent” that takes actions instead of a tool that e.g. describes actions to take? Why are we interested in agents basically? What advantages do they have over e.g. a transformer?  
  > **username_4**: I'm not sure that I can follow. In the most general sense, you could consider every output the model produces as an action (e.g. a language model producing a text output). Such a language model could be based on a transformer architecture but still fall within the definition of being an agent if it its influences upon the environment are interactive. The term 'agent' can thus be used as an umbrella term for a broad range of models with different architectures.  
  > **username_0**: I disagree. A vanilla LM is barely interacting with its environment - you could argue at most that it interacts with the environment only indirectly by influencing the actions of actual agents (humans) with its output. For me, an agent is something that _directly_ interacts with its environment (e.g. manipulate it, guide its own perception of it, etc.). LM’s are just tools at the mercy of agents  
  > **username_4**: The colloquial definition of an agent (in an RL setting) is: "The learner and decision-maker is called the agent. The thing it interacts with, comprising everything outside the agent, is called the environment. These interact continually, the agent selecting actions and the environment responding to those actions and presenting new situations to the agent"(Sutton & Barto). Under this definition, one could consider an LM-based chatbot as an agent. If you have a different definition of an agent, you will of course arrive at a different conclusion.  
  > **username_0**: But an LM-based chatbot is not an LM, it’s an agent that uses an LM as a tool. The definition you quote aligns with what I am saying and I don’t see a vanilla LM aligning with it without performing some minor mental gymnastics.  
  > **username_4**: An LM that is prompted to predict the next token and does so is performing an action in reaction to an input, so one could consider it as an agent. But tbh this is on the 'minimal' end of the spectrum (it's like considering cells as an agent because they have sensitive and locomotive capacities).  
  
  ---
  
  