Answers.Body: <p>You might be interested in the Clarion cognitive architecture, developed by Prof. Ron Sun and collaborators. </p>
  
  <p><strong>Full disclosure:</strong> I am a student in Ron Sun's Cognitive Architecture Lab.</p>
  
  <h1>Brief Description of Clarion</h1>
  
  <p>Clarion agents are composed of several subsystems, each of which may contain several neural networks. These subsystems include the Action-Centered Subsystem (ACS), the Non-Action-Centered Subsytem. The ACS controls action decision making, the NACS stores general knowledge. During an activation cycle, the ACS might request information from the NACS, setting the NACS activation cycle in motion. </p>
  
  <p>Strictly speaking, Clarion does not adopt the BDI framework. But, various components of a Clarion agent can be put in correspondence with BDI concepts. BDI beliefs correspond roughly to knowledge in the ACS and NACS. The Motivational Subsystem (MS) contains agent drives and sets agent goals, which roughly correspond to BDI desires and intentions respectively. </p>
  
  <p>Clarion agents do not directly control when learning happens, but may control which subsystems are active based on task demands, and may output mental actions as part of their processing. Learning generally happens at the end of a subsystem activation cycle based on feedback from the environment or from the agent's own motivational and metacognitive subsystems.</p>
  
  <h1>Links</h1>
  
  <p><a href="https://sites.google.com/site/drronsun/clarion/clarion-publications" rel="nofollow noreferrer">A page</a> on Prof. Ron Sun's site links to several resources on Clarion. </p>
  
  <p>Currently the most up-to-date reference on Clarion is the book <a href="https://global.oup.com/academic/product/anatomy-of-the-mind-9780199794553?cc=us&amp;lang=en&amp;" rel="nofollow noreferrer">Anatomy of the Mind</a>. <a href="https://link.springer.com/article/10.1007/s12559-016-9444-2" rel="nofollow noreferrer">A precis</a> is available in the journal <em>Cognitive Computation</em>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-09-19T11:06:08.753Z
Answers.LastActivityDate: 2018-09-19T11:06:08.753Z
Answers.UserDisplayName: Can Mekik
Body: <p>Is there research work that uses neural network as the (BDI) agent (or even full-scale cognitive architecture like Soar, OpenCog) - that continuously receives information from the environment and act in an environment and modifies its base of belief in parallel? Usually NN are trained to do only one task and TensorFlow/PyTorch supports batch mode only out of the box. Also NN algorithms and theory are constructed assuming that training and inference phases are clearly separated and they have each own algorithms. So - completely new theory and software can be required for this - are there efforts in this direction? If no, then why not? It is so self-evident that such systems can be of benefit.</p>
  
  <p><a href="https://arxiv.org/abs/1802.07569" rel="nofollow noreferrer">https://arxiv.org/abs/1802.07569</a> is good review about incremental learning and it contains chapters of implemented systems, but all of them still separates learning phase from inference phase. Symbolic systems and symbolic agents (like JSON AgentSpeak) can have updating belief/knowledge base and they can also act during  receiving new information or during forming new beliefs. I am specifically seeking research about NNs which <strong>do learning and inference in parallel</strong>. As far as I sought then this separation still persists in self-organizing incremental NNs that are gaining some popularity.</p>
  
  <p>I can image the construction of chained NNs in Tensorflow - there is some controller network that receives input (possibly preprocessed by hierarchically lower networks) and that decides what to the: s.c. mental actions are the ouput of this controller, these actions determine whether some subordinated network is required to undergo additional learning or whether it can be temporary used for the processing of some information. Central network itself, of course, can decide to move into temporary learning phase from time to time to improve its reasoning capabilities. Such pipeline of master-slave networks is indeed possible in TensorFlow but still TensorFlow will have one central clock, not distributed, loosely connected processing. But I don't know whether existence of central clock is any restriction on the generality of capabilities of such system. Well, this hierarchy of networks maybe can be realized inside the one large network as well - maybe this large network can allow separate parts (subsets of neurons) to function in somehow independent and mutually controlling mode, maybe such regions of large neural network can emerge indeed. <em>I am interested in this kind of research - maybe there are available some good papers for this?</em></p>
  
Tags: <neural-networks><intelligent-agent>
Title: Neural network as (BDI) agent - running in continuous mode (that do inference in parallel with learning)?
UserDisplayName: TomR
ThreadText: # Neural network as (BDI) agent - running in continuous mode (that do inference in parallel with learning)?
  
  (Asked by: username_0 on 2018-09-18)
  
  Is there research work that uses neural network as the (BDI) agent (or even full-scale cognitive architecture like Soar, OpenCog) - that continuously receives information from the environment and act in an environment and modifies its base of belief in parallel? Usually NN are trained to do only one task and TensorFlow/PyTorch supports batch mode only out of the box. Also NN algorithms and theory are constructed assuming that training and inference phases are clearly separated and they have each own algorithms. So - completely new theory and software can be required for this - are there efforts in this direction? If no, then why not? It is so self-evident that such systems can be of benefit.
  
  <https://arxiv.org/abs/1802.07569> is good review about incremental learning and it contains chapters of implemented systems, but all of them still separates learning phase from inference phase. Symbolic systems and symbolic agents (like JSON AgentSpeak) can have updating belief/knowledge base and they can also act during receiving new information or during forming new beliefs. I am specifically seeking research about NNs which **do learning and inference in parallel**. As far as I sought then this separation still persists in self-organizing incremental NNs that are gaining some popularity.
  
  I can image the construction of chained NNs in Tensorflow - there is some controller network that receives input (possibly preprocessed by hierarchically lower networks) and that decides what to the: s.c. mental actions are the ouput of this controller, these actions determine whether some subordinated network is required to undergo additional learning or whether it can be temporary used for the processing of some information. Central network itself, of course, can decide to move into temporary learning phase from time to time to improve its reasoning capabilities. Such pipeline of master-slave networks is indeed possible in TensorFlow but still TensorFlow will have one central clock, not distributed, loosely connected processing. But I don't know whether existence of central clock is any restriction on the generality of capabilities of such system. Well, this hierarchy of networks maybe can be realized inside the one large network as well - maybe this large network can allow separate parts (subsets of neurons) to function in somehow independent and mutually controlling mode, maybe such regions of large neural network can emerge indeed. *I am interested in this kind of research - maybe there are available some good papers for this?*
  
  **Tags:** neural-networks, intelligent-agent
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2018-09-19. Score: 1
  
  You might be interested in the Clarion cognitive architecture, developed by Prof. Ron Sun and collaborators.
  
  **Full disclosure:** I am a student in Ron Sun's Cognitive Architecture Lab.
  
  # Brief Description of Clarion
  
  Clarion agents are composed of several subsystems, each of which may contain several neural networks. These subsystems include the Action-Centered Subsystem (ACS), the Non-Action-Centered Subsytem. The ACS controls action decision making, the NACS stores general knowledge. During an activation cycle, the ACS might request information from the NACS, setting the NACS activation cycle in motion.
  
  Strictly speaking, Clarion does not adopt the BDI framework. But, various components of a Clarion agent can be put in correspondence with BDI concepts. BDI beliefs correspond roughly to knowledge in the ACS and NACS. The Motivational Subsystem (MS) contains agent drives and sets agent goals, which roughly correspond to BDI desires and intentions respectively.
  
  Clarion agents do not directly control when learning happens, but may control which subsystems are active based on task demands, and may output mental actions as part of their processing. Learning generally happens at the end of a subsystem activation cycle based on feedback from the environment or from the agent's own motivational and metacognitive subsystems.
  
  # Links
  
  [A page](https://sites.google.com/site/drronsun/clarion/clarion-publications) on Prof. Ron Sun's site links to several resources on Clarion.
  
  Currently the most up-to-date reference on Clarion is the book [Anatomy of the Mind](https://global.oup.com/academic/product/anatomy-of-the-mind-9780199794553?cc=us&lang=en&). [A precis](https://link.springer.com/article/10.1007/s12559-016-9444-2) is available in the journal *Cognitive Computation*.
  
  
  ---
  
  