Answers.Body: <p>The software that you require to provide your AI agent remote access is called Virtual Network Computing (VNC). A VNC is a desktop sharing system that transmits keyboard and mouse movements, over a network connection, from the server (the AI) to the client (training environment). </p>
  
  <p>This allows your AI agent to use a computer like a human does, which is by looking at the screen pixels and operating a virtual keyboard and mouse. Infact VNC's are used conventionally for remote technical support and accessing files on one's computer remotely. </p>
  
  <p>An excellent use case example is Open AI's Universe platform. Alongside Gym, which is Open AI's toolkit for developing RL algorithms. Universe launches programs behind a VNC remote desktop, which enables the training of AI agents by only showing them the screen pixels and allowing them to operate a virtual keyboard or mouse. </p>
  
  <p>Since you want to train your AI agent on an Android based training environment, my suggestion is that you consider installing either Real VNC (android client) or Droid VNC on your Android device. You can then install the VNC server software on your training machine (i.e. an nvidia docker instance). Your AI agent will now have full access to the training environment screen for reinforcement learning. </p>
  
  <p>I also recommend that you look at this similar question on the network.</p>
  
  <p><a href="https://askubuntu.com/questions/414189/how-to-remotely-control-my-android">https://askubuntu.com/questions/414189/how-to-remotely-control-my-android</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-20T11:08:33.897Z
Answers.LastActivityDate: 2018-01-20T12:51:41.507Z
Answers.LastEditDate: 2018-01-20T12:51:41.507Z
Answers.UserDisplayName: Seth Simba
Body: <p>I've been working on a project (Android Game), in which the player has to confront with some obstacles/enemies whom he has to destroy. So, is there a way in which we can monitor how the user of the game plays and accordingly to generate (and timely updating) that trained model, which can be used later in the game to make the user think of a different way to defeat an enemy unlike going in the same streamline flow which he has followed till now.</p>
  
  <p>I've implemented Q-Learning and Genetic algorithms on the PC, to make the Game AI for some games like 'tetris' to make the computer play on it's own. But, haven't done it in Android till now.</p>
  
  <p>Already searched: I've also referred to some websites in which they suggested using Neural Networks to encounter the same.</p>
  
  <p>But, I'm unable to get an accurate procedure in which this can be done. Please suggest me a way in which I can monitor the user's input on how he plays.</p>
  
  <p>Thank you.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-01T18:30:28.630Z
Comments.Text: There seems to be a great deal of emphasis on visual approach to machine learning.  I find this question interesting because it is oriented on the player's tendencies and preferences.  Welcome to AI!
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-01T18:38:32.023Z
Comments.Text: So, is there a way this can be made possible or do we have any tensorflow APIs to proceed with them, I'm really struck with this problem but, I really want to make this project. Please, suggest me..
Comments.UserDisplayName: Praneet Pabolu
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-01T18:41:42.997Z
Comments.Text: Unfortunately, all of my work is in highly compact, combinatorial games, not requiring visual learning, so I can't help you with specifics.  But hopefully someone with experience in this area will find your question soon.
Comments.UserDisplayName: DukeZhou
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-04T18:41:10.940Z
Comments.Text: Do you want to monitor the user's input in a game you've created (and have the source code) or a game provided by a third party?
Comments.UserDisplayName: Demento
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-09-06T05:12:14.847Z
Comments.Text: User's input... And use that input as my model's input to train the model
Comments.UserDisplayName: Praneet Pabolu
Tags: <machine-learning><game-ai>
Title: Is there any way, in which we can monitor what's going on the screen in android to generate a model from it?
UserDisplayName: Praneet Pabolu
ThreadText: # Is there any way, in which we can monitor what's going on the screen in android to generate a model from it?
  
  (Asked by: username_0 on 2017-09-01)
  
  I've been working on a project (Android Game), in which the player has to confront with some obstacles/enemies whom he has to destroy. So, is there a way in which we can monitor how the user of the game plays and accordingly to generate (and timely updating) that trained model, which can be used later in the game to make the user think of a different way to defeat an enemy unlike going in the same streamline flow which he has followed till now.
  
  I've implemented Q-Learning and Genetic algorithms on the PC, to make the Game AI for some games like 'tetris' to make the computer play on it's own. But, haven't done it in Android till now.
  
  Already searched: I've also referred to some websites in which they suggested using Neural Networks to encounter the same.
  
  But, I'm unable to get an accurate procedure in which this can be done. Please suggest me a way in which I can monitor the user's input on how he plays.
  
  Thank you.
  
  **Tags:** machine-learning, game-ai
  > **Comments:**
  > **username_1**: There seems to be a great deal of emphasis on visual approach to machine learning.  I find this question interesting because it is oriented on the player's tendencies and preferences.  Welcome to AI!  
  > **username_0**: So, is there a way this can be made possible or do we have any tensorflow APIs to proceed with them, I'm really struck with this problem but, I really want to make this project. Please, suggest me..  
  > **username_1**: Unfortunately, all of my work is in highly compact, combinatorial games, not requiring visual learning, so I can't help you with specifics.  But hopefully someone with experience in this area will find your question soon.  
  > **username_2**: Do you want to monitor the user's input in a game you've created (and have the source code) or a game provided by a third party?  
  > **username_0**: User's input... And use that input as my model's input to train the model  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2018-01-20. Score: 1
  
  The software that you require to provide your AI agent remote access is called Virtual Network Computing (VNC). A VNC is a desktop sharing system that transmits keyboard and mouse movements, over a network connection, from the server (the AI) to the client (training environment).
  
  This allows your AI agent to use a computer like a human does, which is by looking at the screen pixels and operating a virtual keyboard and mouse. Infact VNC's are used conventionally for remote technical support and accessing files on one's computer remotely.
  
  An excellent use case example is Open AI's Universe platform. Alongside Gym, which is Open AI's toolkit for developing RL algorithms. Universe launches programs behind a VNC remote desktop, which enables the training of AI agents by only showing them the screen pixels and allowing them to operate a virtual keyboard or mouse.
  
  Since you want to train your AI agent on an Android based training environment, my suggestion is that you consider installing either Real VNC (android client) or Droid VNC on your Android device. You can then install the VNC server software on your training machine (i.e. an nvidia docker instance). Your AI agent will now have full access to the training environment screen for reinforcement learning.
  
  I also recommend that you look at this similar question on the network.
  
  <https://askubuntu.com/questions/414189/how-to-remotely-control-my-android>
  
  
  ---
  
  