Answers.Body: <p>Yes, RL is capable of learning complex functions, as it is a very general learning approach. However, if you have a direct goal of learning a complex function from example data, it will not really add anything to that process. Supervised learning will be more efficient.</p>
  <p>RL doesn't directly address or have special mechanisms to deal with the two main issues you have:</p>
  <ul>
  <li><p>Lack of training data</p>
  </li>
  <li><p>Multi-variable optimisation with a very large number of variables</p>
  </li>
  </ul>
  <p>Adding states, timesteps and trail-and-error learning to this problem does not make it more tractable. If the problem <em>naturally</em> presented itself as a sequence of simpler choices, then RL might help somewhat, but as far as I can see you have a very high dimensional function to learn, and there is no benefit from adding a layer of trial-and-error learning on top of it.</p>
  <p>What you probably want is to find ways of constraining the problem, using domain knowledge and/or transfer learning from similar systems. For example, if you already know the types of creature in the photos being turned into meshes, you could probably start by categorising them in order to use some pre-defined meshes that you then adjust, as opposed to starting with figuring out the general plan of the mesh.</p>
  <p>There are already systems that can turn pictures of human subjects into 3D models with approximately correct shape and pose. I would suggest you study those to understand how they work and whether the ideas in them can be re-used for your problem. <a href="https://colab.research.google.com/drive/11z58bl3meSzo6kFqkahMa35G5jmh2Wgt?usp=sharing" rel="nofollow noreferrer">Here is one called PIFuHD</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-18T13:39:08.343Z
Answers.LastActivityDate: 2021-08-18T13:39:08.343Z
Answers.UserDisplayName: Neil Slater
Body: <p>I want to build an AI that can convert an image of a subject into an anatomically accurate 3D model. To do this, I was thinking of adapting the following code for Deep Deterministic Policy Gradient: <a href="https://keras.io/examples/rl/ddpg_pendulum/" rel="nofollow noreferrer">https://keras.io/examples/rl/ddpg_pendulum/</a></p>
  <p>My reasons considering RL:</p>
  <ol>
  <li><p>I don't have the needed skill (3D modelling) to procure a large dataset for the project. I was hoping RL may help me overcome that by adapting to a smaller dataset through state-reward learning. I'm looking mainly for human and animal anatomical models. Those are hard to find in large numbers.</p>
  </li>
  <li><p>My second concern is the required density (polygon count) of these models can be rather high. I'm not sure if it is computationally feasible for a NN to output high density models. However, I'm thinking an RL agent can step through and write each vertex one at a time in a 3D space. As compared to a single output layer in a feed-forward network.</p>
  </li>
  </ol>
  <p>However, that means it will have to handle a rather large state-space (an array of length 50,000 or higher).</p>
  <p>With all of that said, RL has mainly been used in video games and simple control problems from the OpenAI gym. Is it a waste of time to use RL for this level of complexity?</p>
  
Tags: <reinforcement-learning><deep-rl><function-approximation>
Title: Is Reinforcement Learning capable of learning complex functions (such as producing a 3d model given an image)?
UserDisplayName: lazylama
ThreadText: # Is Reinforcement Learning capable of learning complex functions (such as producing a 3d model given an image)?
  
  (Asked by: username_0 on 2021-08-18)
  
  I want to build an AI that can convert an image of a subject into an anatomically accurate 3D model. To do this, I was thinking of adapting the following code for Deep Deterministic Policy Gradient: <https://keras.io/examples/rl/ddpg_pendulum/>
  
  My reasons considering RL:
  
  1. I don't have the needed skill (3D modelling) to procure a large dataset for the project. I was hoping RL may help me overcome that by adapting to a smaller dataset through state-reward learning. I'm looking mainly for human and animal anatomical models. Those are hard to find in large numbers.
  2. My second concern is the required density (polygon count) of these models can be rather high. I'm not sure if it is computationally feasible for a NN to output high density models. However, I'm thinking an RL agent can step through and write each vertex one at a time in a 3D space. As compared to a single output layer in a feed-forward network.
  
  However, that means it will have to handle a rather large state-space (an array of length 50,000 or higher).
  
  With all of that said, RL has mainly been used in video games and simple control problems from the OpenAI gym. Is it a waste of time to use RL for this level of complexity?
  
  **Tags:** reinforcement-learning, deep-rl, function-approximation
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-18. Score: 1
  
  Yes, RL is capable of learning complex functions, as it is a very general learning approach. However, if you have a direct goal of learning a complex function from example data, it will not really add anything to that process. Supervised learning will be more efficient.
  
  RL doesn't directly address or have special mechanisms to deal with the two main issues you have:
  
  * Lack of training data
  * Multi-variable optimisation with a very large number of variables
  
  Adding states, timesteps and trail-and-error learning to this problem does not make it more tractable. If the problem *naturally* presented itself as a sequence of simpler choices, then RL might help somewhat, but as far as I can see you have a very high dimensional function to learn, and there is no benefit from adding a layer of trial-and-error learning on top of it.
  
  What you probably want is to find ways of constraining the problem, using domain knowledge and/or transfer learning from similar systems. For example, if you already know the types of creature in the photos being turned into meshes, you could probably start by categorising them in order to use some pre-defined meshes that you then adjust, as opposed to starting with figuring out the general plan of the mesh.
  
  There are already systems that can turn pictures of human subjects into 3D models with approximately correct shape and pose. I would suggest you study those to understand how they work and whether the ideas in them can be re-used for your problem. [Here is one called PIFuHD](https://colab.research.google.com/drive/11z58bl3meSzo6kFqkahMa35G5jmh2Wgt?usp=sharing).
  
  
  ---
  
  