Answers.Body: <p>The purpose of <em>Reinforcement Learning</em> is to maximize some notion of <strong>cumulative reward</strong>, leading me to the point (1) :
  as far as I understand, there is no timesteps in your problem and the "reward" is immediate.
  Thus, I don't think reinforcement is suitable here.</p>
  
  <p>On an other hand, in <em>supervised learning</em>, <strong>linear regression</strong> is the task of approximating a mapping function (f) from input variables (X) to a <strong>continuous output variable</strong> (y).
  It has much in common with your case. If I am not wrong, you are trying to approximate a function that maps image data to (x,y) coordinates
  (the "sweet spot"). So, I think regression would be a better way to go for you.</p>
  
  <p>You could either <strong>generate a dataset</strong> at first, with images data and associated (x,y) coordinates for only spots validated by your function F, and then train a regression predictive model. Or you could train your model with <strong>online learning</strong>, by generating batch of images and sweet spot coordinates at each step.</p>
  
  <p>Concerning the point (2), it will highly depend on how is made your F function. Since overlapping spots cannot be sweet spots, the simplest would
  be to make your F function return 0 for those spots.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-28T19:44:56.280Z
Answers.LastActivityDate: 2019-05-28T20:01:54.500Z
Answers.LastEditDate: 2019-05-28T20:01:54.500Z
Answers.UserDisplayName: Loheek
Body: <p>I am trying to formulate and solve the following problem of image mutation. Suppose I am trying to insert an object image into a "background" image of several objects, and I will need to look for a "sweet spot" to insert the image:</p>
  
  <p><a href="https://i.sstatic.net/WTxFV.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WTxFV.jpg" alt="enter image description here"></a></p>
  
  <p>I am tentatively trying to formulate the problem into a reinforcement learning process, with the following elements:</p>
  
  <p><strong>0. initial stage:</strong></p>
  
  <ul>
  <li><p>a background image where the location of objects within the image has been marked (let's suppose we have a perfect object detector)</p></li>
  <li><p>another image of a new object, let's say, a human</p></li>
  </ul>
  
  <p><strong>1. action space:</strong></p>
  
  <ul>
  <li>location (x, y) for the object image to be inserted; in that sense the action space is <strong>quite</strong> <strong>large</strong>.</li>
  </ul>
  
  <p><strong>2. environment:</strong></p>
  
  <ul>
  <li><p>each step I will have a new image to "learn from".</p></li>
  <li><p>An oracle function <strong>F</strong> returns 1 or 0 (roughly one computation of <strong>F</strong> takes 30 seconds). 
   This function tells me the latest synthesized image hits the "sweet spot" or not (1 means hit). If so, I will stop the search and return the image.</p></li>
  </ul>
  
  <p><strong>3. constraint:</strong></p>
  
  <p>the newly inserted object shouldn't overlap with the original objects in the figure.  </p>
  
  <p>While my gut feeling is that this problem is somehow similar to the classic "maze escape" problem which can be solved well with reinforcement learning, the action space seems <strong>quite large</strong> in this problem.</p>
  
  <p>So here are my questions:</p>
  
  <ol>
  <li><p>In case I would like to formulate this "beautify" image problem into a "deep" reinforcement learning problem, how can I learn from such large action space? Or is it really suitable for a reinforcement learning process? </p></li>
  <li><p>Can I somehow subsume the "non-overlapping" constraint into the oracle function <strong>F</strong>? If so, how should I decide the reward score? Any principled or empirical way of deciding so? </p></li>
  </ol>
  
Tags: <reinforcement-learning><deep-rl><image-generation>
Title: beautify an image with reinforcement learning
UserDisplayName: lllllllllllll
ThreadText: # beautify an image with reinforcement learning
  
  (Asked by: username_0 on 2019-05-27)
  
  I am trying to formulate and solve the following problem of image mutation. Suppose I am trying to insert an object image into a "background" image of several objects, and I will need to look for a "sweet spot" to insert the image:
  
  [![enter image description here](https://i.sstatic.net/WTxFV.jpg)](https://i.sstatic.net/WTxFV.jpg)
  
  I am tentatively trying to formulate the problem into a reinforcement learning process, with the following elements:
  
  **0. initial stage:**
  
  * a background image where the location of objects within the image has been marked (let's suppose we have a perfect object detector)
  * another image of a new object, let's say, a human
  
  **1. action space:**
  
  * location (x, y) for the object image to be inserted; in that sense the action space is **quite** **large**.
  
  **2. environment:**
  
  * each step I will have a new image to "learn from".
  * An oracle function **F** returns 1 or 0 (roughly one computation of **F** takes 30 seconds).
    This function tells me the latest synthesized image hits the "sweet spot" or not (1 means hit). If so, I will stop the search and return the image.
  
  **3. constraint:**
  
  the newly inserted object shouldn't overlap with the original objects in the figure.
  
  While my gut feeling is that this problem is somehow similar to the classic "maze escape" problem which can be solved well with reinforcement learning, the action space seems **quite large** in this problem.
  
  So here are my questions:
  
  1. In case I would like to formulate this "beautify" image problem into a "deep" reinforcement learning problem, how can I learn from such large action space? Or is it really suitable for a reinforcement learning process?
  2. Can I somehow subsume the "non-overlapping" constraint into the oracle function **F**? If so, how should I decide the reward score? Any principled or empirical way of deciding so?
  
  **Tags:** reinforcement-learning, deep-rl, image-generation
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-05-28. Score: 3 | Accepted answer
  
  The purpose of *Reinforcement Learning* is to maximize some notion of **cumulative reward**, leading me to the point (1) :
  as far as I understand, there is no timesteps in your problem and the "reward" is immediate.
  Thus, I don't think reinforcement is suitable here.
  
  On an other hand, in *supervised learning*, **linear regression** is the task of approximating a mapping function (f) from input variables (X) to a **continuous output variable** (y).
  It has much in common with your case. If I am not wrong, you are trying to approximate a function that maps image data to (x,y) coordinates
  (the "sweet spot"). So, I think regression would be a better way to go for you.
  
  You could either **generate a dataset** at first, with images data and associated (x,y) coordinates for only spots validated by your function F, and then train a regression predictive model. Or you could train your model with **online learning**, by generating batch of images and sweet spot coordinates at each step.
  
  Concerning the point (2), it will highly depend on how is made your F function. Since overlapping spots cannot be sweet spots, the simplest would
  be to make your F function return 0 for those spots.
  
  
  ---
  
  