Answers.Body: <p>Yes this is actually a common problem with modern object detectors, that detections change with the background, which means that the detector looks beyond the bounding box and into the background, which can have positive or negative impacts.</p>
  <p>Take the example below from the paper <a href="https://arxiv.org/abs/1808.03305" rel="nofollow noreferrer">The Elephant in the room</a>.</p>
  <p><a href="https://i.sstatic.net/6NRLo.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/6NRLo.png" alt="Figure from paper The Elephant in the room, showing how detections change with background" /></a></p>
  <p>You can see in the figure how the cat/zebra detections change with different backgrounds, showing how the context/background affects object detectors.</p>
  <p>In your case, you want this to happen, so you can use Faster R-CNN, SSD, or Mask R-CNN, that are known to have this &quot;problem&quot; according to the elephant in the room paper. Note that this is not an exhaustive list, other detectors might show the same issue.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-25T14:26:22.310Z
Answers.LastActivityDate: 2022-12-25T14:31:43.867Z
Answers.LastEditDate: 2022-12-25T14:31:43.867Z
Answers.UserDisplayName: Dr. Snoopy
Answers.Body: <p>Sure, this can happen. If you have toy cars that in your training data were always photographed in toy shops and then during test you provide images of toy cars in a different environment, you may have situations where they will be classified incorrectly. As long as the neural network sees the context around the object during training, it will also capture information about the context along with information about the object. And if the conext is biased towards a specific environment (i.e. a toy shop), that will also affect the decision significantly.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-12-25T11:45:02.563Z
Answers.LastActivityDate: 2022-12-25T11:45:02.563Z
Answers.UserDisplayName: Otniel-Bogdan Mercea
Body: <p>I am doing an experiment. The following image is an example of the annotation I do. There are 2 classes: 1) sun, 2) moon. Red boundary box labels the moon, and the green boundary box labels the sun. I would like the model to learn that: &quot;if the background is dark blue, it is the moon. If it is light blue, it is the sun&quot;</p>
  <p>I intentionally make the boundary box exclude the surrounding (the blue background), so to test whether an algorithm can distinguish the same object as different classes only based on different surroundings.</p>
  <p>This would be useful, for example, to detect a toy car vs a real car. Assuming the toy car and real car looks very similar, the object detection algorithm have to be aware of its surrounding.</p>
  <p>Do you think popular algorithm such as FRCNN can achieve that? If not, what algorithm is available to solve this problem?</p>
  <p><a href="https://i.sstatic.net/A3R0T.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/A3R0T.png" alt="enter image description here" /></a></p>
  
Tags: <object-detection>
Title: Can object detection algorithms distinguish a same object as different classes only based on different surroundings?
UserDisplayName: Raymond Pang
ThreadText: # Can object detection algorithms distinguish a same object as different classes only based on different surroundings?
  
  (Asked by: username_0 on 2022-12-25)
  
  I am doing an experiment. The following image is an example of the annotation I do. There are 2 classes: 1) sun, 2) moon. Red boundary box labels the moon, and the green boundary box labels the sun. I would like the model to learn that: "if the background is dark blue, it is the moon. If it is light blue, it is the sun"
  
  I intentionally make the boundary box exclude the surrounding (the blue background), so to test whether an algorithm can distinguish the same object as different classes only based on different surroundings.
  
  This would be useful, for example, to detect a toy car vs a real car. Assuming the toy car and real car looks very similar, the object detection algorithm have to be aware of its surrounding.
  
  Do you think popular algorithm such as FRCNN can achieve that? If not, what algorithm is available to solve this problem?
  
  [![enter image description here](https://i.sstatic.net/A3R0T.png)](https://i.sstatic.net/A3R0T.png)
  
  **Tags:** object-detection
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-12-25. Score: 2 | Accepted answer
  
  Yes this is actually a common problem with modern object detectors, that detections change with the background, which means that the detector looks beyond the bounding box and into the background, which can have positive or negative impacts.
  
  Take the example below from the paper [The Elephant in the room](https://arxiv.org/abs/1808.03305).
  
  [![Figure from paper The Elephant in the room, showing how detections change with background](https://i.sstatic.net/6NRLo.png)](https://i.sstatic.net/6NRLo.png)
  
  You can see in the figure how the cat/zebra detections change with different backgrounds, showing how the context/background affects object detectors.
  
  In your case, you want this to happen, so you can use Faster R-CNN, SSD, or Mask R-CNN, that are known to have this "problem" according to the elephant in the room paper. Note that this is not an exhaustive list, other detectors might show the same issue.
  
  
  ---
  
  ### Answer by username_2 on 2022-12-25. Score: 1
  
  Sure, this can happen. If you have toy cars that in your training data were always photographed in toy shops and then during test you provide images of toy cars in a different environment, you may have situations where they will be classified incorrectly. As long as the neural network sees the context around the object during training, it will also capture information about the context along with information about the object. And if the conext is biased towards a specific environment (i.e. a toy shop), that will also affect the decision significantly.
  
  
  ---
  
  