Answers.Body: <p>There is no specific image classification dataset that focuses on spatial relations. However, there are some datasets that include images with spatial relations annotations, such as the <strong><a href="https://visualgenome.org/" rel="nofollow noreferrer">Visual Relationship Detection (VRD)</a></strong> dataset which is a collection of images with annotations of spatial relations between objects.</p>
  <p>Here is a paper <a href="https://arxiv.org/pdf/1706.04215.pdf" rel="nofollow noreferrer">Identifying Spatial Relations in Images using Convolutional Neural Networks</a> that uses pre-trained CNN to learn spatial relations between objects in an image, it makes use of SUN09 and a simplified synthetic dataset to trained the network. the network pays attention to a certain parts of images
  while classifying them for spatial relations.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-17T14:11:18.233Z
Answers.Text: Thank you so much for your answer! I already had a look into the VRD dataset and I think it is suitable for learning relations. But my problem is a bit different in that I do not need to learn the spatial relations (my approach looks at important image parts and finds spatial relations just by the coordinates. Do you think it would be possible to get a subset from such datasets where the ground truth of an image depends on the relations? Thank you also for the linked paper as it gives me further ideas for my research.
Answers.UserDisplayName: Johannes R.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-17T14:55:03.510Z
Answers.Text: Yes, I think it would be possible to get a subset from such datasets where the ground truth of an image depends on the relations. However, it is not clear how useful this would be, since the goal of the datasets is to learn generalizable models from data, and not to create models that only work on a specific subset of data.
Answers.UserDisplayName: Faizy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-17T15:08:53.897Z
Answers.Text: Second, the ground truth of an image may not be entirely determined by the relations between objects in the scene, but may also be influenced by other factors such as lighting or the position of the camera.
Answers.UserDisplayName: Faizy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-17T18:26:04.947Z
Answers.Text: Thanks for pointing that out. I think I need to explain my specific case a bit further: I am researching on Explainable AI that goes beyond visual explanations like LIME, GradCAM etc. and build explanations based on first order logic (like: "the image belongs to this class since the important objects A, B, C are in a particular spatial relation to each other"). I am aware that this topic might be very new and so there is not a lot of datasets that are usable for this particular purpose. At least no sets that go beyond toy datasets.
Answers.UserDisplayName: Johannes R.
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-10-17T13:32:41.857Z
Answers.LastActivityDate: 2022-10-17T13:32:41.857Z
Answers.UserDisplayName: Faizy
Body: <p>My question is pretty much the one asked above. To clarify a bit further: I have only found datasets that do object localization and that also have relations between the objects annotated (like: &quot;Here is the horse, here is the rider. The rider rides the horse&quot;. What I am looking for is a dataset where you can do classification based on spatial relations (Like: &quot;I am a positive example, because the rider is above the horse. I am a negative example, because the rider is below the horse (or besides it, etc)&quot;).</p>
  <p>Any help is appreciated</p>
  
Tags: <classification><image-recognition><datasets>
Title: Is there an image classification dataset where the class depends on spatial relations?
UserDisplayName: Johannes R.
ThreadText: # Is there an image classification dataset where the class depends on spatial relations?
  
  (Asked by: username_0 on 2022-10-17)
  
  My question is pretty much the one asked above. To clarify a bit further: I have only found datasets that do object localization and that also have relations between the objects annotated (like: "Here is the horse, here is the rider. The rider rides the horse". What I am looking for is a dataset where you can do classification based on spatial relations (Like: "I am a positive example, because the rider is above the horse. I am a negative example, because the rider is below the horse (or besides it, etc)").
  
  Any help is appreciated
  
  **Tags:** classification, image-recognition, datasets
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-10-17. Score: 2
  
  There is no specific image classification dataset that focuses on spatial relations. However, there are some datasets that include images with spatial relations annotations, such as the **[Visual Relationship Detection (VRD)](https://visualgenome.org/)** dataset which is a collection of images with annotations of spatial relations between objects.
  
  Here is a paper [Identifying Spatial Relations in Images using Convolutional Neural Networks](https://arxiv.org/pdf/1706.04215.pdf) that uses pre-trained CNN to learn spatial relations between objects in an image, it makes use of SUN09 and a simplified synthetic dataset to trained the network. the network pays attention to a certain parts of images
  while classifying them for spatial relations.
  
  > **Comments:**
  > **username_0**: Thank you so much for your answer! I already had a look into the VRD dataset and I think it is suitable for learning relations. But my problem is a bit different in that I do not need to learn the spatial relations (my approach looks at important image parts and finds spatial relations just by the coordinates. Do you think it would be possible to get a subset from such datasets where the ground truth of an image depends on the relations? Thank you also for the linked paper as it gives me further ideas for my research.  
  > **username_1**: Yes, I think it would be possible to get a subset from such datasets where the ground truth of an image depends on the relations. However, it is not clear how useful this would be, since the goal of the datasets is to learn generalizable models from data, and not to create models that only work on a specific subset of data.  
  > **username_1**: Second, the ground truth of an image may not be entirely determined by the relations between objects in the scene, but may also be influenced by other factors such as lighting or the position of the camera.  
  > **username_0**: Thanks for pointing that out. I think I need to explain my specific case a bit further: I am researching on Explainable AI that goes beyond visual explanations like LIME, GradCAM etc. and build explanations based on first order logic (like: "the image belongs to this class since the important objects A, B, C are in a particular spatial relation to each other"). I am aware that this topic might be very new and so there is not a lot of datasets that are usable for this particular purpose. At least no sets that go beyond toy datasets.  
  
  ---
  
  