Answers.Body: <p>This is called multiple instance learning, and its deep version is called deep multiple instance learning, there are many algorithms to train a model considering multiple instances at the same time.</p>
  <p>Here is a survey paper <a href="https://arxiv.org/abs/1612.03365" rel="nofollow noreferrer">Multiple Instance Learning: A Survey of Problem Characteristics and Applications</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-07T08:06:19.143Z
Answers.Text: Thanks for pointing to the keyword! that's exactly what I'm looking for :)
Answers.UserDisplayName: misakayu
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-11-07T08:01:31.797Z
Answers.LastActivityDate: 2022-11-07T08:01:31.797Z
Answers.UserDisplayName: Dr. Snoopy
Body: <p>I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.</p>
  <p>Suppose I have a dataset, each data point contains a few images (&lt; 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.</p>
  <p>For example,<br />
  if the images are {apple, apple, apple} then the label is false.<br />
  if the images are {apple, basketball, basketball} then the label is true.</p>
  <p>I would like to build a model to classify whether those group of images contain such unrelated images.</p>
  <p>Seems the idea appeared in my mind is to have a model such as following</p>
  <ol>
  <li>image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding</li>
  <li>a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.</li>
  <li>flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.</li>
  </ol>
  <p>Loss function would just be the binary cross entropy with the actual label.</p>
  <p>I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!</p>
  
Tags: <machine-learning><deep-learning><computer-vision><attention>
Title: Group image classification for whether containing unrelated images
UserDisplayName: misakayu
ThreadText: # Group image classification for whether containing unrelated images
  
  (Asked by: username_0 on 2022-11-07)
  
  I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.
  
  Suppose I have a dataset, each data point contains a few images (< 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.
  
  For example,  
  if the images are {apple, apple, apple} then the label is false.  
  if the images are {apple, basketball, basketball} then the label is true.
  
  I would like to build a model to classify whether those group of images contain such unrelated images.
  
  Seems the idea appeared in my mind is to have a model such as following
  
  1. image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding
  2. a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.
  3. flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.
  
  Loss function would just be the binary cross entropy with the actual label.
  
  I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!
  
  **Tags:** machine-learning, deep-learning, computer-vision, attention
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-11-07. Score: 1
  
  This is called multiple instance learning, and its deep version is called deep multiple instance learning, there are many algorithms to train a model considering multiple instances at the same time.
  
  Here is a survey paper [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://arxiv.org/abs/1612.03365)
  
  > **Comments:**
  > **username_0**: Thanks for pointing to the keyword! that's exactly what I'm looking for :)  
  
  ---
  
  