<p>This is called multiple instance learning, and its deep version is called deep multiple instance learning, there are many algorithms to train a model considering multiple instances at the same time.</p>
<p>Here is a survey paper <a href="https://arxiv.org/abs/1612.03365" rel="nofollow noreferrer">Multiple Instance Learning: A Survey of Problem Characteristics and Applications</a></p>

CC BY-SA 4.0
2022-11-07T08:06:19.143Z
Thanks for pointing to the keyword! that's exactly what I'm looking for :)
misakayu
CC BY-SA 4.0
2022-11-07T08:01:31.797Z
2022-11-07T08:01:31.797Z
Dr. Snoopy
<p>I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.</p>
<p>Suppose I have a dataset, each data point contains a few images (&lt; 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.</p>
<p>For example,<br />
if the images are {apple, apple, apple} then the label is false.<br />
if the images are {apple, basketball, basketball} then the label is true.</p>
<p>I would like to build a model to classify whether those group of images contain such unrelated images.</p>
<p>Seems the idea appeared in my mind is to have a model such as following</p>
<ol>
<li>image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding</li>
<li>a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.</li>
<li>flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.</li>
</ol>
<p>Loss function would just be the binary cross entropy with the actual label.</p>
<p>I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!</p>

<machine-learning><deep-learning><computer-vision><attention>
Group image classification for whether containing unrelated images
misakayu
# Group image classification for whether containing unrelated images

(Asked by: username_0 on 2022-11-07)

I'm kind of new to computer vision, and wondering whether this is any existing researches / solutions to following scenarios.

Suppose I have a dataset, each data point contains a few images (< 20 images), and a corresponding label. The label indicate whether those images contain any unrelated images.

For example,  
if the images are {apple, apple, apple} then the label is false.  
if the images are {apple, basketball, basketball} then the label is true.

I would like to build a model to classify whether those group of images contain such unrelated images.

Seems the idea appeared in my mind is to have a model such as following

1. image decoder (e.g efficient net) to decoder each image in the data point. Then each datapoint contains a few images after decoding
2. a self-attention layer for every image above after decoding to measure a similarity matrix for all images in this data point.
3. flatten above similarity matrix, and then connect to a fc layer into a single node as the prediction result.

Loss function would just be the binary cross entropy with the actual label.

I searched a bit, seems there isn't too much researches about such group classification, so not very sure whether above model make much sense.. Any idea would be appreciate, thanks!

**Tags:** machine-learning, deep-learning, computer-vision, attention


---

## 1 answers

### Answer by username_1 on 2022-11-07. Score: 1

This is called multiple instance learning, and its deep version is called deep multiple instance learning, there are many algorithms to train a model considering multiple instances at the same time.

Here is a survey paper [Multiple Instance Learning: A Survey of Problem Characteristics and Applications](https://arxiv.org/abs/1612.03365)

> **Comments:**
> **username_0**: Thanks for pointing to the keyword! that's exactly what I'm looking for :)  

---

