Multiclass image classification with ViT - computer vision
Hi, are there building blocks available in the repo to extend current single class to **multilabel classification**? 

It should not be very big of an issue. I guess. 
#15978 <|||||>And also I would want to know if we can do prediction interpretation by visualizing attention maps or something like that. <|||||>@NielsRogge <|||||>Hi,

You can easily fine-tune ViT (or any other sequence classifier) using the `problem_type` argument.

```
from transformers import ViTForImageClassification

model = ViTForImageClassification.from_pretrained("google/vit-base-patch16--224", problem_type="multi_label_classification")
```
This will make sure the appropriate loss function is used (i.e. `BCEWithLogitsLoss`).<|||||>> Hi,
> 
> You can easily fine-tune ViT (or any other sequence classifier) using the `problem_type` argument.
> 
> ```
> from transformers import ViTForImageClassification
> 
> model = ViTForImageClassification.from_pretrained("google/vit-base-patch16--224", problem_type="multi_label_classification")
> ```
> 
> This will make sure the appropriate loss function is used (i.e. `BCEWithLogitsLoss`).

And what about dataset? How do I load it from images folder (specifically for multilabel) <|||||>
> And also I would want to know if we can do prediction interpretation by visualizing attention maps or something like that.

This as well please let me know it.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.<|||||>> > Hi,
> > You can easily fine-tune ViT (or any other sequence classifier) using the `problem_type` argument.
> > ```
> > from transformers import ViTForImageClassification
> > 
> > model = ViTForImageClassification.from_pretrained("google/vit-base-patch16--224", problem_type="multi_label_classification")
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > This will make sure the appropriate loss function is used (i.e. `BCEWithLogitsLoss`).
> 
> And what about dataset? How do I load it from images folder (specifically for multilabel)

Hi, im also having trouble with loading the dataset. specifically - how does the model expect the labels for multilabel? did you manage to solve this?<|||||>Hi,

For multi-label classification, you need to make sure that you provide `pixel_values` of shape (batch_size, num_channels, height, width) and `labels` of shape (batch_size, num_labels). The latter contain the one-hot encoded labels.

So let's say you have an image that contains a cat and a remote and your labels are cat, remote, dog and house.

In that case, your labels need to be this: `torch.tensor([[1, 1, 0, 0]])`.<|||||>Thanks! :)