A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓
text
A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓
new_line
new_line
As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
text
As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
new_line
Dataset 
text
Dataset 
https://huggingface.co/datasets/MM-UPD/MM-UPD
MM-UPD/MM-UPD
dataset
resource
https://huggingface.co/datasets/MM-UPD/MM-UPD
new_line
Paper 
text
Paper 
Unsolvable Problem Detection: Evaluating Trustworthiness of Vision
  Language Models (2403.20331)
https://huggingface.co/papers/2403.20331
2403.20331
paper
resource
https://huggingface.co/papers/2403.20331
A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓

As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
Dataset https://huggingface.co/datasets/MM-UPD/MM-UPD
Paper https://huggingface.co/papers/2403.20331
👍
KingNish qubvel-hf AlekseiPravdin raincandy-u anldrms