content.raw: A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓
content.type: text
content.value: A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓
content.type: new_line
content.type: new_line
content.raw: As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
content.type: text
content.value: As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
content.type: new_line
content.raw: Dataset 
content.type: text
content.value: Dataset 
content.raw: https://huggingface.co/datasets/MM-UPD/MM-UPD
content.id: MM-UPD/MM-UPD
content.type: dataset
content.type: resource
content.url: https://huggingface.co/datasets/MM-UPD/MM-UPD
content.type: new_line
content.raw: Paper 
content.type: text
content.value: Paper 
content.label: Unsolvable Problem Detection: Evaluating Trustworthiness of Vision
    Language Models (2403.20331)
content.raw: https://huggingface.co/papers/2403.20331
content.id: 2403.20331
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2403.20331
rawContent: A great vision language benchmark: MM-UPD evaluates how model responds to unsolvable problems 🤓
  
  As of now, most VLMs, including GPT-4V and LLaVA-Next-34B, struggle with refusing to answer
  Dataset https://huggingface.co/datasets/MM-UPD/MM-UPD
  Paper https://huggingface.co/papers/2403.20331
reactions.reaction: 👍
reactions.users: KingNish qubvel-hf AlekseiPravdin raincandy-u anldrms