Update README.md
Browse files
README.md
CHANGED
@@ -541,34 +541,6 @@ the tested cog map output settings.
|
|
541 |
|
542 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/dGN237xaEFsX38552rX1l.png" width="800"/>
|
543 |
|
544 |
-
|
545 |
-
|
546 |
-
| Method | Overall | Rotation | Among | Around |
|
547 |
-
|--------------------------------------|:-------:|:--------:|:-----:|:------:|
|
548 |
-
| **Baseline** | | | | |
|
549 |
-
| Random (chance) | 32.35 | 36.36 | 32.29 | 30.66 |
|
550 |
-
| Random (frequency) | 33.02 | 38.30 | 32.66 | 35.79 |
|
551 |
-
| **Open-Weight Multi-Image Models** | | | | |
|
552 |
-
| LLaVA-Onevision-7B | 47.43 | 36.45 | 48.42 | 44.09 |
|
553 |
-
| LLaVA-Video-Qwen-7B | 41.96 | 35.71 | 43.55 | 30.12 |
|
554 |
-
| LongVA-7B | 29.46 | 35.89 | 29.55 | 24.88 |
|
555 |
-
| mPLUG-Owl3-7B-241101 | 44.85 | 37.84 | 47.11 | 26.91 |
|
556 |
-
| InternVL2.5-8B | 18.68 | 36.45 | 18.20 | 13.11 |
|
557 |
-
| Qwen2.5-VL-7B-Instruct | 29.26 | 38.76 | 29.50 | 21.35 |
|
558 |
-
| Qwen2.5-VL-3B-Instruct | 33.21 | 37.37 | 33.26 | 30.34 |
|
559 |
-
| Idefics-8B-Llama3 | 35.86 | 35.15 | 35.94 | 35.49 |
|
560 |
-
| DeepSeek-VL2-Small | 47.62 | 37.00 | 50.38 | 26.91 |
|
561 |
-
| Gemma-3-12B-it | 46.67 | 38.39 | 48.38 | 34.63 |
|
562 |
-
| Mantis-8B (SigLip) | 41.05 | 37.65 | 40.23 | 50.99 |
|
563 |
-
| **Proprietary Models** | | | | |
|
564 |
-
| GPT-4o | 38.81 | 32.65 | 40.17 | 29.16 |
|
565 |
-
| Claude-4-Sonnet-20250514 | 44.75 | 48.42 | 44.21 | 47.62 |
|
566 |
-
| **Spatial Models** | | | | |
|
567 |
-
| RoboBrain | 37.38 | 35.80 | 38.28 | 29.53 |
|
568 |
-
| SpaceMantis | 22.81 | 37.65 | 21.29 | 29.32 |
|
569 |
-
| Spatial-MLLM | 32.06 | 38.39 | 20.92 | 32.82 |
|
570 |
-
| Space-Qwen | 33.28 | 38.02 | 33.71 | 26.32 |
|
571 |
-
| 🧘♂️ **SpaceOm** | **39.46** | 33.92 | 37.58 | **48.40** |
|
572 |
|
573 |
See the [results](https://huggingface.co/datasets/salma-remyx/SpaceOm_MindCube_Results/tree/main) of the [MindCube benchmark](https://arxiv.org/pdf/2506.21458) evaluation from [Spatial Mental Modeling from Limited Views](https://arxiv.org/pdf/2506.21458).
|
574 |
|
|
|
541 |
|
542 |
<img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/dGN237xaEFsX38552rX1l.png" width="800"/>
|
543 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
544 |
|
545 |
See the [results](https://huggingface.co/datasets/salma-remyx/SpaceOm_MindCube_Results/tree/main) of the [MindCube benchmark](https://arxiv.org/pdf/2506.21458) evaluation from [Spatial Mental Modeling from Limited Views](https://arxiv.org/pdf/2506.21458).
|
546 |
|