salma-remyx commited on
Commit
5f6674d
·
verified ·
1 Parent(s): 58f9e68

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -28
README.md CHANGED
@@ -541,34 +541,6 @@ the tested cog map output settings.
541
 
542
  <img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/dGN237xaEFsX38552rX1l.png" width="800"/>
543
 
544
-
545
-
546
- | Method | Overall | Rotation | Among | Around |
547
- |--------------------------------------|:-------:|:--------:|:-----:|:------:|
548
- | **Baseline** | | | | |
549
- | &nbsp;&nbsp;Random (chance) | 32.35 | 36.36 | 32.29 | 30.66 |
550
- | &nbsp;&nbsp;Random (frequency) | 33.02 | 38.30 | 32.66 | 35.79 |
551
- | **Open-Weight Multi-Image Models** | | | | |
552
- | &nbsp;&nbsp;LLaVA-Onevision-7B | 47.43 | 36.45 | 48.42 | 44.09 |
553
- | &nbsp;&nbsp;LLaVA-Video-Qwen-7B | 41.96 | 35.71 | 43.55 | 30.12 |
554
- | &nbsp;&nbsp;LongVA-7B | 29.46 | 35.89 | 29.55 | 24.88 |
555
- | &nbsp;&nbsp;mPLUG-Owl3-7B-241101 | 44.85 | 37.84 | 47.11 | 26.91 |
556
- | &nbsp;&nbsp;InternVL2.5-8B | 18.68 | 36.45 | 18.20 | 13.11 |
557
- | &nbsp;&nbsp;Qwen2.5-VL-7B-Instruct | 29.26 | 38.76 | 29.50 | 21.35 |
558
- | &nbsp;&nbsp;Qwen2.5-VL-3B-Instruct | 33.21 | 37.37 | 33.26 | 30.34 |
559
- | &nbsp;&nbsp;Idefics-8B-Llama3 | 35.86 | 35.15 | 35.94 | 35.49 |
560
- | &nbsp;&nbsp;DeepSeek-VL2-Small | 47.62 | 37.00 | 50.38 | 26.91 |
561
- | &nbsp;&nbsp;Gemma-3-12B-it | 46.67 | 38.39 | 48.38 | 34.63 |
562
- | &nbsp;&nbsp;Mantis-8B (SigLip) | 41.05 | 37.65 | 40.23 | 50.99 |
563
- | **Proprietary Models** | | | | |
564
- | &nbsp;&nbsp;GPT-4o | 38.81 | 32.65 | 40.17 | 29.16 |
565
- | &nbsp;&nbsp;Claude-4-Sonnet-20250514 | 44.75 | 48.42 | 44.21 | 47.62 |
566
- | **Spatial Models** | | | | |
567
- | &nbsp;&nbsp;RoboBrain | 37.38 | 35.80 | 38.28 | 29.53 |
568
- | &nbsp;&nbsp;SpaceMantis | 22.81 | 37.65 | 21.29 | 29.32 |
569
- | &nbsp;&nbsp;Spatial-MLLM | 32.06 | 38.39 | 20.92 | 32.82 |
570
- | &nbsp;&nbsp;Space-Qwen | 33.28 | 38.02 | 33.71 | 26.32 |
571
- | &nbsp;&nbsp;🧘‍♂️ **SpaceOm** | **39.46** | 33.92 | 37.58 | **48.40** |
572
 
573
  See the [results](https://huggingface.co/datasets/salma-remyx/SpaceOm_MindCube_Results/tree/main) of the [MindCube benchmark](https://arxiv.org/pdf/2506.21458) evaluation from [Spatial Mental Modeling from Limited Views](https://arxiv.org/pdf/2506.21458).
574
 
 
541
 
542
  <img src="https://cdn-uploads.huggingface.co/production/uploads/647777304ae93470ffc28913/dGN237xaEFsX38552rX1l.png" width="800"/>
543
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
544
 
545
  See the [results](https://huggingface.co/datasets/salma-remyx/SpaceOm_MindCube_Results/tree/main) of the [MindCube benchmark](https://arxiv.org/pdf/2506.21458) evaluation from [Spatial Mental Modeling from Limited Views](https://arxiv.org/pdf/2506.21458).
546