ykarmesh commited on
Commit
9359633
Β·
verified Β·
1 Parent(s): dc5c9a3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -6
README.md CHANGED
@@ -12,15 +12,20 @@ tags:
12
  - embodied-ai
13
  - memory
14
  ---
 
15
  <a href="https://arxiv.org/abs/2506.15635" target="_blank">
16
  <img alt="arXiv" src="https://img.shields.io/badge/arXiv-FindingDory-red?logo=arxiv" height="20" />
17
  </a>
18
  <a href="https://findingdory-benchmark.github.io/" target="_blank">
19
  <img alt="Website" src="https://img.shields.io/badge/🌎_Website-FindingDory-blue.svg" height="20" />
20
  </a>
21
- <a href="https://github.com/findingdory-benchmark/findingdory-trl" target="_blank" style="display: inline-block; margin-right: 10px;">
22
  <img alt="GitHub Code" src="https://img.shields.io/badge/Code-FindingDory--TRL-white?&logo=github&logoColor=white" />
23
  </a>
 
 
 
 
24
 
25
  <center><h1>FindingDory: A Benchmark to Evaluate Memory in Embodied Agents</h1>
26
  <a href="https://www.karmeshyadav.com/">Karmesh Yadav*</a>,
@@ -38,14 +43,14 @@ At deployment the image corresponding to the index is fed into a low-level navig
38
  πŸ‹οΈ Training details
39
  | Property | Value |
40
  | -------- | ----- |
41
- | Epochs | 5 |
42
  | Effective batch | 32 |
43
  | LR schedule | Cosine (LR=5e-6, Warmup ratio=0.1) |
44
- | Image resol. | TODO |
45
- | Compute | β€œ8 Γ— A40 48 GB for ~18 hours” |
46
- | Input frames | 96 Images |
47
  | Optimiser | AdamW(β₁ = 0.9, Ξ²β‚‚ = 0.95) |
48
- | Best checkpoint | TODO |
49
 
50
 
51
  πŸ“Š Evaluation
@@ -57,6 +62,7 @@ We compare the performance of our finetuned `FindingDory-Qwen2.5-VL-3B-SFT` chec
57
  | Gemma3-12B-it | 13.2% | zero-shot |
58
  | GPT-4o | 27.3% | zero-shot |
59
  | Gemini-2.0-Flash | 25.4% | zero-shot |
 
60
  Checkout Fig 2 in the paper for more details.
61
 
62
  πŸ“„ Citation
 
12
  - embodied-ai
13
  - memory
14
  ---
15
+ <center>
16
  <a href="https://arxiv.org/abs/2506.15635" target="_blank">
17
  <img alt="arXiv" src="https://img.shields.io/badge/arXiv-FindingDory-red?logo=arxiv" height="20" />
18
  </a>
19
  <a href="https://findingdory-benchmark.github.io/" target="_blank">
20
  <img alt="Website" src="https://img.shields.io/badge/🌎_Website-FindingDory-blue.svg" height="20" />
21
  </a>
22
+ <a href="https://github.com/findingdory-benchmark/findingdory-trl" target="_blank">
23
  <img alt="GitHub Code" src="https://img.shields.io/badge/Code-FindingDory--TRL-white?&logo=github&logoColor=white" />
24
  </a>
25
+ <a href="https://huggingface.co/datasets/yali30/findingdory/" target="_blank"">
26
+ <img alt="Huggingface" src="https://img.shields.io/badge/Dataset-FindingDory-yellow?logo=huggingface" />
27
+ </a>
28
+ </center>
29
 
30
  <center><h1>FindingDory: A Benchmark to Evaluate Memory in Embodied Agents</h1>
31
  <a href="https://www.karmeshyadav.com/">Karmesh Yadav*</a>,
 
43
  πŸ‹οΈ Training details
44
  | Property | Value |
45
  | -------- | ----- |
46
+ | Epochs | 5 (Total training steps 12840) |
47
  | Effective batch | 32 |
48
  | LR schedule | Cosine (LR=5e-6, Warmup ratio=0.1) |
49
+ | Max Pixels. | 360 x 420 |
50
+ | Compute | β€œ8 Γ— A40 48 GB for ~84 hours” |
51
+ | Input frames | 96 Images (~10k tokens) |
52
  | Optimiser | AdamW(β₁ = 0.9, Ξ²β‚‚ = 0.95) |
53
+ | Best checkpoint | 8800 Steps |
54
 
55
 
56
  πŸ“Š Evaluation
 
62
  | Gemma3-12B-it | 13.2% | zero-shot |
63
  | GPT-4o | 27.3% | zero-shot |
64
  | Gemini-2.0-Flash | 25.4% | zero-shot |
65
+
66
  Checkout Fig 2 in the paper for more details.
67
 
68
  πŸ“„ Citation