Update README.md
Browse files
README.md
CHANGED
@@ -12,15 +12,20 @@ tags:
|
|
12 |
- embodied-ai
|
13 |
- memory
|
14 |
---
|
|
|
15 |
<a href="https://arxiv.org/abs/2506.15635" target="_blank">
|
16 |
<img alt="arXiv" src="https://img.shields.io/badge/arXiv-FindingDory-red?logo=arxiv" height="20" />
|
17 |
</a>
|
18 |
<a href="https://findingdory-benchmark.github.io/" target="_blank">
|
19 |
<img alt="Website" src="https://img.shields.io/badge/π_Website-FindingDory-blue.svg" height="20" />
|
20 |
</a>
|
21 |
-
<a href="https://github.com/findingdory-benchmark/findingdory-trl" target="_blank"
|
22 |
<img alt="GitHub Code" src="https://img.shields.io/badge/Code-FindingDory--TRL-white?&logo=github&logoColor=white" />
|
23 |
</a>
|
|
|
|
|
|
|
|
|
24 |
|
25 |
<center><h1>FindingDory: A Benchmark to Evaluate Memory in Embodied Agents</h1>
|
26 |
<a href="https://www.karmeshyadav.com/">Karmesh Yadav*</a>,
|
@@ -38,14 +43,14 @@ At deployment the image corresponding to the index is fed into a low-level navig
|
|
38 |
ποΈ Training details
|
39 |
| Property | Value |
|
40 |
| -------- | ----- |
|
41 |
-
| Epochs | 5 |
|
42 |
| Effective batch | 32 |
|
43 |
| LR schedule | Cosine (LR=5e-6, Warmup ratio=0.1) |
|
44 |
-
|
|
45 |
-
| Compute | β8 Γ A40 48 GB for ~
|
46 |
-
| Input frames | 96 Images |
|
47 |
| Optimiser | AdamW(Ξ²β = 0.9, Ξ²β = 0.95) |
|
48 |
-
| Best checkpoint |
|
49 |
|
50 |
|
51 |
π Evaluation
|
@@ -57,6 +62,7 @@ We compare the performance of our finetuned `FindingDory-Qwen2.5-VL-3B-SFT` chec
|
|
57 |
| Gemma3-12B-it | 13.2% | zero-shot |
|
58 |
| GPT-4o | 27.3% | zero-shot |
|
59 |
| Gemini-2.0-Flash | 25.4% | zero-shot |
|
|
|
60 |
Checkout Fig 2 in the paper for more details.
|
61 |
|
62 |
π Citation
|
|
|
12 |
- embodied-ai
|
13 |
- memory
|
14 |
---
|
15 |
+
<center>
|
16 |
<a href="https://arxiv.org/abs/2506.15635" target="_blank">
|
17 |
<img alt="arXiv" src="https://img.shields.io/badge/arXiv-FindingDory-red?logo=arxiv" height="20" />
|
18 |
</a>
|
19 |
<a href="https://findingdory-benchmark.github.io/" target="_blank">
|
20 |
<img alt="Website" src="https://img.shields.io/badge/π_Website-FindingDory-blue.svg" height="20" />
|
21 |
</a>
|
22 |
+
<a href="https://github.com/findingdory-benchmark/findingdory-trl" target="_blank">
|
23 |
<img alt="GitHub Code" src="https://img.shields.io/badge/Code-FindingDory--TRL-white?&logo=github&logoColor=white" />
|
24 |
</a>
|
25 |
+
<a href="https://huggingface.co/datasets/yali30/findingdory/" target="_blank"">
|
26 |
+
<img alt="Huggingface" src="https://img.shields.io/badge/Dataset-FindingDory-yellow?logo=huggingface" />
|
27 |
+
</a>
|
28 |
+
</center>
|
29 |
|
30 |
<center><h1>FindingDory: A Benchmark to Evaluate Memory in Embodied Agents</h1>
|
31 |
<a href="https://www.karmeshyadav.com/">Karmesh Yadav*</a>,
|
|
|
43 |
ποΈ Training details
|
44 |
| Property | Value |
|
45 |
| -------- | ----- |
|
46 |
+
| Epochs | 5 (Total training steps 12840) |
|
47 |
| Effective batch | 32 |
|
48 |
| LR schedule | Cosine (LR=5e-6, Warmup ratio=0.1) |
|
49 |
+
| Max Pixels. | 360 x 420 |
|
50 |
+
| Compute | β8 Γ A40 48 GB for ~84 hoursβ |
|
51 |
+
| Input frames | 96 Images (~10k tokens) |
|
52 |
| Optimiser | AdamW(Ξ²β = 0.9, Ξ²β = 0.95) |
|
53 |
+
| Best checkpoint | 8800 Steps |
|
54 |
|
55 |
|
56 |
π Evaluation
|
|
|
62 |
| Gemma3-12B-it | 13.2% | zero-shot |
|
63 |
| GPT-4o | 27.3% | zero-shot |
|
64 |
| Gemini-2.0-Flash | 25.4% | zero-shot |
|
65 |
+
|
66 |
Checkout Fig 2 in the paper for more details.
|
67 |
|
68 |
π Citation
|