A Data Source for Reasoning Embodied Agents
Recent progress in using machine learning models for reasoning tasks has been
driven by novel model architectures, large-scale pre-training protocols, and
dedicated reasoning datasets for fine-tuning. In this work, to further pursue
these advances, we introduce a new data generator for machine reasoning that
integrates with an embodied agent. The generated data consists of templated
text queries and answers, matched with world-states encoded into a database.
The world-states are a result of both world dynamics and the actions of the
agent. We show the results of several baseline models on instantiations of
train sets. These include pre-trained language models fine-tuned on a
text-formatted representation of the database, and graph-structured
Transformers operating on a knowledge-graph representation of the database. We
find that these models can answer some questions about the world-state, but
struggle with others. These results hint at new research directions in
designing neural reasoning models and database representations. Code to
generate the data will be released at github.com/facebookresearch/neuralmemory
6507a684d5578ef7e2ff9514
Jack Lanchantin
claimed_verified
2024-11-20T10:10:41.957Z
65dd30a6ea1a202d3cb469f7
/avatars/14029def026380e6e20822d916b25a72.svg
Jack Lanchantin
user
jcklcn
6507a684d5578ef7e2ff9515
Sainbayar Sukhbaatar
claimed_verified
2024-07-30T16:01:35.863Z
66a8611eb51510d82ed54231
/avatars/ad559e774fee4914091b82c9831ae2a2.svg
Sainbayar Sukhbaatar
user
sainbar
6507a684d5578ef7e2ff9516
Gabriel Synnaeve
admin_assigned
2023-09-18T10:33:39.054Z
630eac7931970d1cd4fbacf2
/avatars/b7ccbddfa745db854dc342be1327cd53.svg
Gabriel Synnaeve
user
gsynnaeve
6507a684d5578ef7e2ff9517
Yuxuan Sun
6507a684d5578ef7e2ff9518
Kavya Srinet
6507a684d5578ef7e2ff9519
Arthur Szlam
A Data Source for Reasoning Embodied Agents
Recent progress in using machine learning models for reasoning tasks has been
driven by novel model architectures, large-scale pre-training protocols, and
dedicated reasoning datasets for fine-tuning. In this work, to further pursue
these advances, we introduce a new data generator for machine reasoning that
integrates with an embodied agent. The generated data consists of templated
text queries and answers, matched with world-states encoded into a database.
The world-states are a result of both world dynamics and the actions of the
agent. We show the results of several baseline models on instantiations of
train sets. These include pre-trained language models fine-tuned on a
text-formatted representation of the database, and graph-structured
Transformers operating on a knowledge-graph representation of the database. We
find that these models can answer some questions about the world-state, but
struggle with others. These results hint at new research directions in
designing neural reasoning models and database representations. Code to
generate the data will be released at github.com/facebookresearch/neuralmemory
A new data generator for machine reasoning is introduced, integrating with an embodied agent, and tests baseline models including pre-trained language models and graph-structured Transformers on a knowledge-graph representation of the database.
machine reasoning data generator embodied agent templated text queries pre-trained language models graph-structured Transformers knowledge-graph representation