Learning Semantically Meaningful Representations Through Embodiment
How do humans acquire a meaningful understanding of the world with little to no supervision or semantic labels provided by the environment? Here we investigate embodiment and a closed loop between action and perception as one key component in this process. We take a close look at the representations learned by a deep reinforcement learning agent that is trained with visual and vector observations collected in a 3D environment with sparse rewards. We show that this agent learns semantically meaningful and stable representations of its environment without receiving any semantic labels. Our results show that the agent learns to represent the action relevant information extracted from pixel input in a wide variety of sparse activation patterns. The quality of the representations learned shows the strength of embodied learning and its advantages over fully supervised approaches with regards to robustness and generalizability.
This work builds on the embodied cognition literature, hypothesizing that representations learned in embodied agents will be of improved “quality” compared to non-embodied models, such as neural networks trained on static supervised datasets. The authors provide an excellent motivation to the work, with the introduction nicely laying the groundwork for their hypothesis. In general the motivation is quite strong, and research along these directions will no doubt be of value to the field. To assess their hypothesis, the authors compare the representations learned in trained vs. random agents on the Unity Obstacle Tower Challenge, and demonstrate that trained agents develop semantically meaningful, sparse representations, without explicit regularization. Unfortunately, the work doesn’t provide adequate baselines to properly assess the hypothesis. With the data presented, we can only make claims about *trained* vs. *untrained* agents (both of which are embodied!). In other words, an embodied agent with a random policy is not equivalent to a non-embodied agent. Thus, the data only support the conclusion that performance and task-relevant policies drive good representation learning in embodied agents, which is altogether not surprising, as one wouldn’t expect representations to be good in a randomly initialized network. To assess wither *embodiment* is a critical factor for driving good representations, the authors should compare to a model that learns from a static supervised dataset. Curiously, this idea is alluded to in the introduction, but not followed up on. Overall, there are some nicely presented ideas but the work is unfortunately incomplete, and the results cannot support the hypothesis laid out. As a final note, the authors are encouraged to remove any assignments of gender to the agent (“he/him” is unnecessarily used throughout).
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Learning Semantically Meaningful Representations Through Embodiment
How do humans acquire a meaningful understanding of the world with little to no supervision or semantic labels provided by the environment? Here we investigate embodiment and a closed loop between action and perception as one key component in this process. We take a close look at the representations learned by a deep reinforcement learning agent that is trained with visual and vector observations collected in a 3D environment with sparse rewards. We show that this agent learns semantically meaningful and stable representations of its environment without receiving any semantic labels. Our results show that the agent learns to represent the action relevant information extracted from pixel input in a wide variety of sparse activation patterns. The quality of the representations learned shows the strength of embodied learning and its advantages over fully supervised approaches with regards to robustness and generalizability.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This work builds on the embodied cognition literature, hypothesizing that representations learned in embodied agents will be of improved “quality” compared to non-embodied models, such as neural networks trained on static supervised datasets. The authors provide an excellent motivation to the work, with the introduction nicely laying the groundwork for their hypothesis. In general the motivation is quite strong, and research along these directions will no doubt be of value to the field. To assess their hypothesis, the authors compare the representations learned in trained vs. random agents on the Unity Obstacle Tower Challenge, and demonstrate that trained agents develop semantically meaningful, sparse representations, without explicit regularization. Unfortunately, the work doesn’t provide adequate baselines to properly assess the hypothesis. With the data presented, we can only make claims about *trained* vs. *untrained* agents (both of which are embodied!). In other words, an embodied agent with a random policy is not equivalent to a non-embodied agent. Thus, the data only support the conclusion that performance and task-relevant policies drive good representation learning in embodied agents, which is altogether not surprising, as one wouldn’t expect representations to be good in a randomly initialized network. To assess wither *embodiment* is a critical factor for driving good representations, the authors should compare to a model that learns from a static supervised dataset. Curiously, this idea is alluded to in the introduction, but not followed up on. Overall, there are some nicely presented ideas but the work is unfortunately incomplete, and the results cannot support the hypothesis laid out. As a final note, the authors are encouraged to remove any assignments of gender to the agent (“he/him” is unnecessarily used throughout).
assistant
This work builds on the embodied cognition literature, hypothesizing that representations learned in embodied agents will be of improved “quality” compared to non-embodied models, such as neural networks trained on static supervised datasets. The authors provide an excellent motivation to the work, with the introduction nicely laying the groundwork for their hypothesis. In general the motivation is quite strong, and research along these directions will no doubt be of value to the field. To assess their hypothesis, the authors compare the representations learned in trained vs. random agents on the Unity Obstacle Tower Challenge, and demonstrate that trained agents develop semantically meaningful, sparse representations, without explicit regularization. Unfortunately, the work doesn’t provide adequate baselines to properly assess the hypothesis. With the data presented, we can only make claims about *trained* vs. *untrained* agents (both of which are embodied!). In other words, an embodied agent with a random policy is not equivalent to a non-embodied agent. Thus, the data only support the conclusion that performance and task-relevant policies drive good representation learning in embodied agents, which is altogether not surprising, as one wouldn’t expect representations to be good in a randomly initialized network. To assess wither *embodiment* is a critical factor for driving good representations, the authors should compare to a model that learns from a static supervised dataset. Curiously, this idea is alluded to in the introduction, but not followed up on. Overall, there are some nicely presented ideas but the work is unfortunately incomplete, and the results cannot support the hypothesis laid out. As a final note, the authors are encouraged to remove any assignments of gender to the agent (“he/him” is unnecessarily used throughout).