RT-2 AI model translates vision and language into robotic actions
This seem like an cool upgrade from RT1, judging from the result. It seems to now also output the delta of the end-effector pose, which was previously handled by a different motion planner.<p>It does seem like this work (and a lot of robot learning works) are still stuck on position&#x2F;velocity control and not impedance control. Which is essentially output <i>where</i> to go, either closed-loop with a controller or open-loop with a motion planner. This seem to dramatically lower the data requirement but it feel like a fundamental limit to what task we can accomplish.<p>The reason robot manipulation <i>is hard</i> is because we need to take in to the account not just what&#x27;s happening in the world but also how our interaction alters it and how we need to react to that.<p>Robot Learning right now is either Reinforcement Learning or Imitation Learning (often latter) and I&#x27;m not sure <i>how</i> one would collect data that capture this.<p>Edit: I&#x27;m surprised Google still do robotics work, I thought Everyday Robotics was shut down [1]<p>[1] <a href="https:&#x2F;&#x2F;www.therobotreport.com&#x2F;alphabet-closes-everyday-robots-among-layoffs&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.therobotreport.com&#x2F;alphabet-closes-everyday-robo...</a>