LAVA: Long-horizon Visual Action based Food Acquisition
<|reference_start|>LAVA: Long-horizon Visual Action based Food Acquisition: Robotic Assisted Feeding (RAF) addresses the fundamental need for individuals with mobility impairments to regain autonomy in feeding themselves. The goal of RAF is to use a robot arm to acquire and transfer food to individuals from the table. Existing RAF methods primarily focus on solid foods, leaving a gap in manipulation strategies for semi-solid and deformable foods. This study introduces Long-horizon Visual Action (LAVA) based food acquisition of liquid, semisolid, and deformable foods. Long-horizon refers to the goal of "clearing the bowl" by sequentially acquiring the food from the bowl. LAVA employs a hierarchical policy for long-horizon food acquisition tasks. The framework uses high-level policy to determine primitives by leveraging ScoopNet. At the mid-level, LAVA finds parameters for primitives using vision. To carry out sequential plans in the real world, LAVA delegates action execution which is driven by Low-level policy that uses parameters received from mid-level policy and behavior cloning ensuring precise trajectory execution. We validate our approach on complex real-world acquisition trials involving granular, liquid, semisolid, and deformable food types along with fruit chunks and soup acquisition. Across 46 bowls, LAVA acquires much more efficiently than baselines with a success rate of 89 +/- 4% and generalizes across realistic plate variations such as different positions, varieties, and amount of food in the bowl. Code, datasets, videos, and supplementary materials can be found on our website.<|reference_end|>
@article{bhaskar2024lava:,
    title={LAVA: Long-horizon Visual Action based Food Acquisition},
    author={Amisha Bhaskar, Rui Liu, Vishnu D. Sharma, Guangyao Shi, Pratap
  Tokekar},
    journal={arXiv preprint arXiv:2403.12876},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.12876},
    primaryClass={cs.RO cs.HC}
}