  This paper presents a hierarchical framework for planning and control of
in-hand manipulation of a rigid object involving grasp changes using
fully-actuated multifingered robotic hands. While the framework can be applied
to the general dexterous manipulation, we focus on a more complex definition of
in-hand manipulation, where at the goal pose the hand has to reach a grasp
suitable for using the object as a tool. The high level planner determines the
object trajectory as well as the grasp changes, i.e. adding, removing, or
sliding fingers, to be executed by the low-level controller. While the grasp
sequence is planned online by a learning-based policy to adapt to variations,
the trajectory planner and the low-level controller for object tracking and
contact force control are exclusively model-based to robustly realize the plan.
By infusing the knowledge about the physics of the problem and the low-level
controller into the grasp planner, it learns to successfully generate grasps
similar to those generated by model-based optimization approaches, obviating
the high computation cost of online running of such methods to account for
variations. By performing experiments in physics simulation for realistic tool
use scenarios, we show the success of our method on different tool-use tasks
and dexterous hand models. Additionally, we show that this hybrid method offers
more robustness to trajectory and task variations compared to a model-based
method.

Robotics