  Offline imitation learning (IL) refers to learning expert behavior solely
from demonstrations, without any additional interaction with the environment.
Despite significant advances in offline IL, existing techniques find it
challenging to learn policies for long-horizon tasks and require significant
re-training when task specifications change. Towards addressing these
limitations, we present GO-DICE an offline IL technique for goal-conditioned
long-horizon sequential tasks. GO-DICE discerns a hierarchy of sub-tasks from
demonstrations and uses these to learn separate policies for sub-task
transitions and action execution, respectively; this hierarchical policy
learning facilitates long-horizon reasoning. Inspired by the expansive
DICE-family of techniques, policy learning at both the levels transpires within
the space of stationary distributions. Further, both policies are learnt with
goal conditioning to minimize need for retraining when task goals change.
Experimental results substantiate that GO-DICE outperforms recent baselines, as
evidenced by a marked improvement in the completion rate of increasingly
challenging pick-and-place Mujoco robotic tasks. GO-DICE is also capable of
leveraging imperfect demonstration and partial task segmentation when
available, both of which boost task performance relative to learning from
expert demonstrations alone.

Machine Learning;Artificial Intelligence