A computer-implemented method is provided for hierarchical multi-agent imitation learning. The method includes learning (510) sub-policies for sub-tasks of a hierarchical multi-agent imitation learning task by imitating expert trajectories of expert demonstrations of the subtasks with guidance from a high-level policy corresponding to the hierarchical multi-agent imitation learning task. The method further includes collecting (520) feedback from the sub-policies relating to updating the high-level-policy with a new observation. The method also includes updating (530) the high-level policy with the new observation responsive to the feedback from the sub-policies. The high-level policy is configured as a contextual multi-arm bandit that sequentially selects k best sub-policies at each of a plurality of time steps based on contextual information derived from the expert demonstrations (510).
HIERARCHICAL MULTI-AGENT IMITATION LEARNING WITH CONTEXTUAL BANDITS