Bayesian Inference of Temporal Task Specifications from Demonstrations
When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task. Prior research into learning from demonstrations (LfD) has failed to capture this notion of the acceptability of an execution; meanwhile, temporal logics provide a flexible language for expressing task specifications. Inspired by this, we present Bayesian specification inference, a probabilistic model for inferring task specification as a temporal logic formula. We incorporate methods from probabilistic programming to define our priors, along with a domain-independent likelihood function to enable sampling-based inference. We demonstrate the efficacy of our model for inferring specifications with over 90% similarity between the inferred specification and the ground truth, both within a synthetic domain and a real-world table setting task.
Summary: The authors extend Bayesian concept learning to infer Linear Temporal Logic (LTL) specifications from demonstrations. The concept space is that of a finite subset of all possible well formed formulas of LTL constrained to a conjunction between three well founded structures for capturing tasks (Global satisfaction, Eventual completion, Temporal ordering). The authors propose three different priors on the structure of the Temporal ordering (Linear chain, Set of linear chains, Forest of sub-tasks) which allow for efficient sampling of candidate formulas. The posteriors using the different priors have different explanatory power of the observed data. The explanatory power depends on how well the observed task adheres to the structure of the respective prior, which further provides insights on which temporal order structure structure that is more suitable for a specific demonstrated task. The authors propose a likelihood (likelihood odds ratio) which they evaluate together with the priors on both toy data and a real world task scenario with good results.   Strengths: I really like the interaction between Bayesian machine learning and temporal logics in the paper. The paper reminds me of the first part of Chapter 3 about Bayesian concept learning in [1] where they show the elegance and power of Bayesian machine learning for learning from only positive demonstrations, although under much simpler circumstances. This is a direction I would like to see more of within the ML community. This paper is original to the best of my knowledge in the sense that the concept space is a finite subset of well formed formulas of a Modal logic (LTL) consisting of three proposed, for the problem well founded, structures as well as the proposal of three distinctly different but suitable priors used for the inference which are also empirically investigated.  I deem the contributions significant and novel. The contributions are important for advancing this line of research into both the integration of ML and logics as well as opening up for future valuable contributions to the reinforcement and learning from demonstration communities.  The paper is clearly written and have a fair presentation of both the probabilistic aspects of Bayesian machine learning and of Linear Temporal Logic (LTL). Everything is not easy to understand at first, but it is rather due to the many important technical details across the applied Bayesian inference and the temporal logic than anything else. The approach makes sense, is reasonable and the experimentation is suitable.  The Algorithm listings help a lot and increase clarity of the paper.  The paper seem to be technically correct and the quality of the paper is in my opinion high.  The experiments are explained in detail and seem to be reproducible with access to the data.   Weaknesses: For an audience which is less familiar with Bayesian machine learning, such as from other parts of the ML field or from the fields of more logic based AI, some parts of the paper can be made clearer. One example is the likelihood odds ratio in equation (7) which is not explicitly shown how it is used as a likelihood (e.g. in the relation to Bayes theorem). I understand why it is sufficient to state it as is, with the context of Bayesian model selection in mind, but it might not be self-evident for a wider audience.   Other comments: It is my understanding that the three priors can be made into a single prior (e.g. with a probability of 1/3 to draw a sample from either of them). What would be the reason for doing this versus how you do it in the paper where you keep them separate as three different inferences?  Some figures are very small. Especially Figure 1, but also Figure 2 and Figure 4. Figure 1 can be made bigger by limiting the plotting to y >= -2.  The caption of Figure 2 seem to be inconsistent (e.g. "Scenario 1" and "Scenario 2 L(\phi)")  [1] Murphy, K. P., Machine Learning - A probabilistic Perspective, 2012, The MIT Press.   --- Thank you for your detailed author feedback.