Diverse Trajectory Forecasting with Determinantal Point Processes
The ability to forecast a set of likely yet diverse possible future behaviors of an agent (e.g., future trajectories of a pedestrian) is essential for safety-critical perception systems (e.g., autonomous vehicles). In particular, a set of possible future behaviors generated by the system must be diverse to account for all possible outcomes in order to take necessary safety precautions. It is not sufficient to maintain a set of the most likely future outcomes because the set may only contain perturbations of a dominating single outcome (major mode). While generative models such as variational autoencoders (VAEs) have been shown to be a powerful tool for learning a distribution over future trajectories, randomly drawn samples from the learned implicit likelihood model may not be diverse -- the likelihood model is derived from the training data distribution and the samples will concentrate around the major mode of the data. In this work, we propose to learn a diversity sampling function (DSF) that generates a diverse yet likely set of future trajectories. The DSF maps forecasting context features to a set of latent codes which can be decoded by a generative model (e.g., VAE) into a set of diverse trajectory samples. Concretely, the process of identifying the diverse set of samples is posed as DSF parameter estimation. To learn the parameters of the DSF, the diversity of the trajectory samples is evaluated by a diversity loss based on a determinantal point process (DPP). Gradient descent is performed over the DSF parameters, which in turn moves the latent codes of the sample set to find an optimal set of diverse yet likely trajectories. Our method is a novel application of DPPs to optimize a set of items (forecasted trajectories) in continuous space. We demonstrate the diversity of the trajectories produced by our approach on both low-dimensional 2D trajectory data and high-dimensional human motion data.
The authors propose a method to diversify samples generated from a VAE. The method is based on determinantal point processes in the latent space and relies on specifying a kernel in the sample space and a quality metric in the latent space. Increasing diversity of samples from VAE-like models is an important and common problem and the authors present a reasonably generic method for solving it. The experiments are sensible and show clear improvement over a reasonable selection of baselines. One improvement I would suggest is adding a more explicit discussion of how the proposed method avoids generating overly diverse, very improbable trajectories. It is also somewhat suspicious that the trajectories shown in Figure 3 are terminated so early.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Diverse Trajectory Forecasting with Determinantal Point Processes
The ability to forecast a set of likely yet diverse possible future behaviors of an agent (e.g., future trajectories of a pedestrian) is essential for safety-critical perception systems (e.g., autonomous vehicles). In particular, a set of possible future behaviors generated by the system must be diverse to account for all possible outcomes in order to take necessary safety precautions. It is not sufficient to maintain a set of the most likely future outcomes because the set may only contain perturbations of a dominating single outcome (major mode). While generative models such as variational autoencoders (VAEs) have been shown to be a powerful tool for learning a distribution over future trajectories, randomly drawn samples from the learned implicit likelihood model may not be diverse -- the likelihood model is derived from the training data distribution and the samples will concentrate around the major mode of the data. In this work, we propose to learn a diversity sampling function (DSF) that generates a diverse yet likely set of future trajectories. The DSF maps forecasting context features to a set of latent codes which can be decoded by a generative model (e.g., VAE) into a set of diverse trajectory samples. Concretely, the process of identifying the diverse set of samples is posed as DSF parameter estimation. To learn the parameters of the DSF, the diversity of the trajectory samples is evaluated by a diversity loss based on a determinantal point process (DPP). Gradient descent is performed over the DSF parameters, which in turn moves the latent codes of the sample set to find an optimal set of diverse yet likely trajectories. Our method is a novel application of DPPs to optimize a set of items (forecasted trajectories) in continuous space. We demonstrate the diversity of the trajectories produced by our approach on both low-dimensional 2D trajectory data and high-dimensional human motion data.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The authors propose a method to diversify samples generated from a VAE. The method is based on determinantal point processes in the latent space and relies on specifying a kernel in the sample space and a quality metric in the latent space. Increasing diversity of samples from VAE-like models is an important and common problem and the authors present a reasonably generic method for solving it. The experiments are sensible and show clear improvement over a reasonable selection of baselines. One improvement I would suggest is adding a more explicit discussion of how the proposed method avoids generating overly diverse, very improbable trajectories. It is also somewhat suspicious that the trajectories shown in Figure 3 are terminated so early.
assistant
The authors propose a method to diversify samples generated from a VAE. The method is based on determinantal point processes in the latent space and relies on specifying a kernel in the sample space and a quality metric in the latent space. Increasing diversity of samples from VAE-like models is an important and common problem and the authors present a reasonably generic method for solving it. The experiments are sensible and show clear improvement over a reasonable selection of baselines. One improvement I would suggest is adding a more explicit discussion of how the proposed method avoids generating overly diverse, very improbable trajectories. It is also somewhat suspicious that the trajectories shown in Figure 3 are terminated so early.