summaries: This paper aims at taking techniques from motion interpolation into the regime where one is able to generate longer range motion sequences, in the domain of physically plausible computer animation of characters. In the way that the authors have set up the problem, an initial database seeds the search for plausible transitions between two given poses. So, the technique being proposed must address how to keep physical realism (the long-standing question of "dynamics filtering" along the lines of Yamane, Katsu, and Yoshihiko Nakamura. "Dynamics filter-concept and implementation of online motion generator for human figures." IEEE transactions on robotics and automation 19.3 (2003): 421-432. Of course the problem here also needs to address "style" which needs different models).  The paper tackles the problem of generating long-range, diverse and natural looking motion sequence between initial and end states, and proposes to use a semi-parametric approach consisting of local and global models. Specifically, first the proposed approach extracts local motion feature from a reference subsequence and style feature from another, and then generates a new motion sequence. Then, global motion composition is done to interpolated generated local subsequences by bi-directional composition. In experimental validation, the approach outperforms two baselines (GAN and VAE).  The paper proposed ''composable semi-parametric modeling'' for generating long-range diverse and distinctive behaviors to achieve a specific goal location. The non-parametric part is a memory bank that is used to retrieve motion patterns from source materials. The parametric part contains several deep neural networks which are to compose the retrieved materials for high quality and smooth motion generation. The overall idea is novel in the sense that they aim to combine the strength of the non-parametric method (with rich pattern and diversities) and the parametric method (powerfull ability to generate coherent results). The proposed ideas are evaluated on two datasets and outperform compared approaches qualitatively.
abstractText: Learning diverse and natural behaviors is one of the longstanding goal for creating intelligent characters in the animated world. In this paper, we propose “COmposable Semi-parametric MOdelling” (COSMO), a method for generating long range diverse and distinctive behaviors to achieve a specific goal location. Our proposed method learns to model the motion of human by combining the complementary strengths of both non-parametric techniques and parametric ones. Given the starting and ending state, a memory bank is used to retrieve motion references that are provided as source material to a deep network. The synthesis is performed by the deep network that controls the style of the provided motion material and modifies it to become natural. On skeleton datasets with diverse motion, we show that the proposed method outperforms existing parametric and non-parametric baselines. We also demonstrate the generated sequences are useful as subgoals for actual physical execution in the animated world. Please refer to our project page 1 for more synthesised results.
references.authors: Jacopo Aleotti Stefano Caselli
references.title: Grasp recognition in virtual reality for robot pregrasp planning by demonstration
references.venue: In Proceedings 2006 IEEE International Conference on Robotics and Automation,
references.authors: Sanjeev Arora Yi Zhang
references.title: Do gans actually learn the distribution? an empirical study
references.authors: Mohammad Babaeizadeh Chelsea Finn Dumitru Erhan Roy H. Campbell Sergey Levine
references.title: Stochastic variational video prediction
references.venue: In ICLR,
references.authors: Shane T. Barratt Rishi Sharma
references.title: A note on the inception
references.venue: score. CoRR,
references.authors: Emad Barsoum John Kender Zicheng Liu
references.title: HP-GAN: probabilistic 3d human motion prediction via GAN
references.venue: In CVPR Workshops,
references.authors: Apratim Bhattacharyya Mario Fritz Bernt Schiele
references.title: Bayesian prediction of future street scenes using synthetic likelihoods
references.venue: In ICLR,
references.authors: Richard H. Byrd Peihuang Lu Jorge Nocedal Ciyou Zhu
references.title: A limited memory algorithm for bound constrained optimization
references.venue: SIAM J. Scientific Computing,
references.authors: Haoye Cai Chunyan Bai Yu-Wing Tai Chi-Keung Tang
references.title: Deep video generation, prediction and completion of human action sequences
references.venue: In ECCV,
references.authors: Emily Denton Rob Fergus
references.title: Stochastic video generation with a learned prior
references.venue: In ICML,
references.authors: Emily L. Denton Vighnesh Birodkar
references.title: Unsupervised learning of disentangled representations from video
references.venue: In NIPS,
references.authors: Chelsea Finn Ian J. Goodfellow Sergey Levine
references.title: Unsupervised learning for physical interaction through video prediction
references.venue: In NIPS,
references.authors: Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron C. Courville Yoshua Bengio
references.title: Generative adversarial nets
references.venue: In NIPS,
references.authors: Agrim Gupta Justin Johnson Li Fei-Fei Silvio Savarese Alexandre Alahi
references.title: Social GAN: socially acceptable trajectories with generative adversarial networks
references.venue: In CVPR,
references.authors: Adrian Haarbach Tolga Birdal Slobodan Ilic
references.title: Survey of higher order rigid body motion interpolation methods for keyframe animation and continuous-time trajectory estimation
references.venue: In 3DV,
references.authors: Sepp Hochreiter Jürgen Schmidhuber
references.title: Long short-term memory
references.venue: Neural Computation,
references.authors: Daniel Holden Jun Saito Taku Komura
references.title: A deep learning framework for character motion synthesis and editing
references.venue: ACM Trans. Graph.,
references.authors: Daniel Holden Taku Komura Jun Saito
references.title: Phase-functioned neural networks for character control
references.venue: ACM Trans. Graph.,
references.authors: Karim Iskakov
references.title: Semi-parametric image inpainting
references.venue: CoRR, abs/1807.02855,
references.authors: Xiaojie Jin Huaxin Xiao Xiaohui Shen Jimei Yang Zhe Lin Yunpeng Chen Zequn Jie Jiashi Feng Shuicheng Yan
references.title: Predicting scene parsing and motion dynamics in the future
references.venue: In NIPS,
references.authors: Diederik P. Kingma Jimmy Ba
references.title: Adam: A method for stochastic optimization
references.venue: In ICLR,
references.authors: Diederik P. Kingma Max Welling
references.title: Auto-encoding variational bayes
references.venue: In ICLR,
references.authors: Lucas Kovar Michael Gleicher
references.title: Flexible automatic motion blending with registration curves
references.venue: In SIGGRAPH/Eurographics Symposium,
references.authors: Tejas D. Kulkarni Karthik Narasimhan Ardavan Saeedi Josh Tenenbaum
references.title: Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation
references.venue: In NIPS,
references.authors: Martin Lawitzky Jose Ramon Medina Dongheui Lee Sandra Hirche
references.title: Feedback motion planning and learning from demonstration in physical robotic assistance: differences and synergies
references.venue: In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems,
references.authors: Youngwoon Lee Shao-Hua Sun Sriram Somasundaram Edward S. Hu Joseph J. Lim
references.title: Composing complex skills by learning transition policies
references.venue: In ICLR,
references.authors: Sergey Levine Jack M. Wang Alexis Haraux Zoran Popovic Vladlen Koltun
references.title: Continuous character control with low-dimensional embeddings
references.venue: ACM Trans. Graph.,
references.authors: Yunpeng Li Dominik Roblek Marco Tagliasacchi
references.title: From here to there: Video inbetweening using direct 3d convolutions
references.authors: Ziwei Liu Raymond A. Yeh Xiaoou Tang Yiming Liu Aseem Agarwala
references.title: Video frame synthesis using deep voxel flow
references.venue: In ICCV,
references.authors: Xuezhe Ma Eduard H. Hovy
references.title: End-to-end sequence labeling via bi-directional lstm-cnns-crf. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016
references.venue: August 7-12,
references.authors: Simone Meyer Oliver Wang Henning Zimmer Max Grosse Alexander Sorkine-Hornung
references.title: Phase-based frame interpolation for video
references.venue: In CVPR, pp
references.authors: Donald R. Myers
references.title: Robot Motion: Planning and Control edited by michael brady M.I.T
references.venue: press, cambridge ma, usa,
references.authors: Simon Niklaus Long Mai Feng Liu
references.title: Video frame interpolation via adaptive separable convolution
references.venue: In ICCV,
references.authors: Sang Il Park Hyun Joon Shin Sung Yong Shin
references.title: On-line locomotion generation based on motion blending
references.venue: In SIGGRAPH/Eurographics Symposium,
references.authors: Xue Bin Peng Pieter Abbeel Sergey Levine Michiel van de Panne
references.title: Deepmimic: Example-guided deep reinforcement learning of physics-based character skills
references.venue: ACM Transactions on Graphics (TOG),
references.authors: Dean A Pomerleau
references.title: Alvinn: An autonomous land vehicle in a neural network
references.venue: In Advances in neural information processing systems,
references.authors: Xiaojuan Qi Qifeng Chen Jiaya Jia Vladlen Koltun
references.title: Semi-parametric image synthesis
references.venue: In CVPR,
references.authors: Nitish Srivastava Elman Mansimov Ruslan Salakhutdinov
references.title: Unsupervised learning of video representations using lstms
references.venue: In ICML, pp
references.authors: Chen Sun Per Karlsson Jiajun Wu Joshua B. Tenenbaum Kevin Murphy
references.title: Stochastic prediction of multi-agent interactions from partial observations
references.venue: In ICLR,
references.authors: Charlie Irawan Tan Wen-Kai Tai
references.title: Characteristics preserving racer animation: a data-driven race path synthesis in formation space
references.venue: Journal of Visualization and Computer Animation,
references.authors: Raquel Urtasun David J. Fleet Andreas Geiger Jovan Popovic Trevor Darrell Neil D. Lawrence
references.title: Topologically-constrained latent variable models
references.venue: In ICML,
references.authors: Laurens van der Maaten Geoffrey Hinton
references.title: Visualizing data using t-sne
references.venue: In Journal of Machine Learning Research,
references.authors: Miao Wang Guo-Ye Yang Ruilong Li Runze Liang Song-Hai Zhang Peter M. Hall Shi-Min Hu
references.title: Example-guided style consistent image synthesis from semantic labeling
references.authors: Xiaolong Wang Rohit Girdhar Abhinav Gupta
references.title: Binge watching: Scaling affordance learning from sitcoms
references.venue: In CVPR,
references.authors: Yonatan Wexler Eli Shechtman Michal Irani
references.title: Space-time completion of video
references.venue: IEEE Trans. Pattern Anal. Mach. Intell.,
references.authors: Nevan Wichers Ruben Villegas Dumitru Erhan Honglak Lee
references.title: Hierarchical long-term video prediction without supervision
references.venue: In ICML,
references.authors: Guiyu Xia Huaijiang Sun Qingshan Liu Renlong Hang
references.title: Learning-based sphere nonlinear interpolation for motion synthesis
references.venue: IEEE Trans. Industrial Informatics,
references.authors: Jingwei Xu Bingbing Ni Xiaokang Yang
references.title: Video prediction via selective sampling
references.venue: In NIPS, pp
references.authors: Yanyu Xu Zhixin Piao Shenghua Gao
references.title: Encoding crowd interaction with deep neural network for pedestrian trajectory prediction
references.venue: In CVPR,
references.authors: Xinchen Yan Akash Rastogi Ruben Villegas Kalyan Sunkavalli Eli Shechtman Sunil Hadap Ersin Yumer Honglak Lee
references.title: MT-VAE: learning motion transformations to generate multimodal human dynamics
references.authors: Gu Ye Ron Alterovitz
references.title: guided motion planning
references.venue: In Robotics research,
references.authors: Jason Y. Zhang Panna Felsen Angjoo Kanazawa Jitendra Malik
references.title: Predicting 3d human dynamics from video
references.authors: Tianyang Zhao Yifei Xu Mathew Monfort Wongun Choi Chris Baker Yibiao Zhao Yizhou Wang Ying Nian Wu
references.title: Multi-agent tensor fusion for contextual trajectory prediction
sections.heading: 1 INTRODUCTION
sections.text: When faced with a specific goal in another location, humans can effortlessly find multiple distinctive trajectories and control their body to approach the goal with diverse and natural behaviours. However, we are still at the early stage for such sophisticated controlling of simulated characters. Recent reinforcement learning approaches (Peng et al., 2018) struggle to generate diverse motion. Other methods like imitation learning (Ye & Alterovitz, 2017; Aleotti & Caselli, 2006; Lawitzky et al., 2012) generalize badly to large scale demonstration data. The heart of this challenge lies in that real-world human behaviours are inherently multi-modal distributed. Direct behaviour learning is difficult without access to the explicit distribution of motion states.
  In this paper, we take a step toward generating long-range, diverse and physically plausible motion sequences given starting and ending states, rather than learning a policy for a physical simulator. Meanwhile, we expect the model could generate novel behaviour, i.e., unseen motion in training set. This has several valuable applications: (1) Synthesised vivid motion for animation production without excessive human labor. (2) Generated novel behaviours for player customization of action skills in video games. (3) Interpolated sequences as subgoals for policy training through reinforcement learning (Peng et al., 2018).
  To fulfil the above requirements, we need to address the following difficulties: (1) How to guarantee both diversity and naturalness, which is usually trade-off in the domain of generation (Srivastava et al., 2015)? (2) How to achieve long-range behaviour synthesis, which is stuck by error accumulation problem in many temporal modelling tasks (Denton & Fergus, 2018)? (3) How to generate unseen behaviours without loss of diversity or naturalness, which is hardly addressed in previous researches? As shown in Fig 1, the current two main branches of motion generation methods, i.e., parametric and non-parametric, are yet to deal with these difficulties properly. Parametric (Goodfellow et al., 2014) (e.g., GAN) methods could not maintain the reality/naturalness of generated sequences, and diversity (e.g., VAE) is hard to preserve. Non-parametric methods (Haarbach et al., 2018) involving motion clip copy/paste or blending has less superiority in smooth transition based on diverse reference sequences.
  1https://sites.google.com/view/cosmo-supp/home
  In this work, we propose COmposable Semi-parametric MOdelling (COSMO), which is a method that can leverage the large spectrum of motion skills from unlabeled data in a semi-parametric manner. Our method combines the complementary strengths of both non-parametric techniques and parametric ones. First, to avoid mode collapse in the sequences, we initially sample reference subsequences from a held-out reference set to encourage multi-modal behaviors in the generated sequence, which is mainly inspired from non-parametric methods (Haarbach et al., 2018). Second, we propose a self-supervised disentanglement model for extracting the content and style from each reference subsequence respectively. Here content refers to the characteristic of state at each time step, e.g., moving speed, direction, and general gesture of upper body, while style refers to the long-range motion pattern which keeps relative constant across whole subsequence. An embedded latent space is constructed where the learned style vectors could be combined to obtain novel style feature. A new subsequence is obtained by composing content and style freely from different reference subsequences. Considering that this operation only produces a single subsequence, we refer this step as local motion composition. Finally, for all new sequences, we compose them along the temporal direction in a sequential order. However, from a hodgepodge of reference segments, it is non-trivial to generate natural and meaningful behaviors. To guarantee naturalness at long-range scale, we then propose to use goal conditioned bi-directional interpolation for modeling the long-range nature needed in the task. This step is regarded as global motion composition, which covers the whole temporal scale (often more than 100 steps). On two human motion datasets, we show that the proposed method outperforms existing parametric and non-parametric baselines. We also demonstrate the generated sequences are useful as subgoals for actual physical execution in the animated world.
  Our paper makes three contributions. First, we proposed COmposable Semi-parametric MOtion generation (COSMO), a method that can generate the required sequences in a semi-parametric, composed way. Second, we conducted experiments on CMU Mocap dataset 2 and SFU Mocap dataset 3 and shows our method outperforms strong baselines such as VAEs and GANs significantly. Third, we proved empirically that the generated sequences can serve as subgoals to learn an actual policy.
sections.heading: 2 METHOD
sections.heading: 2.1 SEMI-PARAMETRIC MODELLING FOR MOTION GENERATION
sections.text: Given starting and ending states (denoted as s0 and sL respectively) as inputs, our goal is to synthesize intermediate states that have natural transition and diverse behaviors. As illustrated in Fig 2, NR reference subsequences (denoted as Ri, i = 1, .., NR) are sampled and fed into local motion composition
  2http://mocap.cs.cmu.edu/ 3http://mocap.cs.sfu.ca/
  sub-module φloc. Invalid subsequences are first filtered out with predefined requirements and please refer to appendix 6 for detailed description. φloc is used for altering the motion style of reference subsequence Ri based on another sequence R as input or directly sampling a style feature from learned latent space: R̂i = φloc(Ri,R), where R is optional input. With this in hand, we compose edited subsequences {R̂i}NRi=1 along temporal direction smoothly. Concretely, we generate a new clip R̃i−1:i = φglo(R̂i−1, R̂i) which bridges the gap between R̂i−1 and R̂i. The final composed sequence is obtained through concatenation along the temporal direction for all subsequences and generated clips.
  Why semi-parametric modelling? Regarding both motion category and transition dynamics (from current to next state), human behaviors are intrinsically multi-modal distribution (Holden et al., 2017). Parametric models (e.g., VAE (Kingma & Welling, 2014) or GAN (Goodfellow et al., 2014)) assume prior distribution to be uni-modal and predefined, which is not suitable to handle real world motion sequences (Kingma & Welling, 2014; Arora & Zhang, 2017). Non-parametric model does not require explicit prior distribution but not able to deal with long-range and highly diverse motion properly, mainly restricted by modelling capability. We also conduct targeted comparison experiments in Section 4. Compared to parametric methods, semi-parametric modelling naturally guarantees motion diversity by introducing reference sequence also maintaining a large degree of freedom for generation. Moreover, semi-parametric modelling does not require access to or a model of data distribution. It implies that our method is also directly applicable for unseen motion sequence.
  Constrained by starting and ending states with unchangeable locations, it is not feasible to randomly sample reference subsequences for generation. To construct a complete motion sequence, we execute a manually designed procedure for searching reference subsequences. For detailed description, please refer to appendix 6
sections.heading: 2.2 LOCAL MOTION COMPOSITION WITH CONTENT/STYLE DISENTANGLEMENT
sections.text: Local motion composition stands for generating a relative short-range of motion clip based on reference subsequences as inputs. Content and style features are learned jointly, whose free-form composition is used for synthesising new motion sequence. We first describe the motion style modelling part.
  Motion Style Modelling. Style is considered as motion information which stays the same throughout the whole subsequence. To this end, we extract style pattern by conducting feature fusion along the temporal direction. More specifically, for a sequence with T frames, we gradually fuse all frames into one constant feature with Cs channels (denoted as hs). We adopt 1D convolution along the temporal direction for each operation. Three convolution layers are stacked together with a kernel size of (T/2, T/4, T/8) respectively, where large kernel size facilitates capturing motion style throughout the whole subsequence.
  To get better generalization ability for unseen reference or starting/ending states, we propose to construct an embedding space for style pattern modelling. More specifically, a set of embedding vectors (denoted as Hs = {his}Mi=1) are learned together with above convolution layers (denoted as ψsty). The corresponding outputs (denoted as As ∈ RM ) of ψsty are a family of coefficients for a weighted sum of Hs. This is mainly inspired by the intuition that complex human activity could be considered as a combination of several simple actions. When trained with sufficiently large motion data, the model tends to learn basic style patterns for human motion. Compared to directly outputting a feature vector, this helps better generalize to unseen motion during testing. The final style feature (denoted as fs ∈ RCs ) is generated as follows:
  f is = ψsty(R i)Hs. (2.1)
  Motion Content Modelling. In this part, we focus on motion dynamics reflected by each single state. To this end, 1D convolution layer with kernel size of 3, i.e., covering consecutive 3 states, is adopted for content modelling. The output channel of this layer is designed less than original motion dimension for extracting the most relevant information. This is empirically set to 5 as a design choice. Meanwhile, velocity, position for root joint and step pattern for foot joint are fed as inputs. We denote all the above factors as fc with T steps and Cc channels. During local motion composition, style feature fs is combined with content feature fc through concatenating along the channel axis:
  f ic = ψcnt(R i), R̂i = ψrec(f i c , f i s), (2.2)
  where fs is tiled T times to match with fc and ψrec outputs reconstructed R̂i. ψrec follows stacked three layer 1D convolution operation with kernel size of 3. Finally, reconstruction loss is used to learn the style and content feature jointly as follows:
  LRrec = 1
  NR ΣNRi=1||R̂ i −Ri||22. (2.3)
sections.heading: 2.3 GLOBAL MOTION COMPOSITION VIA GOAL CONDITIONED BI-DIRECTIONAL MODELLING
sections.text: Global motion composition is connecting edited short-range subsequences into a completed and long-range one. Generating smooth and natural transition between R̂i and R̂i+1 is critical to obtain a long-range sequence. The major challenge comes from large states variation between R̂i and R̂i+1. Inspired by recent work of Bi-LSTM modelling (Ma & Hovy, 2016), bi-directional motion composition is utilized for this part. More specifically, motion states are first mapped to a higher dimensional space with ϕenc and ϕdec, which is defined as follows:
  f it = ϕenc(r i t), r̂ i t = ϕdec(f i t ). (2.4)
  Both ϕenc and ϕdec are 1D convolution layers. Then we predict the possible states at both directions with f it , where higher dimension (256) provides a sparser space enabling better composition results. For forward prediction, we take f iT−4:T as inputs to obtain forward states f i:i+1 1 . Meanwhile, f i+1 1:5 are fed as inputs treated as goal conditions. We conduct this procedure in a recurrent manner with P -step prediction in total. Predicted f i:i+11 is concatenated with last four states forming next step inputs. Backward direction takes exactly the reverse procedure, i.e., f i+1T−4:T are treated as inputs while f i 1:5 are as goal: f i:i+1p+1 = ϕfpre(f i p−4:p, f i+1 1:5 ), f i+1:i p+1 = ϕbpre(f i+1 p−4:p, f i T−4:T ), (2.5)
  where ϕfpre and ϕbpre share the same architecture.. Moreover, we utilize another model to produce coefficients for the weighted sum of outputs. In this way, we are able to construct more flexible latent space to facilitate composition:
  c = ϕcom(f i:i+1 p , f i+1:i P−p , p), f̂p = cf i:i+1 p + (1− c)f i+1:iP−p . (2.6)
  ϕcom is one layer 1D convolution with sigmoid function for producing c. More specifically we introduce p and linearly map it to p̂ ∈ (−1, 1) and c = p̂+ 0.1 ∗ σ(conv([f i:i+1p , f i+1:iP−p ])). During training, we randomly select a motion clip with a length of 10 + P . First as well as last 5 states are fed as inputs to get intermediate P states. Reconstruction loss is used for training:
  Lfrec = 1
  P ΣPt=1||r̂t − ϕdec(f̂ t)||22. (2.7)
  Compared with goal-conditioned single forward prediction model, the starting and ending states are treated equally in our model, which avoids drifting away from the ending state. Meanwhile, we provide corresponding empirical results in Section 4. After global motion composition, final composed sequence is directly obtained by concatenating all edited subsequences (local part) as well as transition clips (global part) along the temporal direction.
  We use tensorflow (Abadi et al., 2015) to implement all our models. All activation function used in our model is ReLU operation. During training, learning rate is set to 1e-4 and optimized with Adam optimizer (Kingma & Ba, 2015). β1 and β2 are set as 0.9 and 0.999 respectively. All models are trained with 30 epochs in total. Note that the local and global motion composition submodule do not need joint training. They are used jointly during testing.
sections.heading: 3 RELATED WORK
sections.text: Motion Interpolation. Given start and end states, this task aims to synthesize intermediate states which smoothly translate between them (Urtasun et al., 2008). For video interpolation (Liu et al., 2017; Li et al., 2019; Meyer et al., 2015; Niklaus et al., 2017) where start and end states are two consecutive frames, the final result is expected to increase frame rate of original video to a higher value. Previous researches often utilize phase dynamics (Meyer et al., 2015), flow based feature (Liu et al., 2017) and other motion information (Niklaus et al., 2017) to facilitate this task. Our work is different from this branch of work because there exists large motion gap between start and end states in our settings. Another branch of work is video completion (Cai et al., 2018; Li et al., 2019; Wexler et al., 2007). It receives two nonconsecutive frames as input and aims to fill the motion gap between start and end states. Cai et al. (2018) firstly attempts to solve this task and more specifically, propose to select out a rational path in the latent space with BFGS (Byrd et al., 1995) algorithm. Li et al. (2019) incorporates the 3D convolution layers and LSTM network into a unified model, which tries to automatically find the optimal results for intermediate frames. Despite much progress has been made in this filed, the high dimensional data (i.e., video frames) severely restricts video completion within simple and seen motion categories. However, we do not limit the start and end states belonging to the same sequence. Meanwhile, we expect the interpolated sequence as diverse as possible meanwhile with natural transition between synthesised states. This has not been deeply addressed in previous motion completion works (Xia et al., 2019; Lee et al., 2019). As a potential downstream application, our model could be used to construct motion planning (Myers, 1983) algorithm. Compared to goal-driven RL (Kulkarni et al., 2016; Lee et al., 2019), our model gets rid of requirements hard to achieve, i.e., known dynamics of agent, which is more general and applicable to more motion planning scenarios.
  Temporal Data Prediction. There have been many researches focusing on temporal data prediction. This task aims to infer the future possible states conditioned on history states as input. Video prediction (Srivastava et al., 2015) takes a major part in this field, which involves pixel-wise forecasting for every following frames. Previous methods (Denton & Fergus, 2018; Babaeizadeh et al., 2018; Denton & Birodkar, 2017; Finn et al., 2016) have made it to produce high quality prediction on bouncing digits (Srivastava et al., 2015), robot motion (Finn et al., 2016) and semantic map (Jin et al., 2017). However, recent work (Xu et al., 2018a) still encounters much difficulties in forecasting complex movements involving articulated subject (Wichers et al., 2018). Moreover, these works are prone to have motion blur (Finn et al., 2016) and error accumulation (Denton & Fergus, 2018) problems. Out of above concerns it is more appropriate for us to formulate our method as interpolation instead of prediction model. In terms of lower dimension data, human pose (Sun et al., 2019; Zhang et al., 2019) as well as path trajectory (Xu et al., 2018b) are also hot spots in this task. Several works adopt probabilistic Bayesian model (Bhattacharyya et al., 2019) to dig out latent factors which influence future dynamics. Another branch of researches (Gupta et al., 2018; Xu et al., 2018b) utilize deep recursive model (e.g., LSTM Hochreiter & Schmidhuber (1997)) to extract critical feature for prediction. Low dimension data enables all these works could easily scale to multi-target settings (Zhao et al., 2019). However, prediction model loses its power when encountering out of distribution data (e.g., unseen motion category in test set). As comparison, our interpolation model could naturally generalize to unseen motion facilitated by semi-parametric modelling, which gets rid of explicit representation for future dynamics.
  Generative Semi-parametric Modelling. For generative task such as image synthesis (Qi et al., 2018), translation (Wang et al., 2019) and inpainting (Iskakov, 2018), semi-parametric modelling achieves considerably more realistic visual quality and better style consistency between source and target images. Semi-parametric modelling is also utilized in reinforcement learning (Kulkarni et al., 2016) for navigation task (Eysenbach et al.). More specifically, it searches rational path on replay buffer, which enables agents to solve sparse reward tasks over one hundred steps. Wang et al. (2017) combines semi-parametric modelling and VAE for large-scaled motion affordance. In this work we use semi-parametric modelling for two main reasons: (1) it could substantially increase the length of interpolated sequence through acting as example guidance, (2) it enables our model to synthesise visually appealing results even with out of distribution inputs. Meanwhile, we would like to highlight that to our best knowledge this is the first attempt to use semi-parametric modelling on motion interpolation task.
  Motion Generation in Computer Graphics. In the context of computer graphics, there is a branch of researches (Kovar & Gleicher, 2003; Park et al., 2002; Levine et al., 2012; Tan & Tai, 2012; Holden et al., 2016) which also concentrate on motion generation, i.e., obtaining a continuous trajectory from a discrete set of poses. Haarbach et al. (2018) provides a comprehensive study on this topic. It analyzes the characteristics of higher order rigid body motion interpolation methods. Our works shares similar target with this branch of work. However, we would like to emphasise that these works (Tan & Tai, 2012; Kovar & Gleicher, 2003; Park et al., 2002) are in parallel with ours and have completely different research routine on this task. More specifically, graphics methods focus on finding a optimal and explicit mathematical solution regardless of input motion sequences. Different from them, our work is data-driven and encourages both reality and diversity of interpolated results.
  Imitation Learning. Imitation learning (Pomerleau, 1989; Ye & Alterovitz, 2017; Lawitzky et al., 2012) is commonly adopted as a standard method in the domain of robotics and many other areas. Behavioral Cloning (Osa et al., 2018) is one of the underlying approaches that utilize a demonstrations as supervisory signal. The most relevant work to ours is (Peng et al., 2018) which leverages reinforcement learning to imitate natural motions. However, our method does not imitate specific motion trajectories but to generate natural and diverse actions that are reasonable between given states. Our method can be further differentiated from past literature in two aspects: (1) our method is orthogonal to imitation learning because it interpolates states rather than predicts actions when states are given. More importantly, (2) the proposed method is able to produce between unseen states while imitation learning focuses on imitating and capturing the demonstrated distribution.
sections.heading: 4 EXPERIMENTS
sections.heading: 4.1 EVALUATION SETTINGS
sections.text: We use CMU Mocap 4 and SFU Mocap 5 datasets for evaluation. Both datasets contain diverse daily human motion sequences which are suitable for training. Considering that the original sequences possess different length, we get reference subsequence in a sliding window manner, where T = 120 and P = 40. For both datasets the dimension of state is 63, which is 21 joints with 3D coordinates. Here we would like to emphasise that we keep a held-out reference set (denoted as DR) from training data (denoted as DT ) for further testing, to demonstrate that our model could generate novel behaviour never seen during training. More specifically, DR is used for sampling novel motion and starting/ending states during testing. Note that the following all experiments are with SFU Mocap datasets. For the visual results on CMU Mocap datasets, please refer to our project page.
  For local motion composition, we compare our model with two strong parametric baselines: VAE (Kingma & Welling, 2014) and GAN (Goodfellow et al., 2014). For fair comparison, we train both models with all data, i.e., DT and DR. Meanwhile we follow Yan et al. (2018) and Barsoum et al. (2018) about the hyper-parameter setting of both models. But the input dimension is adjusted to match our data. For global motion composition, we compare our model with temporal prediction baselines: (1) Baseline1: ϕfpre without last 5 states as goal condition, (2) Baseline2: ϕfpre with last 5 states as goal condition, (3) Baseline3: both ϕfpre and ϕbpre but without last/first
  4http://mocap.cs.cmu.edu/ 5http://mocap.cs.cfu.ca/
  5 states as goal condition. All three baselines are trained with the same hyper-parameter setting as our global composition model.
sections.heading: 4.2 EVALUATION FOR LOCAL MOTION COMPOSITION
sections.text: Is generated motion diverse enough? We compare our model with parametric model (i.e., VAE (Kingma & Welling, 2014) and GAN (Goodfellow et al., 2014)) in evaluation of motion diversity. For the majority of generative models, the diversity of generated result is upper bounded by training data Barratt & Sharma (2018). Motivated by this, we compare motion diversity with VAE (Kingma & Welling, 2014) and GAN (Goodfellow et al., 2014) based models under different percentages of data used for training. As illustrated in Fig 3, 10%, 30%, 50% and 100% training data are used respectively. After training converged, we calculate the averaged standard diviation of all joints with a higher value indicating more diverse. We can see that both parametric models keep relative constant motion diversity which is comparable with training data used less than 30%, but largely inferior to the diversity of full training data. On the contrary, with an increasing percentage of data used for training, our model achieves higher motion diversity, which mainly benefits from the general semi-parametric modelling framework.
  Is generated motion visually natural? For evaluation of visual naturalness, we provide generated motion results and compare with VAE (Kingma & Welling, 2014) and GAN (Goodfellow et al., 2014). As shown Fig 5, from top to bottom generated motion sequences correspond to our model (COSMO), GAN (Goodfellow et al., 2014) and VAE (Kingma & Welling, 2014), respectively. Note that all three sequences are with a length of 440 time steps. Our model (COSMO) generate the sequence with three reference subsequences. While both GAN (Goodfellow et al., 2014) and VAE (Kingma & Welling, 2014) models directly produce the motion in a recurrent manner. The GAN based model fails to synthesise a normal walking sequence with large pose distortion. The VAE based model is able to generate a visually natural walking sequence facilitated by KL loss during training but struggles to produce diverse motion behaviour. Different from all these parametric models, Fig 5
  shows that our semi-parametric modelling based model achieves natural state transition throughout the whole sequence, meanwhile provides natural and diverse motion (i.e., walking-turning-walking) behaviours during generation. We encourage readers to refer to our project page for more video demonstrations.
  Do proposed model learn a meaningful style space? Recall that for representation learning of style feature, we construct an embedded space instead of directly generating style feature. In this way, we expect our model able to map a novel style pattern into learned embedded space. For validation we randomly select 10% training data then extract corresponding style feature (denoted as Strain) with ϕsty. Regarding the held-out set (denoted as Sref ) we conduct the same operation for all reference subsequences. We visualize the distribution of both Strn and Sref with t-SNE (van der Maaten & Hinton, 2008) in a two-dimensional plane. As shown in Fig 4, the purple dots indicate training data while green dots stand for reference data. We can see that the style feature of training data spread evenly across the plane. Meanwhile, the majority of style of feature of reference subsequences is covered by that of training data. Within the learned style space, embedded layer acts as a set of style bases where novel style can be approximated by the combination of these bases.
  Can COSMO compose two reference subsequences into a novel one? Part of the generation diversity of our model results from free composition of reference subsequences. As shown in Fig 6, we provide two edited sequence examples which possess the general motion style from one subsequence, but detailed motion pattern from another one. Taking the second sequence (bottom row in Fig 6) for example, the target style motion shows a spinning motion with both hands raised up (style), while the reference motion is a regular walking sequence. We can notice that the final edited reference motion (third column, bottom row) fully captures the style of upper body meanwhile maintains the walking pattern from reference sequence. Moreover, both top and bottom rows use the same reference motion, but with different style as inputs, our model can still produce highly diverse behaviours. For the space restriction of paper writing, we provide more examples whose new style features are sampled from constructed embedded space. We encourage readers to refer to our project page for more video demonstrations.
sections.heading: 4.3 EVALUATION FOR GLOBAL MOTION COMPOSITION
sections.text: Do COSMO learn better motion transition than basic prediction models? Different from prediction model, COSMO produces motion transition given starting or ending states from different sequences, i.e., there is no ground truth for evaluation. To this end, we provide training error for evaluation of motion transition. As shown in Fig 7(A), we can see that our model (COSMO) outperforms other methods by a large margin. Base1 achieves lowest accuracy caused by no ending states are provided during composition. Baseline2 and Baseline3 models perform better than base1 model, which indicates that both bi-directional and goal-conditional modelling scheme facilitates motion composition by a large margin. Our global composition sub-module combines the strength of both Baseline2 and Baseline3 models, which evenly utilize the information from starting and ending states. However, merely considering the training error is not sufficient. Next, we further evaluate the performance of our model given two different sequences.
  Can COSMO guarantee smooth transition between two different sequences? Fig 8 demonstrates motion transition results given starting and ending states from different reference subsequences, respectively. Note that for all sequences shown in Fig 8, the starting as well as ending states are from held-out reference set. We can observe that our model is able to generate smooth and natural transition when starting and ending states are similar. Moreover, when encountered large motion difference, e.g., from walking to greeting, turning back with a relatively large degree, our model still makes it to generate visually natural transition sequence. We encourage readers to refer to our project page for more video demonstrations.
  Is global motion composition merely linear interpolation? One possible trivial solution learned by our transition model is to simply linear interpolation between starting and ending states. However, common human motion generally involves non-linear trajectories. Linear interpolation is prone to produce unnatural motion which is easy to detect by human eyes. To valid that whether our model conducts linear interpolation between two sequences, we record the height variation of the right foot
  in a composed motion sequence. Meanwhile, we manually rotate the second sequence w.r.t. the final state of the first sequence to show that our model is robust to a large range of direction difference between two sequences. As shown in Fig 7(A), two black arrows indicate the starting and ending steps for composition. Here we present multiple curves which correspond to different rotation angle mentioned earlier. All recorded curves are highly non-linear but smooth between starting and ending points. Moreover, our model adaptively changes foot height with different rotation configurations, which indicates smooth and natural motion for motion composition.
  Visualization of final composed sequences. Combining local and global motion together, we are able to generate final sequences. Recall that our model is constrained by given starting and ending states based on motion interpolation. To this end, we present three composed sequences with a length of 480 time steps, i.e., three edited reference sequences (length of 120) and four generated clips for global motion composition. As shown in left part of Fig 9, starting from the same state, we are able to generate long-range and visually natural motion boosted by the local and global motion composition. Meanwhile, we are able to generate diverse behaviour (shown as complex hand and foot motion) facilitated semi-parametric modelling. We encourage readers to refer to our project page for more video demonstrations.
sections.heading: 4.4 APPLICATION
sections.text: In this section, we present several downstream applications related to our motion generation model. The first one is diverse motion generation under fixed rout constrain. The second one is demonstration motion guidance for imitation learning (Peng et al., 2018).
  Diverse motion generation. As shown in the right part of Fig 9, we manually design four routes for motion generation. We can see that our semi-parametric model is able to produce intermediate motion states clearly following the predefined route. Note that the longest motion sequence indicating character "R" is over 1500 time steps. We encourage readers to refer to our project page for corresponding video demonstrations.
  Expert demonstration guidance for imitation learning. Under a simulated environment with gravity constrain (Peng et al., 2018), unnatural motion violating physical law (e.g., severe joint twisting) is hard for a simulated agent to follow. To further show our model produces realistic motion, generated results are used for demonstration guidance of imitation learning. As shown in Fig 10, the left part is demonstration synthesised by COSMO (top) while the bottom one is learned policy with Peng et al. (2018). We can see that the learned motion succeeds to follow the synthesised one. The right part is return curve during training, which also shows that our generated motion is realistic enough for the guidance of imitation learning. We encourage readers to refer to our project page for corresponding video demonstrations.
sections.heading: 5 CONCLUSION
sections.text: In this work, we propose to generate long-range motion in a semi-parametric way. We first sample reference motion subsequence from the held-out set and change the motion style with a local motion composition scheme. We then compose all reference subsequences with the proposed global motion composition scheme. Given the same starting and ending states, the proposed model is able to
  generate long-range, diverse and natural motion sequences over 1000 time steps without loss of visual quality.
sections.heading: 6 APPENDIX
sections.text: Requirements for reference sequence Ri. Our model does not restrict the length of Ri, neither any specific/predefined starting or ending states of it. However, there are still restrictions for Ri:
  • Physically reachable. In this work we do not have access to the dynamics of other objects, which should be excluded from candidate references if involved. For example, our model does not intend for synthesising transition sequence from running to lifting. The latter behaviour needs dynamic information (bell) to model. Therefore, in our experiments reference subsequence is first restricted to single character motion without external interaction. • Moving on flat ground. Besides physically reachable, we also restrict the character moving
  on flat ground. Because in our task there is no predefined ground information available for modelling. To get rid of unexpected composition results, we filter out these sequences recorded from uneven terrain.
  Searching procedure for reference subsequences
  • Step 0: Calculate overall root locations for all reference subsequences. We denote corresponding minimum/maximum value as tmin/tmax respectively. • Step 1: Given s0 and sL, extract corresponding root locations (l0 and lL) respectively. Here
  root is defined by the ground projection of hip joint. Then sample NR + 1 mid locations, which forms NR + 2 pairs, i.e, [l0, l1], ..., [lNR , lL]. Distance defined by each pair dm is bounded, i.e., tmin < dm < tmax. • Step 2: For each location pair, we search reference subsequence for similar translation
  defined by dm with maximum tolerance σt. We conduct local motion composition for searched reference subsequence (in Section 2.2). • Step 3: We sequentially conduct goal conditioned bi-directional motion interpolation for
  two consecutive reference subsequences (in Section 2.3). As a preprocessing procedure, the overall location and direction of latter sequence is adjusted w.r.t former one for interpolation. The final sequence is obtained through concatenating all synthesised clips along temporal axis.