circleclockwise”“Fold wings”“Surprise”Figure1.Examplesofgeneratedvideoframesandlatentﬂowse-quencesusingourproposedLFDM.Theﬁrstcolumnshowsthegivenimagesx0andconditionsy.Thelatentﬂowmapsareback-wardopticalﬂowtox0inthelatentspace.Weusethecolorcod-ingschemein[4]tovisualizeﬂow,wherethecolorindicatesthedirectionandmagnitudeoftheﬂow.ation,entertainmentanddataaugmentationformachinelearning[30].Givenasingleimagex0andaconditiony,conditionalimage-to-video(cI2V)generationaimstosynthesizearealisticvideowithframes0tok,ˆxK0={x0,ˆx1,...,ˆxK},startingfromthegivenframex0andsat-isfyingtheconditiony.Similartoconditionalimagesyn-thesisworks[43,82],mostexistingcI2Vgenerationmeth-ods[16,19,21,30,36,77]directlysynthesizeeachframeinthewholevideobasedonthegivenimagex0andconditiony.However,theyoftenstrugglewithsimultaneouslypre-servingspatialdetailsandkeepingtemporalcoherenceinthegeneratedframes.Inthispaper,weproposenovellatent1arXiv:2303.13744v1  [cs.CV]  24 Mar 2023…DDPM Reverse Process%𝐟<=’𝐦<=𝑥:Encoder
Conditional Image-to-Video Generation with Latent Flow Diffusion Models