Sekai: A Video Dataset towards World Exploration
Video generation techniques have made remarkable progress, promising to be
the foundation of interactive world exploration. However, existing video
generation datasets are not well-suited for world exploration training as they
suffer from some limitations: limited locations, short duration, static scenes,
and a lack of annotations about exploration and the world. In this paper, we
introduce Sekai (meaning ``world'' in Japanese), a high-quality first-person
view worldwide video dataset with rich annotations for world exploration. It
consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from
over 100 countries and regions across 750 cities. We develop an efficient and
effective toolbox to collect, pre-process and annotate videos with location,
scene, weather, crowd density, captions, and camera trajectories. Experiments
demonstrate the quality of the dataset. And, we use a subset to train an
interactive video world exploration model, named YUME (meaning ``dream'' in
Japanese). We believe Sekai will benefit the area of video generation and world
exploration, and motivate valuable applications.
6853946599bf39f9665c79e0
Zhen Li
6853946599bf39f9665c79e1
Chuanhao Li
claimed_verified
2025-06-23T08:17:13.350Z
674ea59a8f2e7614a6c72f26
/avatars/86fd4c6d7d33435de49b35659bf65265.svg
Chuanhao
user
ChuanhaoLi
6853946599bf39f9665c79e2
Xiaofeng Mao
6853946599bf39f9665c79e3
Shaoheng Lin
6853946599bf39f9665c79e4
Ming Li
6853946599bf39f9665c79e5
Shitian Zhao
6853946599bf39f9665c79e6
Zhaopan Xu
admin_assigned
2025-06-20T12:27:05.675Z
684ad9abc8d5ae11cae05781
/avatars/19b444eb086acdf6798c30fbb6277ddc.svg
Zhaopan Xu
user
zhaopan1
6853946599bf39f9665c79e7
Xinyue Li
6853946599bf39f9665c79e8
Yukang Feng
admin_assigned
2025-06-20T12:27:21.470Z
66d94f2a36aa5055694dfe04
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/grAN83brH0E4_S0__yLdv.jpeg
fengyukang
user
finyorko
6853946599bf39f9665c79e9
Jianwen Sun
6853946599bf39f9665c79ea
Zizhen Li
6853946599bf39f9665c79eb
Fanrui Zhang
6853946599bf39f9665c79ec
Jiaxin Ai
6853946599bf39f9665c79ed
Zhixiang Wang
claimed_verified
2025-06-23T08:17:15.626Z
6307a98795b2ab342fec0cf7
/avatars/85b261bcdda4717a6e40491f6c7b7a89.svg
Zhixiang Wang
user
wangzx1994
6853946599bf39f9665c79ee
Yuwei Wu
6853946599bf39f9665c79ef
Tong He
claimed_verified
2025-07-17T13:55:48.179Z
6747ede3a9c72aebe1322382
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/inILqQ05sESbYLdsEldJ_.png
Tong He
user
tonghe90
6853946599bf39f9665c79f0
Jiangmiao Pang
admin_assigned
2025-06-20T12:28:34.272Z
65783ee6ee33d547aecc3ffc
https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
Jiangmiao Pang
user
Jiangmiao
6853946599bf39f9665c79f1
Yu Qiao
6853946599bf39f9665c79f2
Yunde Jia
6853946599bf39f9665c79f3
Kaipeng Zhang
admin_assigned
2025-06-19T10:09:38.100Z
63527f4e7d071f23d085ad45
/avatars/99a51adef5673b3ac1a8c02eb47759c4.svg
KAIPENG ZHANG
user
kpzhang
Sekai: A Video Dataset towards World Exploration
Video generation techniques have made remarkable progress, promising to be
the foundation of interactive world exploration. However, existing video
generation datasets are not well-suited for world exploration training as they
suffer from some limitations: limited locations, short duration, static scenes,
and a lack of annotations about exploration and the world. In this paper, we
introduce Sekai (meaning ``world'' in Japanese), a high-quality first-person
view worldwide video dataset with rich annotations for world exploration. It
consists of over 5,000 hours of walking or drone view (FPV and UVA) videos from
over 100 countries and regions across 750 cities. We develop an efficient and
effective toolbox to collect, pre-process and annotate videos with location,
scene, weather, crowd density, captions, and camera trajectories. Experiments
demonstrate the quality of the dataset. And, we use a subset to train an
interactive video world exploration model, named YUME (meaning ``dream'' in
Japanese). We believe Sekai will benefit the area of video generation and world
exploration, and motivate valuable applications.
Sekai, a worldwide video dataset with comprehensive annotations, is introduced to support world exploration applications, enhancing video generation models.
first-person view worldwide video dataset rich annotations FPV UVA video collection pre-processing camera trajectories interactive video world exploration model