title: VideoRoPE: What Makes for Good Video Rotary Position Embedding?
summary: While Rotary Position Embedding (RoPE) and its variants are widely adopted
  for their long-context capabilities, the extension of the 1D RoPE to video,
  with its complex spatio-temporal structure, remains an open challenge. This
  work first introduces a comprehensive analysis that identifies four key
  characteristics essential for the effective adaptation of RoPE to video, which
  have not been fully considered in prior work. As part of our analysis, we
  introduce a challenging V-NIAH-D (Visual Needle-In-A-Haystack with Distractors)
  task, which adds periodic distractors into V-NIAH. The V-NIAH-D task
  demonstrates that previous RoPE variants, lacking appropriate temporal
  dimension allocation, are easily misled by distractors. Based on our analysis,
  we introduce VideoRoPE, with a 3D structure designed to
  preserve spatio-temporal relationships. VideoRoPE features
  low-frequency temporal allocation to mitigate periodic oscillations, a
  diagonal layout to maintain spatial symmetry, and adjustable
  temporal spacing to decouple temporal and spatial indexing. VideoRoPE
  consistently surpasses previous RoPE variants, across diverse downstream tasks
  such as long video retrieval, video understanding, and video hallucination. Our
  code will be available at
  https://github.com/Wiselnn570/VideoRoPE{https://github.com/Wiselnn570/VideoRoPE}.
paper_authors._id: 67a97a47174028234b74f687
paper_authors.name: Xilin Wei
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-10T13:12:02.432Z
paper_authors._id: 62eb70462f0f5e54df42f778
paper_authors.avatarUrl: /avatars/456049dba67638d3cdb330cdf383f272.svg
paper_authors.fullname: Xilin Wei
paper_authors.type: user
paper_authors.user: Wiselnn
paper_authors._id: 67a97a47174028234b74f688
paper_authors.name: Xiaoran Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-10T09:49:59.999Z
paper_authors._id: 64f033ef82c6eea604c4da8b
paper_authors.avatarUrl: /avatars/51b93fea7fd68b4274ee03701245dcca.svg
paper_authors.fullname: Liu Xiaoran
paper_authors.type: user
paper_authors.user: LiuXR
paper_authors._id: 67a97a47174028234b74f689
paper_authors.name: Yuhang Zang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-10T09:50:02.011Z
paper_authors._id: 63859cf3b2906edaf83af9f0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/mCgynEtoXdILvzyoPexii.png
paper_authors.fullname: Yuhang Zang
paper_authors.type: user
paper_authors.user: yuhangzang
paper_authors._id: 67a97a47174028234b74f68a
paper_authors.name: Xiaoyi Dong
paper_authors._id: 67a97a47174028234b74f68b
paper_authors.name: Pan Zhang
paper_authors._id: 67a97a47174028234b74f68c
paper_authors.name: Yuhang Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:01:29.622Z
paper_authors._id: 65000bef18830fabea469fdd
paper_authors.avatarUrl: /avatars/b320c77dfad039d9f9c54127f610d44f.svg
paper_authors.fullname: Cao Yuhang
paper_authors.type: user
paper_authors.user: yhcao
paper_authors._id: 67a97a47174028234b74f68d
paper_authors.name: Jian Tong
paper_authors._id: 67a97a47174028234b74f68e
paper_authors.name: Haodong Duan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:01:41.685Z
paper_authors._id: 63ee1379190ddd6214efd73a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
paper_authors.fullname: HAODONG DUAN
paper_authors.type: user
paper_authors.user: KennyUTC
paper_authors._id: 67a97a47174028234b74f68f
paper_authors.name: Qipeng Guo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:01:48.883Z
paper_authors._id: 6491cd52b1e5d3444528edb1
paper_authors.avatarUrl: /avatars/a85635d886c7f157b6723dec5c01c030.svg
paper_authors.fullname: Qipeng Guo
paper_authors.type: user
paper_authors.user: QipengGuo
paper_authors._id: 67a97a47174028234b74f690
paper_authors.name: Jiaqi Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-21T09:09:11.292Z
paper_authors._id: 64b4eec4faa3181a5eab9c46
paper_authors.avatarUrl: /avatars/bcc9bf5cbf67546ad2b4c9ec8b96ac96.svg
paper_authors.fullname: Jiaqi Wang
paper_authors.type: user
paper_authors.user: myownskyW7
paper_authors._id: 67a97a47174028234b74f691
paper_authors.name: Xipeng Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:02:04.834Z
paper_authors._id: 61457b8deff2c9fdb4de4988
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1632381702899-61457b8deff2c9fdb4de4988.jpeg
paper_authors.fullname: Xipeng Qiu
paper_authors.type: user
paper_authors.user: xpqiu
paper_authors._id: 67a97a47174028234b74f692
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-10T16:02:10.781Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_title: VideoRoPE: What Makes for Good Video Rotary Position Embedding?
paper_summary: While Rotary Position Embedding (RoPE) and its variants are widely adopted
  for their long-context capabilities, the extension of the 1D RoPE to video,
  with its complex spatio-temporal structure, remains an open challenge. This
  work first introduces a comprehensive analysis that identifies four key
  characteristics essential for the effective adaptation of RoPE to video, which
  have not been fully considered in prior work. As part of our analysis, we
  introduce a challenging V-NIAH-D (Visual Needle-In-A-Haystack with Distractors)
  task, which adds periodic distractors into V-NIAH. The V-NIAH-D task
  demonstrates that previous RoPE variants, lacking appropriate temporal
  dimension allocation, are easily misled by distractors. Based on our analysis,
  we introduce VideoRoPE, with a 3D structure designed to
  preserve spatio-temporal relationships. VideoRoPE features
  low-frequency temporal allocation to mitigate periodic oscillations, a
  diagonal layout to maintain spatial symmetry, and adjustable
  temporal spacing to decouple temporal and spatial indexing. VideoRoPE
  consistently surpasses previous RoPE variants, across diverse downstream tasks
  such as long video retrieval, video understanding, and video hallucination. Our
  code will be available at
  https://github.com/Wiselnn570/VideoRoPE{https://github.com/Wiselnn570/VideoRoPE}.
paper_ai_summary: VideoRoPE, a 3D position embedding that preserves spatio-temporal relationships, improves performance on video tasks by addressing limitations in previous RoPE variants.
paper_ai_keywords: Rotary Position Embedding RoPE long-context capabilities V-NIAH-D diagonal layout low-frequency temporal allocation adjustable temporal spacing long video retrieval video understanding video hallucination