title: Atlas: Multi-Scale Attention Improves Long Context Image Modeling
summary: Efficiently modeling massive images is a long-standing challenge in machine
  learning. To this end, we introduce Multi-Scale Attention (MSA). MSA relies on
  two key ideas, (i) multi-scale representations (ii) bi-directional cross-scale
  communication. MSA creates O(log N) scales to represent the image across
  progressively coarser features and leverages cross-attention to propagate
  information across scales. We then introduce Atlas, a novel neural network
  architecture based on MSA. We demonstrate that Atlas significantly improves the
  compute-performance tradeoff of long-context image modeling in a
  high-resolution variant of ImageNet 100. At 1024px resolution, Atlas-B achieves
  91.04% accuracy, comparable to ConvNext-B (91.92%) while being 4.3x faster.
  Atlas is 2.95x faster and 7.38% better than FasterViT, 2.25x faster and 4.96%
  better than LongViT. In comparisons against MambaVision-S, we find Atlas-S
  achieves 5%, 16% and 32% higher accuracy at 1024px, 2048px and 4096px
  respectively, while obtaining similar runtimes. Code for reproducing our
  experiments and pretrained models is available at
  https://github.com/yalalab/atlas.
paper_authors._id: 67daf5dff34884a28b17bb53
paper_authors.name: Kumar Krishna Agrawal
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-03-19T16:50:44.848Z
paper_authors._id: 6254623af74165c7d2df0c18
paper_authors.avatarUrl: /avatars/4ddd1aa858723dd9348fc51966f700c3.svg
paper_authors.fullname: Kumar Krishna Agrawal
paper_authors.type: user
paper_authors.user: kumarkrishna
paper_authors._id: 67daf5dff34884a28b17bb54
paper_authors.name: Long Lian
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-20T10:45:56.150Z
paper_authors._id: 63797c273f575acc2f6893c0
paper_authors.avatarUrl: /avatars/32d7a6a8881c8c4d80a097b732ed24b6.svg
paper_authors.fullname: Long(Tony) Lian
paper_authors.type: user
paper_authors.user: longlian
paper_authors._id: 67daf5dff34884a28b17bb55
paper_authors.name: Longchao Liu
paper_authors._id: 67daf5dff34884a28b17bb56
paper_authors.name: Natalia Harguindeguy
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-20T04:19:22.595Z
paper_authors._id: 6333a9195a032dcd095dda13
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1664329996201-noauth.jpeg
paper_authors.fullname: Adam Yala
paper_authors.type: user
paper_authors.user: yala
paper_authors._id: 67daf5dff34884a28b17bb57
paper_authors.name: Boyi Li
paper_authors._id: 67daf5dff34884a28b17bb58
paper_authors.name: Alexander Bick
paper_authors._id: 67daf5dff34884a28b17bb59
paper_authors.name: Maggie Chung
paper_authors._id: 67daf5dff34884a28b17bb5a
paper_authors.name: Trevor Darrell
paper_authors._id: 67daf5dff34884a28b17bb5b
paper_authors.name: Adam Yala
paper_title: Atlas: Multi-Scale Attention Improves Long Context Image Modeling
paper_summary: Efficiently modeling massive images is a long-standing challenge in machine
  learning. To this end, we introduce Multi-Scale Attention (MSA). MSA relies on
  two key ideas, (i) multi-scale representations (ii) bi-directional cross-scale
  communication. MSA creates O(log N) scales to represent the image across
  progressively coarser features and leverages cross-attention to propagate
  information across scales. We then introduce Atlas, a novel neural network
  architecture based on MSA. We demonstrate that Atlas significantly improves the
  compute-performance tradeoff of long-context image modeling in a
  high-resolution variant of ImageNet 100. At 1024px resolution, Atlas-B achieves
  91.04% accuracy, comparable to ConvNext-B (91.92%) while being 4.3x faster.
  Atlas is 2.95x faster and 7.38% better than FasterViT, 2.25x faster and 4.96%
  better than LongViT. In comparisons against MambaVision-S, we find Atlas-S
  achieves 5%, 16% and 32% higher accuracy at 1024px, 2048px and 4096px
  respectively, while obtaining similar runtimes. Code for reproducing our
  experiments and pretrained models is available at
  https://github.com/yalalab/atlas.
paper_ai_summary: Atlas, a multi-scale attention-based neural network, improves computational efficiency without sacrificing accuracy in high-resolution image modeling.
paper_ai_keywords: Multi-Scale Attention MSA multi-scale representations bi-directional cross-scale communication cross-attention Atlas neural network ImageNet 100 ConvNext-B FasterViT LongViT MambaVision-S