title: iFormer: Integrating ConvNet and Transformer for Mobile Application
summary: We present a new family of mobile hybrid vision networks, called iFormer,
  with a focus on optimizing latency and accuracy on mobile applications. iFormer
  effectively integrates the fast local representation capacity of convolution
  with the efficient global modeling ability of self-attention. The local
  interactions are derived from transforming a standard convolutional network,
  i.e., ConvNeXt, to design a more lightweight mobile network. Our newly
  introduced mobile modulation attention removes memory-intensive operations in
  MHA and employs an efficient modulation mechanism to boost dynamic global
  representational capacity. We conduct comprehensive experiments demonstrating
  that iFormer outperforms existing lightweight networks across various tasks.
  Notably, iFormer achieves an impressive Top-1 accuracy of 80.4\% on ImageNet-1k
  with a latency of only 1.10 ms on an iPhone 13, surpassing the recently
  proposed MobileNetV4 under similar latency constraints. Additionally, our
  method shows significant improvements in downstream tasks, including COCO
  object detection, instance segmentation, and ADE20k semantic segmentation,
  while still maintaining low latency on mobile devices for high-resolution
  inputs in these scenarios.
paper_authors._id: 6798706dabdc35456a92212d
paper_authors.name: Chuanyang Zheng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:16:05.976Z
paper_authors._id: 65019cc870367843160fbb33
paper_authors.avatarUrl: /avatars/f5455482fe9efbdeeea1bd3a3c119f02.svg
paper_authors.fullname: ZhengChuanyang
paper_authors.type: user
paper_authors.user: BillionZheng
paper_title: iFormer: Integrating ConvNet and Transformer for Mobile Application
paper_summary: We present a new family of mobile hybrid vision networks, called iFormer,
  with a focus on optimizing latency and accuracy on mobile applications. iFormer
  effectively integrates the fast local representation capacity of convolution
  with the efficient global modeling ability of self-attention. The local
  interactions are derived from transforming a standard convolutional network,
  i.e., ConvNeXt, to design a more lightweight mobile network. Our newly
  introduced mobile modulation attention removes memory-intensive operations in
  MHA and employs an efficient modulation mechanism to boost dynamic global
  representational capacity. We conduct comprehensive experiments demonstrating
  that iFormer outperforms existing lightweight networks across various tasks.
  Notably, iFormer achieves an impressive Top-1 accuracy of 80.4\% on ImageNet-1k
  with a latency of only 1.10 ms on an iPhone 13, surpassing the recently
  proposed MobileNetV4 under similar latency constraints. Additionally, our
  method shows significant improvements in downstream tasks, including COCO
  object detection, instance segmentation, and ADE20k semantic segmentation,
  while still maintaining low latency on mobile devices for high-resolution
  inputs in these scenarios.
paper_ai_summary: iFormer combines convolution and self-attention to create a lightweight, high-performance mobile vision network with low latency across various tasks.
paper_ai_keywords: mobile hybrid vision networks iFormer convolution self-attention local interactions ConvNeXt mobile modulation attention memory-intensive operations modulation mechanism ImageNet-1k Top-1 accuracy latency iPhone 13 MobileNetV4 COCO object detection instance segmentation ADE20k semantic segmentation high-resolution inputs