title: Efficient Hybrid Zoom using Camera Fusion on Mobile Phones
summary: DSLR cameras can achieve multiple zoom levels via shifting lens distances or
  swapping lens types. However, these techniques are not possible on smartphone
  devices due to space constraints. Most smartphone manufacturers adopt a hybrid
  zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T)
  camera at a high zoom level. To simulate zoom levels between W and T, these
  systems crop and digitally upsample images from W, leading to significant
  detail loss. In this paper, we propose an efficient system for hybrid zoom
  super-resolution on mobile devices, which captures a synchronous pair of W and
  T shots and leverages machine learning models to align and transfer details
  from T to W. We further develop an adaptive blending method that accounts for
  depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment
  errors. To minimize the domain gap, we design a dual-phone camera rig to
  capture real-world inputs and ground-truths for supervised training. Our method
  generates a 12-megapixel image in 500ms on a mobile platform and compares
  favorably against state-of-the-art methods under extensive evaluation on
  real-world scenarios.
paper_authors._id: 65963006d61d4cbc1be7d058
paper_authors.name: Xiaotong Wu
paper_authors._id: 65963006d61d4cbc1be7d059
paper_authors.name: Wei-Sheng Lai
paper_authors._id: 65963006d61d4cbc1be7d05a
paper_authors.name: YiChang Shih
paper_authors._id: 65963006d61d4cbc1be7d05b
paper_authors.name: Charles Herrmann
paper_authors._id: 65963006d61d4cbc1be7d05c
paper_authors.name: Michael Krainin
paper_authors._id: 65963006d61d4cbc1be7d05d
paper_authors.name: Deqing Sun
paper_authors._id: 65963006d61d4cbc1be7d05e
paper_authors.name: Chia-Kai Liang
paper_title: Efficient Hybrid Zoom using Camera Fusion on Mobile Phones
paper_summary: DSLR cameras can achieve multiple zoom levels via shifting lens distances or
  swapping lens types. However, these techniques are not possible on smartphone
  devices due to space constraints. Most smartphone manufacturers adopt a hybrid
  zoom system: commonly a Wide (W) camera at a low zoom level and a Telephoto (T)
  camera at a high zoom level. To simulate zoom levels between W and T, these
  systems crop and digitally upsample images from W, leading to significant
  detail loss. In this paper, we propose an efficient system for hybrid zoom
  super-resolution on mobile devices, which captures a synchronous pair of W and
  T shots and leverages machine learning models to align and transfer details
  from T to W. We further develop an adaptive blending method that accounts for
  depth-of-field mismatches, scene occlusion, flow uncertainty, and alignment
  errors. To minimize the domain gap, we design a dual-phone camera rig to
  capture real-world inputs and ground-truths for supervised training. Our method
  generates a 12-megapixel image in 500ms on a mobile platform and compares
  favorably against state-of-the-art methods under extensive evaluation on
  real-world scenarios.
paper_ai_keywords: hybrid zoom system machine learning models adaptive blending method depth-of-field mismatches scene occlusion flow uncertainty alignment errors dual-phone camera rig supervised training state-of-the-art methods