title: Vision CoreML Object Detection Full Screen Landscape
body: <p>How can I get my <code>VNCoreMLRequest</code> to detect objects appearing anywhere within the <strong>fullscreen</strong> view?</p>
  <hr />
  <p>I am currently using the Apple sample project for object recognition in breakfast foods:<a href="https://developer.apple.com/documentation/vision/recognizing_objects_in_live_capture" rel="nofollow noreferrer"><strong>BreakfastFinder</strong></a>. The model and recognition works well, and generally gives the correct bounding box (visual) of the objects it is detecting / finding.</p>
  <p>The issue arises here with changing the <strong>orientation</strong> of this detection.</p>
  <hr />
  <p>In <strong>portrait mode</strong>, the default orientation for this project, the model identifies objects well in the <strong>full bounds</strong> of the view. Naturally, given the properties of the SDK objects, rotating the camera causes poor performance and visual identification.</p>
  <p>In <strong>landscape mode</strong>, the model behaves strangely. The <em>window / area</em> of which the model is detecting objects is <strong>not the full view</strong>. Instead, it is (what seems like) the same aspect ratio of the phone itself, but centered and in portrait mode. I have a screenshot below showing approximately where the model stops detecting objects when in landscape:</p>
  <p><a href="https://i.stack.imgur.com/wDqTC.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/wDqTC.jpg" alt="enter image description here" /></a>
  <a href="https://i.stack.imgur.com/yl2ou.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/yl2ou.jpg" alt="enter image description here" /></a>
  <a href="https://i.stack.imgur.com/5T95Y.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/5T95Y.jpg" alt="enter image description here" /></a></p>
  <hr />
  <p>The <em>blue box with red outline</em> is approximately where the detection stops. It behaves strangely, but consistently <strong>does not</strong> find any objects outside this approbate view / near the left or right edge. <em>However</em>, the top and bottom edges near the center detect without any issue.</p>
  <h3>regionOfInterest</h3>
  <p>I have adjusted this to be the maximum: <code>x: 0, y: 0, width: 1, height: 1</code>. This made no difference</p>
  <h3>imageCropAndScaleOption</h3>
  <p>This is the only setting that allows detection in the full screen, however, the performance became noticeably worse, and that's not really an allowable con.</p>
  <hr />
  <p>Is there a scale / size setting somewhere in this process that I have not set properly? Or perhaps a mode I am not using. Any help would be most appreciated. Below is my detection controller:</p>
  <h4>ViewController.swift</h4>
  <p>// All unchanged from the download in Apples folder</p>
  <pre><code>&quot; &quot;
      
  session.sessionPreset = .hd1920x1080 // Model image size is smaller.
  ... 
  previewLayer.connection?.videoOrientation = .landscapeRight
  
  &quot; &quot;
  </code></pre>
  <h4>VisionObjectRecognitionViewController</h4>
  <pre><code>@discardableResult
  func setupVision() -&gt; NSError? {
      // Setup Vision parts
      let error: NSError! = nil
      
      guard let modelURL = Bundle.main.url(forResource: &quot;ObjectDetector&quot;, withExtension: &quot;mlmodelc&quot;) else {
          return NSError(domain: &quot;VisionObjectRecognitionViewController&quot;, code: -1, userInfo: [NSLocalizedDescriptionKey: &quot;Model file is missing&quot;])
      }
      do {
          let visionModel = try VNCoreMLModel(for: MLModel(contentsOf: modelURL))
          let objectRecognition = VNCoreMLRequest(model: visionModel, completionHandler: { (request, error) in
              DispatchQueue.main.async(execute: {
                  // perform all the UI updates on the main queue
                  if let results = request.results {
                      self.drawVisionRequestResults(results)
                  }
              })
          })
          
          // These are the only properties that impact the detection area
          objectRecognition.regionOfInterest = CGRect(x: 0, y: 0, width: 1, height: 1)
          objectRecognition.imageCropAndScaleOption = VNImageCropAndScaleOption.scaleFit
  
          self.requests = [objectRecognition]
  
      } catch let error as NSError {
          print(&quot;Model loading went wrong: \(error)&quot;)
      }
      
      return error
  }
  </code></pre>
  <h3>EDIT:</h3>
  <p>When running the project in <em>portrait</em> mode only (locked by selecting <em>only</em> <strong>Portrait</strong> in Targets -&gt; General), then rotating the device to landscape, the detection occurs perfectly across the entire screen.</p>