title: Multi Threading OpencCV's VideoCapture
body: <p>I am wondering if I can benefit from multi-threading the VideoCapture's stream, I am using OpenGL and OpenCV, and my bottleneck is definetely the getting and setting to texture of my video camera's stream.</p>
  
  <pre><code>...This goes in the Render Loop.
  
  // We set our video.
  try
  {
  
      cap &gt;&gt; frame;
  
      if( frame.empty() )
      {
  
          break;
  
      }
  
      else
      {
  
          image = cvMat2TexInput( frame );
  
      }
  
      }
  
      catch( Exception&amp; e )
      {
  
          std::cout &lt;&lt; e.msg &lt;&lt; std::endl;
  
      }
  
      if( frame.empty() )
      {
  
          std::cout &lt;&lt; "No luck with the VideoCapture..." &lt;&lt; std::endl;
  
      }
  
      else
      {
  
          image = cvMat2TexInput( frame );
  
      }
  
  
  
      if( image )
      {
  
          glTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, videoWidth, videoHeight, 0, GL_RGB, GL_UNSIGNED_BYTE, image );
  
      }
  
      else
      {
  
          std::cout &lt;&lt; "Failed to load video texture." &lt;&lt; std::endl;
  
      }
  
      ...
  </code></pre>
  
  <p>This is the function that returns an unsigned char* for our OpenGL's texture input</p>
  
  <pre><code>// Utility function to link OpenCV.
  unsigned char* cvMat2TexInput( Mat&amp; img )
  {
  
      cvtColor( img, img, COLOR_BGR2RGB );
      transpose( img, img );
      flip( img, img, 1 );
      flip( img, img, 0 );
      return img.data;
  
  }
  </code></pre>
  
  <p>If it is more performant, how would I go about it? I have tried:</p>
  
  <pre><code>// Before main:
  void multi( VideoCapture cap, Mat&amp; frame )
  {
  
      try
      {
  
          cap &gt;&gt; frame;
  
      }
  
      catch( Exception&amp; e )
      {
  
          std::cout &lt;&lt; e.msg &lt;&lt; std::endl;
  
      }
  
  }
  
  // In main's render loop:
  std::thread t1( multi, cap, std::ref( frame ) );
  t1.join();
  
  if( frame.empty() )
  {
  
      std::cout &lt;&lt; "No luck with the VideoCapture..." &lt;&lt; std::endl;
  
  }
  
  else
  {
  
      image = cvMat2TexInput( frame );
  
  }
  
  
  
  if( image )
  {
  
      glTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, videoWidth, videoHeight, 0, GL_RGB, GL_UNSIGNED_BYTE, image );
  
  }
  
  else
  {
  
      std::cout &lt;&lt; "Failed to load video texture." &lt;&lt; std::endl;
  
  }
  </code></pre>
  
  <p>But I have a 10 FPS decrease. I have also tried to migrate the whole setting of the video from:</p>
  
  <pre><code>cap &gt;&gt; frame
  </code></pre>
  
  <p>to:</p>
  
  <pre><code>if( image )
  {
  
      glTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, videoWidth, videoHeight, 0, GL_RGB, GL_UNSIGNED_BYTE, image );
  
  }
  
  else
  {
  
      std::cout &lt;&lt; "Failed to load video texture." &lt;&lt; std::endl;
  
  }
  </code></pre>
  
  <p>Into my multi void function but, I get no output at all if not a huge memory leak. I've also tried having two different loops, one inside the threaded function and the main's render loop, but I get no output at all.</p>
  
  <p>So my questions are actually three:</p>
  
  <p>Can I benefit from multi-threading for this process? If yes, how would I implement it? And another one that is related but that can serve me for other processes, how would I call an unsigned char* by reference? The last one was something I wasn't able to do in my multi-threaded function, I tried something like:</p>
  
  <pre><code>void multi( ...., unsigned char*&amp; image ){ ... }
  </code></pre>
  
  <p>and calling it like:</p>
  
  <pre><code>std::thread t1( multi, ...., std::ref( image ) )
  </code></pre>
  
  <p>or </p>
  
  <pre><code>std::thread t1( multi, ...., image )
  </code></pre>
  
  <p>but only got memory leaks!</p>
  
  <p>Since C++11 I guess the OS won't matter, but I am using Windows 10 and Visual Studio 2017.</p>
  
  <p>Thanks!</p>