title: SurfaceTexture's onFrameAvailable() method always called too late
body: <p>I'm trying to get the following MediaExtractor example to work:</p>
  
  <p><a href="http://bigflake.com/mediacodec/ExtractMpegFramesTest.java.txt">http://bigflake.com/mediacodec/ - ExtractMpegFramesTest.java (requires 4.1, API 16)</a></p>
  
  <p>The problem I have is that outputSurface.awaitNewImage(); seems to always throw RuntimeException("frame wait timed out"), which is thrown whenever the <code>mFrameSyncObject.wait(TIMEOUT_MS)</code> call times out. No matter what I set <code>TIMEOUT_MS</code> to be, <code>onFrameAvailable()</code> always gets called <em>right after</em> the timeout occurs. I tried with 50ms and with 30000ms and it's the same.</p>
  
  <p>It seems like the <code>onFrameAvailable()</code> call can't be done while the thread is busy, and once the timeout happens which ends the thread code execution, it can parse the <code>onFrameAvailable()</code> call.</p>
  
  <p>Has anyone managed to get this example to work, or knows how MediaExtractor is supposed to work with GL textures?</p>
  
  <p><strong>Edit:</strong> tried this on devices with API 4.4 and 4.1.1 and the same happens on both.</p>
  
  <p><strong>Edit 2:</strong></p>
  
  <p>Got it working on 4.4 thanks to fadden. The issue was that the <code>ExtractMpegFramesWrapper.runTest()</code> method called <code>th.join();</code> which blocked the main thread and prevented the <code>onFrameAvailable()</code> call from being processed. Once I commented <code>th.join();</code> it works on 4.4. I guess maybe the <code>ExtractMpegFramesWrapper.runTest()</code> itself was supposed to run on yet another thread so the main thread didn't get blocked.</p>
  
  <p>There was also a small issue on 4.1.2 when calling <code>codec.configure()</code>, it gave the error:</p>
  
  <pre><code>A/ACodec(2566): frameworks/av/media/libstagefright/ACodec.cpp:1041 CHECK(def.nBufferSize &gt;= size) failed.
  A/libc(2566): Fatal signal 11 (SIGSEGV) at 0xdeadbaad (code=1), thread 2625 (CodecLooper)
  </code></pre>
  
  <p>Which I solved by adding the following before the call:</p>
  
  <pre><code>format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 0);
  </code></pre>
  
  <p>However the problem I have now on both 4.1.1 (Galaxy S2 GT-I9100) and 4.1.2 (Samsung Galaxy Tab GT-P3110) is that they both always set info.size to 0 for all frames. Here is the log output:</p>
  
  <pre><code>loop
  input buffer not available
  no output from decoder available
  loop
  input buffer not available
  no output from decoder available
  loop
  input buffer not available
  no output from decoder available
  loop
  input buffer not available
  no output from decoder available
  loop
  submitted frame 0 to dec, size=20562
  no output from decoder available
  loop
  submitted frame 1 to dec, size=7193
  no output from decoder available
  loop
  [... skipped 18 lines ...]
  submitted frame 8 to dec, size=6531
  no output from decoder available
  loop
  submitted frame 9 to dec, size=5639
  decoder output format changed: {height=240, what=1869968451, color-format=19, slice-height=240, crop-left=0, width=320, crop-bottom=239, crop-top=0, mime=video/raw, stride=320, crop-right=319}
  loop
  submitted frame 10 to dec, size=6272
  surface decoder given buffer 0 (size=0)
  loop
  [... skipped 1211 lines ...]
  submitted frame 409 to dec, size=456
  surface decoder given buffer 1 (size=0)
  loop
  sent input EOS
  surface decoder given buffer 0 (size=0)
  loop
  surface decoder given buffer 1 (size=0)
  loop
  surface decoder given buffer 0 (size=0)
  loop
  surface decoder given buffer 1 (size=0)
  loop
  [... skipped 27 lines all with size=0 ...]
  surface decoder given buffer 1 (size=0)
  loop
  surface decoder given buffer 0 (size=0)
  output EOS
  Saving 0 frames took ? us per frame // edited to avoid division-by-zero error
  </code></pre>
  
  <p>So no images get saved. However the same code and video works on 4.3. The video I am using is an .mp4 file with "H264 - MPEG-4 AVC (avc1)" video codec and "MPEG AAAC Audio (mp4a)" audio codec.</p>
  
  <p>I also tried other video formats, but they seem to die even sooner on 4.1.x, while both work on 4.3.</p>
  
  <p><strong>Edit 3:</strong></p>
  
  <p>I did as you suggested, and it seems to save the frame images correctly. Thank you.</p>
  
  <p>Regarding KEY_MAX_INPUT_SIZE, I tried not setting, or setting it to 0, 20, 200, ... 200000000, all with the same result of info.size=0.</p>
  
  <p>I am now unable to set the render to a SurfaceView or TextureView on my layout. I tried replacing this line:</p>
  
  <pre><code>mSurfaceTexture = new SurfaceTexture(mTextureRender.getTextureId());
  </code></pre>
  
  <p>with this, where <code>surfaceTexture</code> is a SurfaceTexture defined in my xml-layout:</p>
  
  <pre><code>mSurfaceTexture = textureView.getSurfaceTexture();
  mSurfaceTexture.attachToGLContext(mTextureRender.getTextureId());
  </code></pre>
  
  <p>but it throws a weird error with <code>getMessage()==null</code> on the second line. I couldn't find any other way to get it to draw on a View of some kind. How can I change the decoder to display the frames on a Surface/SurfaceView/TextureView instead of saving them?</p>