Answers.Body: <blockquote>
  <p><em>&quot;would the output data be in a specific format, such as JPEG (identified by the <code>FF D8</code> marker), or would it be raw bytes representing pixel data?&quot;</em></p>
  </blockquote>
  <p>An H.264 decoder should decode to raw YUV by default, although it's generally appreciated for standalone use to have a YUV4MPEG2 mode as well, for easier piping, and to have basic format metadata preserved.</p>
  <p>Usually the encoding is in YU420P format. There are many examples of converting (via a <code>For-loop</code>) any YUV420p data into RGB data. You can finally encode the newly converted RGB into JPEG.</p>
  <blockquote>
  <p><em>&quot;Do you foresee any need for me to purchase a third documentation source?&quot;</em></p>
  </blockquote>
  <p>No, unless you want to also handle audio then you'll need the AAC documentation. <br>That is <code>iso/iec 13818-7</code>. You can also learn by analysing AAC bytes whilst following the source code of some open-source projects that are decoding AAC data. Articles online also explain structure of AAC. You can still get a result without the ISO document itself (<em>eg:</em> read articles from those who repeat its words as they teach about AAC audio's bytes structure).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-30T18:08:54.557Z
Answers.Text: Good, that made me confident.
Answers.UserDisplayName: Daniel
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-29T17:48:45.113Z
Answers.LastActivityDate: 2023-06-29T18:40:13.730Z
Answers.LastEditDate: 2023-06-29T18:40:13.730Z
Answers.UserDisplayName: damian101
Body: <p>I am currently working on a personal programming challenge to develop an MP4 decoder using VB.NET and the .NET Framework 4.8.1 as a WinForms application. I have obtained the <code>ISO 14496-12</code> documentation, which serves as the basic structure for MP4 files.</p>
  <p>At this stage, I have successfully prepared various lists containing frame sizes, chunk positions, and information from the stsc and stts boxes. Additionally, I have been able to identify keyframes within the MP4 files.</p>
  <p>However, I have come to realize that the <code>ISO 14496-12</code> documentation is not sufficient for decoding the frame bytes, which are encoded in H.264/AVC. To proceed with decoding, I understand that I need to refer to the <code>ISO/IEC 14496-10</code> documentation specifically for H.264/AVC.</p>
  <p>My question is twofold:</p>
  <ul>
  <li>When decoding the frame bytes according to the documentation, would the output data be in a specific format, such as JPEG (identified by the FF D8 marker), or would it be raw bytes representing pixel data? This clarification would help me. In the former case, I could simply save an image using .NET functions.</li>
  <li>Considering my current progress and the information I have mentioned, do you foresee any need for me to purchase a third documentation source?</li>
  </ul>
  <p>Any guidance or suggestions would be appreciated.</p>
  
Tags: <mp4>
Title: Decoding H.264/AVC Frames in MP4: What's the Output Format?
UserDisplayName: Daniel
ThreadText: # Decoding H.264/AVC Frames in MP4: What's the Output Format?
  
  (Asked by: username_0 on 2023-06-28)
  
  I am currently working on a personal programming challenge to develop an MP4 decoder using VB.NET and the .NET Framework 4.8.1 as a WinForms application. I have obtained the `ISO 14496-12` documentation, which serves as the basic structure for MP4 files.
  
  At this stage, I have successfully prepared various lists containing frame sizes, chunk positions, and information from the stsc and stts boxes. Additionally, I have been able to identify keyframes within the MP4 files.
  
  However, I have come to realize that the `ISO 14496-12` documentation is not sufficient for decoding the frame bytes, which are encoded in H.264/AVC. To proceed with decoding, I understand that I need to refer to the `ISO/IEC 14496-10` documentation specifically for H.264/AVC.
  
  My question is twofold:
  
  * When decoding the frame bytes according to the documentation, would the output data be in a specific format, such as JPEG (identified by the FF D8 marker), or would it be raw bytes representing pixel data? This clarification would help me. In the former case, I could simply save an image using .NET functions.
  * Considering my current progress and the information I have mentioned, do you foresee any need for me to purchase a third documentation source?
  
  Any guidance or suggestions would be appreciated.
  
  **Tags:** mp4
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-06-29. Score: 2 | Accepted answer
  
  > *"would the output data be in a specific format, such as JPEG (identified by the `FF D8` marker), or would it be raw bytes representing pixel data?"*
  
  An H.264 decoder should decode to raw YUV by default, although it's generally appreciated for standalone use to have a YUV4MPEG2 mode as well, for easier piping, and to have basic format metadata preserved.
  
  Usually the encoding is in YU420P format. There are many examples of converting (via a `For-loop`) any YUV420p data into RGB data. You can finally encode the newly converted RGB into JPEG.
  
  > *"Do you foresee any need for me to purchase a third documentation source?"*
  
  No, unless you want to also handle audio then you'll need the AAC documentation.   
  That is `iso/iec 13818-7`. You can also learn by analysing AAC bytes whilst following the source code of some open-source projects that are decoding AAC data. Articles online also explain structure of AAC. You can still get a result without the ISO document itself (*eg:* read articles from those who repeat its words as they teach about AAC audio's bytes structure).
  
  > **Comments:**
  > **username_0**: Good, that made me confident.  
  
  ---
  
  