Answers.Body: <p>As per my reading of the demuxer, this is the method to derive the PTS from a base 16 dump.</p>
  
  <pre><code>b'4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37...
  </code></pre>
  
  <p>Skip the first 7 bytes, or more precisely, the first 3 bytes after the PID (<code>00ff</code>). The timestamp is derived from the next 6 bytes.</p>
  
  <pre><code>a7686c2b43f5
  </code></pre>
  
  <p>There are two components to the PCR - the 90 kHz clock and a 27 Mhz extension.</p>
  
  <p>First left-shift the initial 4 bytes by 1</p>
  
  <pre><code>a7686c2b &lt;&lt; 1 = 14ed0d856
  </code></pre>
  
  <p>Then right-shift the 5th byte by 7 i.e. isolate the high bit.</p>
  
  <pre><code>43 &gt;&gt; 7 = 0
  </code></pre>
  
  <p>Bitwise-OR the above two results.</p>
  
  <pre><code>14ed0d856 | 0 = 14ed0d856
  </code></pre>
  
  <p>The above result is the first component C1.</p>
  
  <p>Now, isolate the low bit of the 5th byte and left-shift it by 8</p>
  
  <pre><code>(43 &amp; 1) &lt;&lt; 8 = 1
  </code></pre>
  
  <p>Bitwise-OR this result with the 6th byte</p>
  
  <pre><code>100000000 | f5 = 01f5
  </code></pre>
  
  <p>This is the 2nd component C2.</p>
  
  <p>The timestamp in the 27 Mhz timebase is <code>C1 * 300 + C2</code>. In seconds, that is </p>
  
  <pre><code>(5617277014 * 300 + 501) / 27000000 = 62414.189063s
  </code></pre>
  
  <p>You can ignore C2 and just divide C1 by 90000 to get a less precise timestamp.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-20T12:27:18.420Z
Answers.Text: I have used a TS analyzer software to extract the PCR from the payload and for this payload: `4700FF36071001922857FE9AC1EFE05B1A54D5EFD373CA543CA6C9F07D50E442` the PCR is constituted by the 7th till 11th byte: `01 92 28 57 FE` so for these 5 bytes form the PCR base: 32450AF (527115999 or 27MHz  15813479854 and the human time is 00:09:45:6844). So apparently your approach for calculating the PCR isn't correct.
Answers.UserDisplayName: Georgi Stoyanov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-20T12:59:09.030Z
Answers.Text: What is the PCR supposed to be?
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-20T13:19:22.770Z
Answers.Text: PCR is the program clock reference. I found a source where this is all explained and I have answered my own question. I have made a mistake in my previous comment and actually, for the PCR calculation, you need 6 bytes, between the 7th and the 12th.
Answers.UserDisplayName: Georgi Stoyanov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-21T14:25:43.040Z
Answers.Text: on top of that, I have figured out that this packet has the `adaptation_field_control = 01` which means that there isn't adaptation field present in the packet and after the header starts the payload directly and PCR is actually carried in the adaptation field.
Answers.UserDisplayName: Georgi Stoyanov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-16T05:59:25.160Z
Answers.LastActivityDate: 2018-11-20T14:02:00.877Z
Answers.LastEditDate: 2018-11-20T14:02:00.877Z
Answers.UserDisplayName: Gyan
Answers.Body: <p>The PCR is constructed by two components PCR base and PCR extension. In order to extract it you need to get the binary(hex) data between the 7th and 12th bytes (6 in total). So the first 33 bits are constructing the pcr_base, then there are 6 bits which are reserved and 9 bits constructing the pcr_extension. The <code>PCR = pcr_base * 300 + pcr_extension</code>. So in my example for this VPID:</p>
  
  <pre><code>4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37e10409a1dcbdd95dd1b9492ccbf4d75518e357d00cdb41536b54c2e5eb093d46f006835f9ebc39d6e5fe2401f1495cb8d2f9a6190ac3f3d2e519115ca05df63295d94a2c4b4b1c1b921e1a5610e9b7f6ccc69693a142807f4d09ac1ae53b61125f8a0e3efe1f11cb95f0e75dd49231dc0aa02aa3e72045783cb27ecfb591cdae658b04fd75e3f68968da6710f3a08ec3dc84d1eb04fac0e337b21d47560d457b58ad7f6
  </code></pre>
  
  <p>The PCR will be <code>0xa7686c2b43f5</code> so when we convert the bytes into binary we have <code>0b101001110110100001101100001010110100001111110101</code>, so the first 33 bits constitute the pcr_base and the last 9 the pcr_extension, the six bits in between are reserved, so you can ignore them. </p>
  
  <pre><code>pcr_base = 0b101001110110100001101100001010110 = 5617277014
  pcr_extension = 0b111110101 = 501 
  PCR = 5617277014 * 300 + 501 = 1685183104701
  </code></pre>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-20T13:32:43.153Z
Answers.Text: Ok, I got the offset wrong - it's 3 bytes after the PID, not 6.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-20T13:45:14.273Z
Answers.Text: I think you don't need to left shift the initial 4 bytes, the Bitwise-OR also isn't needed in the case, you just need to concatenate the high bit of the 5th byte to the binary representation of the first 4 bytes at the end and then to add at the beginning the lower bit of the fifth byte to the binary representation of the last byte.
Answers.UserDisplayName: Georgi Stoyanov
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-20T13:47:25.453Z
Answers.Text: Yes, when doing this manually or as a string op. The code I demonstrated is when working in C or similar env.
Answers.UserDisplayName: Gyan
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-20T13:14:01.637Z
Answers.LastActivityDate: 2018-11-20T13:14:01.637Z
Answers.UserDisplayName: Georgi Stoyanov
Body: <p>I am trying to write a Python script which will connect the TDT with the PTS in a MPEG-2 TS. So far I have managed to convert the TS file into a <code>.dat</code> file where I have all the hex representation of the 188 bytes on a different line. I have couple of problems:</p>
  
  <ol>
  <li>I want to separate the frames and put an incremental number in front of the packets from this frame. For example, all the packets belonging to frame Nr. 1 to have fr%5d in front. I can easily do that using Python as long as I know how I can separate between the frames. </li>
  <li>I want to decode the PTS from the PCR. So far I have extracted the PTS values using FFMPEG and I have a list for each frame which PTS it has. I have tried to grep the <code>.dat</code> file for the hex representation of the PTS but it doesn't find any matches. So I guess the PTS is coded in a different form, but so far I wasn't able to figure out the syntax of it. </li>
  </ol>
  
  <p>My <code>.dat</code> file has the following content: </p>
  
  <pre><code>b'4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37e10409a1dcbdd95dd1b9492ccbf4d75518e357d00cdb41536b54c2e5eb093d46f006835f9ebc39d6e5fe2401f1495cb8d2f9a6190ac3f3d2e519115ca05df63295d94a2c4b4b1c1b921e1a5610e9b7f6ccc69693a142807f4d09ac1ae53b61125f8a0e3efe1f11cb95f0e75dd49231dc0aa02aa3e72045783cb27ecfb591cdae658b04fd75e3f68968da6710f3a08ec3dc84d1eb04fac0e337b21d47560d457b58ad7f6'  b'4700ff1e59ab778d0013a08a07f9733c31daab4375ce1b44b39e9426d26963dff9a87601533d3a7c326849248ee8f0ab6f9677c8b19c198f3da107a76a10bda99dfec628136c5dd7508ec2c7546db32eef7027ed91d8dffe1c64672d41b5d848478bb2ae8c41c62f3305b29005912965a222948a77a81de7cf59cfd745c4a469d595d50432ce05704e7e4e8edadeb250cb792780a410223d4810e72bc67ddba39b7c5d0bf48e6aea508f2dec5c1a2573210610c412f4020d3e0bd130'
  b'4700ff1f2b05cbec47587f67c444b52f95d51cf52f2e54ad60cb2089015c1286b88bde1ada2290f7de9a8955c112d122dd83884127a85d60d783d0c7ac62ce7d2176d98f8b7ffa82c45d661394e3f6765ba4226e030c485f709f8cd57da40dfbaceed7332d82f4dc8bce05ca6ed5bc3262399b606f9c16daada74b65691729a48659f63aa847c96d41467a98134ee2eda6edf7e5569f3b5ec04fc274d8026bf182542be6529985159df52fda6db38908853577ac3ecb716126186b3a'
  b'4700ff10625a2814a692660f91c9802011f0cfefd21029bc4d574921a5ff61daed60f4865765272d7885a5f6d1f00a6b911566041f9b09d5fb8543286f027e5d1790c883e9fc30f11e67e64b4eba58845563976d5055333c9b5292c28b202dd74457d7900356ce4581e63e7be76ac741685039f37f345adcaab4d77c38b9cc98c9969dbd5016b361c6a641b60f76773f3b0e11fee995bded7012f80fad299ca44c4a7c4715114c52edcaf44521d1bbe12cddb89a4711094a8853920d'
  b'4700ff1179fb10469aedae24c5e85cdda33de151c1bd83e8f7573f44d101d3a11b42ba9599c02f134a4bd4795e5afcdade291bc1a3e8a1da4da150cbe94c863e8cd0c04ce686333af6ba25f2f645170c387c4346df8c7e85cda2be546136e09e12b612104ada9990c12d745e1de68a587114c9e268b1e14dd5f68c5037ed2c3acb2aa65e988ac990f2ae1089e962ffd495e5580e8f3525fa3386ed752432d443ff3ca93ce86a20b9114c3d2554b5e701e41d9548c903a3c17498a49b'
  </code></pre>
  
  <p>So as you can see each packet starts with 0x47 which is the sync byte. I have all kind of PIDs inside the TS. Here I have shown only the VPID 0x00ff. The 0x47 is the sync byte and immediately after it comes the PID. </p>
  
  <p>[EDIT]: I have uploaded the binary file here: <a href="https://wetransfer.com/downloads/bc970064ed13e9480f694bc4298393a420181115204357/456ae9" rel="nofollow noreferrer">wetransfer_link</a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-15T18:04:12.293Z
Comments.Text: What is the timestamp of the first packet above supposed to be? Your 2nd, 3rd and 5th lines don't look like actual data.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-15T18:51:51.103Z
Comments.Text: @Gyan, yes, that's because I have stuffing in my TS. 0x47 is the syncing byte and then there are 4 bytes for the PID. My VPID is 0x00ff, corresponds to the first and third line. PID 0x1fff is actually the last PID, which is also the null PID, so you should ignore line 2, 3, 5. As a matter of fact, my file is much larger. If you want I can upload it somewhere.
Comments.UserDisplayName: Georgi Stoyanov
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-15T18:59:31.030Z
Comments.Text: Anyway, I have put only information about the VPIDs in the thread, so no more stuffing bytes. I just want to know to which frame the packets in the TS belong, in order to be able to connect the PTS with the TDT.
Comments.UserDisplayName: Georgi Stoyanov
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-15T20:05:07.690Z
Comments.Text: What is the timestamp of the first packet above supposed to be?
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-11-15T21:01:52.677Z
Comments.Text: @Gyan, this is what I want to know, I want to connect the frames with their respective PTS timestamps and with human time extracted from the TDT. I have already written a script decoding the TDT time, I just need to define which packet to which frame belong and to find a way to extract the PTS from the PCR PID. I have also added a link to the binary file. Each packet is starting on a new line similar to the format from my main post.
Comments.UserDisplayName: Georgi Stoyanov
Tags: <ffmpeg><mpeg>
Title: Extract PTS, Frame Number from a TS file using the bytes representation of it
UserDisplayName: Georgi Stoyanov
ThreadText: # Extract PTS, Frame Number from a TS file using the bytes representation of it
  
  (Asked by: username_0 on 2018-11-15)
  
  I am trying to write a Python script which will connect the TDT with the PTS in a MPEG-2 TS. So far I have managed to convert the TS file into a `.dat` file where I have all the hex representation of the 188 bytes on a different line. I have couple of problems:
  
  1. I want to separate the frames and put an incremental number in front of the packets from this frame. For example, all the packets belonging to frame Nr. 1 to have fr%5d in front. I can easily do that using Python as long as I know how I can separate between the frames.
  2. I want to decode the PTS from the PCR. So far I have extracted the PTS values using FFMPEG and I have a list for each frame which PTS it has. I have tried to grep the `.dat` file for the hex representation of the PTS but it doesn't find any matches. So I guess the PTS is coded in a different form, but so far I wasn't able to figure out the syntax of it.
  
  My `.dat` file has the following content:
  
  ```
  b'4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37e10409a1dcbdd95dd1b9492ccbf4d75518e357d00cdb41536b54c2e5eb093d46f006835f9ebc39d6e5fe2401f1495cb8d2f9a6190ac3f3d2e519115ca05df63295d94a2c4b4b1c1b921e1a5610e9b7f6ccc69693a142807f4d09ac1ae53b61125f8a0e3efe1f11cb95f0e75dd49231dc0aa02aa3e72045783cb27ecfb591cdae658b04fd75e3f68968da6710f3a08ec3dc84d1eb04fac0e337b21d47560d457b58ad7f6'  b'4700ff1e59ab778d0013a08a07f9733c31daab4375ce1b44b39e9426d26963dff9a87601533d3a7c326849248ee8f0ab6f9677c8b19c198f3da107a76a10bda99dfec628136c5dd7508ec2c7546db32eef7027ed91d8dffe1c64672d41b5d848478bb2ae8c41c62f3305b29005912965a222948a77a81de7cf59cfd745c4a469d595d50432ce05704e7e4e8edadeb250cb792780a410223d4810e72bc67ddba39b7c5d0bf48e6aea508f2dec5c1a2573210610c412f4020d3e0bd130'
  b'4700ff1f2b05cbec47587f67c444b52f95d51cf52f2e54ad60cb2089015c1286b88bde1ada2290f7de9a8955c112d122dd83884127a85d60d783d0c7ac62ce7d2176d98f8b7ffa82c45d661394e3f6765ba4226e030c485f709f8cd57da40dfbaceed7332d82f4dc8bce05ca6ed5bc3262399b606f9c16daada74b65691729a48659f63aa847c96d41467a98134ee2eda6edf7e5569f3b5ec04fc274d8026bf182542be6529985159df52fda6db38908853577ac3ecb716126186b3a'
  b'4700ff10625a2814a692660f91c9802011f0cfefd21029bc4d574921a5ff61daed60f4865765272d7885a5f6d1f00a6b911566041f9b09d5fb8543286f027e5d1790c883e9fc30f11e67e64b4eba58845563976d5055333c9b5292c28b202dd74457d7900356ce4581e63e7be76ac741685039f37f345adcaab4d77c38b9cc98c9969dbd5016b361c6a641b60f76773f3b0e11fee995bded7012f80fad299ca44c4a7c4715114c52edcaf44521d1bbe12cddb89a4711094a8853920d'
  b'4700ff1179fb10469aedae24c5e85cdda33de151c1bd83e8f7573f44d101d3a11b42ba9599c02f134a4bd4795e5afcdade291bc1a3e8a1da4da150cbe94c863e8cd0c04ce686333af6ba25f2f645170c387c4346df8c7e85cda2be546136e09e12b612104ada9990c12d745e1de68a587114c9e268b1e14dd5f68c5037ed2c3acb2aa65e988ac990f2ae1089e962ffd495e5580e8f3525fa3386ed752432d443ff3ca93ce86a20b9114c3d2554b5e701e41d9548c903a3c17498a49b'
  
  ```
  
  So as you can see each packet starts with 0x47 which is the sync byte. I have all kind of PIDs inside the TS. Here I have shown only the VPID 0x00ff. The 0x47 is the sync byte and immediately after it comes the PID.
  
  [EDIT]: I have uploaded the binary file here: [wetransfer\_link](https://wetransfer.com/downloads/bc970064ed13e9480f694bc4298393a420181115204357/456ae9)
  
  **Tags:** ffmpeg, mpeg
  
  > **Comments:**
  > **username_1**: What is the timestamp of the first packet above supposed to be? Your 2nd, 3rd and 5th lines don't look like actual data.  
  > **username_0**: @username_1, yes, that's because I have stuffing in my TS. 0x47 is the syncing byte and then there are 4 bytes for the PID. My VPID is 0x00ff, corresponds to the first and third line. PID 0x1fff is actually the last PID, which is also the null PID, so you should ignore line 2, 3, 5. As a matter of fact, my file is much larger. If you want I can upload it somewhere.  
  > **username_0**: Anyway, I have put only information about the VPIDs in the thread, so no more stuffing bytes. I just want to know to which frame the packets in the TS belong, in order to be able to connect the PTS with the TDT.  
  > **username_1**: What is the timestamp of the first packet above supposed to be?  
  > **username_0**: @username_1, this is what I want to know, I want to connect the frames with their respective PTS timestamps and with human time extracted from the TDT. I have already written a script decoding the TDT time, I just need to define which packet to which frame belong and to find a way to extract the PTS from the PCR PID. I have also added a link to the binary file. Each packet is starting on a new line similar to the format from my main post.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-11-16. Score: 1
  
  As per my reading of the demuxer, this is the method to derive the PTS from a base 16 dump.
  
  ```
  b'4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37...
  
  ```
  
  Skip the first 7 bytes, or more precisely, the first 3 bytes after the PID (`00ff`). The timestamp is derived from the next 6 bytes.
  
  ```
  a7686c2b43f5
  
  ```
  
  There are two components to the PCR - the 90 kHz clock and a 27 Mhz extension.
  
  First left-shift the initial 4 bytes by 1
  
  ```
  a7686c2b << 1 = 14ed0d856
  
  ```
  
  Then right-shift the 5th byte by 7 i.e. isolate the high bit.
  
  ```
  43 >> 7 = 0
  
  ```
  
  Bitwise-OR the above two results.
  
  ```
  14ed0d856 | 0 = 14ed0d856
  
  ```
  
  The above result is the first component C1.
  
  Now, isolate the low bit of the 5th byte and left-shift it by 8
  
  ```
  (43 & 1) << 8 = 1
  
  ```
  
  Bitwise-OR this result with the 6th byte
  
  ```
  100000000 | f5 = 01f5
  
  ```
  
  This is the 2nd component C2.
  
  The timestamp in the 27 Mhz timebase is `C1 * 300 + C2`. In seconds, that is
  
  ```
  (5617277014 * 300 + 501) / 27000000 = 62414.189063s
  
  ```
  
  You can ignore C2 and just divide C1 by 90000 to get a less precise timestamp.
  
  > **Comments:**
  > **username_0**: I have used a TS analyzer software to extract the PCR from the payload and for this payload: `4700FF36071001922857FE9AC1EFE05B1A54D5EFD373CA543CA6C9F07D50E442` the PCR is constituted by the 7th till 11th byte: `01 92 28 57 FE` so for these 5 bytes form the PCR base: 32450AF (527115999 or 27MHz  15813479854 and the human time is 00:09:45:6844). So apparently your approach for calculating the PCR isn't correct.  
  > **username_1**: What is the PCR supposed to be?  
  > **username_0**: PCR is the program clock reference. I found a source where this is all explained and I have answered my own question. I have made a mistake in my previous comment and actually, for the PCR calculation, you need 6 bytes, between the 7th and the 12th.  
  > **username_0**: on top of that, I have figured out that this packet has the `adaptation_field_control = 01` which means that there isn't adaptation field present in the packet and after the header starts the payload directly and PCR is actually carried in the adaptation field.  
  
  ---
  
  ### Answer by username_0 on 2018-11-20. Score: 0
  
  The PCR is constructed by two components PCR base and PCR extension. In order to extract it you need to get the binary(hex) data between the 7th and 12th bytes (6 in total). So the first 33 bits are constructing the pcr\_base, then there are 6 bits which are reserved and 9 bits constructing the pcr\_extension. The `PCR = pcr_base * 300 + pcr_extension`. So in my example for this VPID:
  
  ```
  4700ff1de05ca7686c2b43f5e37e6dafd388761c36900ab37e10409a1dcbdd95dd1b9492ccbf4d75518e357d00cdb41536b54c2e5eb093d46f006835f9ebc39d6e5fe2401f1495cb8d2f9a6190ac3f3d2e519115ca05df63295d94a2c4b4b1c1b921e1a5610e9b7f6ccc69693a142807f4d09ac1ae53b61125f8a0e3efe1f11cb95f0e75dd49231dc0aa02aa3e72045783cb27ecfb591cdae658b04fd75e3f68968da6710f3a08ec3dc84d1eb04fac0e337b21d47560d457b58ad7f6
  
  ```
  
  The PCR will be `0xa7686c2b43f5` so when we convert the bytes into binary we have `0b101001110110100001101100001010110100001111110101`, so the first 33 bits constitute the pcr\_base and the last 9 the pcr\_extension, the six bits in between are reserved, so you can ignore them.
  
  ```
  pcr_base = 0b101001110110100001101100001010110 = 5617277014
  pcr_extension = 0b111110101 = 501 
  PCR = 5617277014 * 300 + 501 = 1685183104701
  
  ```
  
  > **Comments:**
  > **username_1**: Ok, I got the offset wrong - it's 3 bytes after the PID, not 6.  
  > **username_0**: I think you don't need to left shift the initial 4 bytes, the Bitwise-OR also isn't needed in the case, you just need to concatenate the high bit of the 5th byte to the binary representation of the first 4 bytes at the end and then to add at the beginning the lower bit of the fifth byte to the binary representation of the last byte.  
  > **username_1**: Yes, when doing this manually or as a string op. The code I demonstrated is when working in C or similar env.  
  
  ---
  
  