Answers.Body: <p>The data are stored within the file disregarding of the .mp4 file standard.</p>
  
  <p>the chunk containing those per frame data starts with: <code>#¿Ttlyd</code> or <code>23C05474 6C7964</code>in hex
  <a href="https://i.sstatic.net/qAIyv.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/qAIyv.png" alt="enter image description here"></a></p>
  
  <p>Each frame has 36 bytes of data, those seem to be 9 floats of normalized (from -1 to 1) numbers</p>
  
  <p>Those numbers seem to be a 3x3 matrix representing the current coordinate system for this frame.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-28T16:13:21.117Z
Answers.Text: Did you find out if it is really a 3x3 rotation matrix, and what is the format of the floats?
Answers.UserDisplayName: Juan Cespedes
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-12-29T15:44:16.810Z
Answers.Text: Well… kind of but it's a mess to get the proper axis. Check my repo for a working viewer POC made in angular. https://github.com/Xample/mi-sphere-gyro-extractor
Answers.UserDisplayName: Flavien Volken
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-09T19:10:35.250Z
Answers.LastActivityDate: 2018-04-28T20:43:30.163Z
Answers.LastEditDate: 2018-04-28T20:43:30.163Z
Answers.UserDisplayName: Flavien Volken
Answers.Body: <p>There is a <code>madv</code> box in the <code>udta</code> atom that is not normally seen in MP4s</p>
  
  <pre><code>type:'madv' parent:'udta' sz: 3145728 115 3145835
  </code></pre>
  
  <p>It has sufficient size for it to be a candidate.</p>
  
  <p>This is a dump of the first ~350 bytes</p>
  
  <pre><code>madv: s=   3145728 (0x00300000), o=   1950821 (0x001dc465)
    000000   00 00 00 ac 63 75 74 6c  00 00 00 00 00 00 00 00  |....cutl........|
    000010   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000020   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000030   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000040   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000050   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000060   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000070   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000080   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000090   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    0000a0   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 0c  |................|
    0000b0   66 6c 74 72 00 00 00 00  00 00 00 88 63 61 69 66  |fltr........caif|
    0000c0   32 30 31 38 30 33 33 31  00 00 00 00 00 00 00 00  |20180331........|
    0000d0   31 31 00 00 30 00 00 00  00 00 00 00 00 00 00 00  |11..0...........|
    0000e0   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    0000f0   51 4a 58 4a 30 31 46 4a  00 00 00 00 00 00 00 00  |QJXJ01FJ........|
    000100   31 2e 33 2e 31 30 34 2e  31 31 31 2e 35 2e 31 37  |1.3.104.111.5.17|
    000110   33 33 32 00 00 00 00 00  00 00 00 00 00 00 00 00  |332.............|
    000120   31 35 39 30 38 2f 30 30  30 30 38 30 36 32 00 00  |15908/00008062..|
    000130   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000140   00 00 02 5c 67 70 73 78  00 00 00 00 00 00 00 00  |...\gpsx........|
    000150   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  </code></pre>
  
  <p>Although not knowing the storage format, I can't confirm it or parse it.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-04-09T12:53:03.417Z
Answers.LastActivityDate: 2018-04-09T12:53:03.417Z
Answers.UserDisplayName: Gyan
Body: <p>I bought a Xiaomi Mijia MI SPHERE 360 Camera, it has an embedded gyroscope. The iOS app allows exporting stabilized movies combining the video and gyro data.</p>
  
  <p>The camera records .mp4 files of the following format: <code>ISO Media, MPEG v4 system, 3GPP JVT AVC [ISO 14496-12:2005]</code></p>
  
  <p>I would like to retrieve the gyro information of my movie along the movie. I can imagine there is exif like metadata for each frame where the camera would store specific information but I have not found any way yet to put the hand on it.</p>
  
  <p>How could I retrieve the metadata per frame containing this information?</p>
  
  <p>[Edit]
  Here is a <a href="https://github.com/Xample/mi-sphere-gyro-extractor/tree/master/data" rel="nofollow noreferrer">github</a> project containing 2 short (10sec) clips I made. I recorded in the dark. One is steady (camera laid on the ground) the second one shaking. Both have a useless? audio track.</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-06T10:06:42.923Z
Comments.Text: Does the iOS app work with a rewrapped file: `ffmpeg -i orig.mp4 -c copy new.mp4`
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-07T22:52:44.990Z
Comments.Text: @Mulvya I made the copy and put it back on the key, the file is seen by the iOS app as corrupted, there is no preview thumbnail and downloading it results in a 360 video without the gyro calibration and without the lens calibration neither. It is like the app takes a FOV of +200° instead of 190°. I, therefore, suspect the file to contain a padding of proprietary data holding the gyro, thumbnail and lens calibration information. Next step… to find it and decrypt.
Comments.UserDisplayName: Flavien Volken
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-09T12:15:36.883Z
Comments.Text: @Mulvya you can post an answer as there is none yet and I've 16 hours remaining to award the bounty (or it will just be lost forever).
Comments.UserDisplayName: Flavien Volken
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-04-09T12:53:25.730Z
Comments.Text: Added some info as answer.
Comments.UserDisplayName: Gyan
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2018-09-04T19:30:41.220Z
Comments.Text: In the photos the data is for sure stored in a proprietary way. Usually such data is stored in custom exif tags, but there are no exif tags in the images even though they contain the information. The exif tag called User comment actually has exactly 36 bytes of binary data, which corresponds to the answer by Flavien Volken.
Comments.UserDisplayName: simonmikkelsen
Tags: <360video><metadata>
Title: Mi Sphere 360, where is stored the gyro data?
UserDisplayName: Flavien Volken
ThreadText: # Mi Sphere 360, where is stored the gyro data?
  
  (Asked by: username_0 on 2018-03-31)
  
  I bought a Xiaomi Mijia MI SPHERE 360 Camera, it has an embedded gyroscope. The iOS app allows exporting stabilized movies combining the video and gyro data.
  
  The camera records .mp4 files of the following format: `ISO Media, MPEG v4 system, 3GPP JVT AVC [ISO 14496-12:2005]`
  
  I would like to retrieve the gyro information of my movie along the movie. I can imagine there is exif like metadata for each frame where the camera would store specific information but I have not found any way yet to put the hand on it.
  
  How could I retrieve the metadata per frame containing this information?
  
  [Edit]
  Here is a [github](https://github.com/Xample/mi-sphere-gyro-extractor/tree/master/data) project containing 2 short (10sec) clips I made. I recorded in the dark. One is steady (camera laid on the ground) the second one shaking. Both have a useless? audio track.
  
  **Tags:** 360video, metadata
  
  > **Comments:**
  > **username_1**: Does the iOS app work with a rewrapped file: `ffmpeg -i orig.mp4 -c copy new.mp4`  
  > **username_0**: @Mulvya I made the copy and put it back on the key, the file is seen by the iOS app as corrupted, there is no preview thumbnail and downloading it results in a 360 video without the gyro calibration and without the lens calibration neither. It is like the app takes a FOV of +200° instead of 190°. I, therefore, suspect the file to contain a padding of proprietary data holding the gyro, thumbnail and lens calibration information. Next step… to find it and decrypt.  
  > **username_0**: @Mulvya you can post an answer as there is none yet and I've 16 hours remaining to award the bounty (or it will just be lost forever).  
  > **username_1**: Added some info as answer.  
  > **username_2**: In the photos the data is for sure stored in a proprietary way. Usually such data is stored in custom exif tags, but there are no exif tags in the images even though they contain the information. The exif tag called User comment actually has exactly 36 bytes of binary data, which corresponds to the answer by username_0 username_0.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_0 on 2018-04-09. Score: 2
  
  The data are stored within the file disregarding of the .mp4 file standard.
  
  the chunk containing those per frame data starts with: `#¿Ttlyd` or `23C05474 6C7964`in hex
  [![enter image description here](https://i.sstatic.net/qAIyv.png)](https://i.sstatic.net/qAIyv.png)
  
  Each frame has 36 bytes of data, those seem to be 9 floats of normalized (from -1 to 1) numbers
  
  Those numbers seem to be a 3x3 matrix representing the current coordinate system for this frame.
  
  > **Comments:**
  > **username_3**: Did you find out if it is really a 3x3 rotation matrix, and what is the format of the floats?  
  > **username_0**: Well… kind of but it's a mess to get the proper axis. Check my repo for a working viewer POC made in angular. https://github.com/Xample/mi-sphere-gyro-extractor  
  
  ---
  
  ### Answer by username_1 on 2018-04-09. Score: 1
  
  There is a `madv` box in the `udta` atom that is not normally seen in MP4s
  
  ```
  type:'madv' parent:'udta' sz: 3145728 115 3145835
  
  ```
  
  It has sufficient size for it to be a candidate.
  
  This is a dump of the first ~350 bytes
  
  ```
  madv: s=   3145728 (0x00300000), o=   1950821 (0x001dc465)
    000000   00 00 00 ac 63 75 74 6c  00 00 00 00 00 00 00 00  |....cutl........|
    000010   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000020   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000030   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000040   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000050   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000060   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000070   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000080   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000090   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    0000a0   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 0c  |................|
    0000b0   66 6c 74 72 00 00 00 00  00 00 00 88 63 61 69 66  |fltr........caif|
    0000c0   32 30 31 38 30 33 33 31  00 00 00 00 00 00 00 00  |20180331........|
    0000d0   31 31 00 00 30 00 00 00  00 00 00 00 00 00 00 00  |11..0...........|
    0000e0   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    0000f0   51 4a 58 4a 30 31 46 4a  00 00 00 00 00 00 00 00  |QJXJ01FJ........|
    000100   31 2e 33 2e 31 30 34 2e  31 31 31 2e 35 2e 31 37  |1.3.104.111.5.17|
    000110   33 33 32 00 00 00 00 00  00 00 00 00 00 00 00 00  |332.............|
    000120   31 35 39 30 38 2f 30 30  30 30 38 30 36 32 00 00  |15908/00008062..|
    000130   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
    000140   00 00 02 5c 67 70 73 78  00 00 00 00 00 00 00 00  |...\gpsx........|
    000150   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
  
  ```
  
  Although not knowing the storage format, I can't confirm it or parse it.
  
  
  ---
  
  