Answers.Body: <p>This doesn't work, at least not exclusively (more on that after some background).  With stabilization, there are two main ways to do it.  You can either stabilize during shooting by keeping the camera steady or after by taking unstable video, counteracting some of the motion and adjusting the image to account for other types of motion.  </p>
  
  <p>When you stabilize during shooting, gyroscopes, gimbals and springs are used to cancel out forces trying to move the camera to actually keep it stable.  This works because the camera doesn't actually move.  Things like optical image stabilization and steadicams perform this function.</p>
  
  <p>Correction after the fact is possible, but harder.  When a camera moves, the problem is that it changes what the shot looks like.  Both angular movement and perspective can vary based on the actual settings of the camera and not just the movement, so there isn't a direct mapping from movement to correction.  </p>
  
  <p>There are two main types of camera movement, the camera can move through space, which results in near objects appearing to move more than far away objects as the relative change is greater.  (Think of a block directly in front of the lens vs a mountain in the distance, the block can clear the screen entirely while the mountain doesn't appear to have moved.)  </p>
  
  <p>The other form of movement is rotation.  This makes the rate of change depend on how zoomed in we are (the focal length of our lens).  If we have a very wide shot, then slight vibrations don't make much difference.  If we zoom in a long ways to a tight shot however, even the smallest vibrations have a major impact on the shot.</p>
  
  <p>None of this information would be available to the data logging on the smartphone, so a lot still needs to be figured out by analyzing the footage.  You additionally would need to somehow keep the smartphone and camera together which a) adds weight, b) adds bulk and c) decreases the ergonomics.  The data really isn't that valuable as that information and all the other information we need can easily be gained simply from looking for high contrast areas of the image.</p>
  
  <p>Stabilizing after the fact works by doing edge detection on each frame and looking at how the edges move from frame to frame.  These edges provide details about where things are in the scene and allow us to actually recreate the motions of the camera and the objects within the scene by looking at all the changes.  If they are consistent overtime with a stationary scene, we know they are static objects, if they don't, we know they are moving objects.</p>
  
  <p>So, basically, the data the smartphone would gather isn't needed, isn't that helpful and adds too much disadvantage in trying to capture it to justify the gathering in the first place.  It could possibly save on some post processing load to simplify calculations from edges or provide improvements in cases where the footage lacks good trackable subjects, but generally, I wouldn't expect it to be worth much.  If it costs more than $100 or so, you are better off going with hardware stabilization that will provide far more benefit.</p>
  
  <p>For more of an idea about what can be determined about video just from analysing the footage itself, I suggest taking a look at Microsoft Research's <a href="http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/" rel="nofollow">Hyperlapse Project</a>.  Pay particular attention to the path they are able to derive as that is the same thing a device lack this would provide the rotational and acceleration data would be helpful towards determining.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-10-16T16:11:06.510Z
Answers.LastActivityDate: 2014-10-16T20:18:07.097Z
Answers.LastEditDate: 2014-10-16T20:18:07.097Z
Answers.UserDisplayName: AJ Henderson
Answers.Body: <p><a href="http://steadxp.com" rel="nofollow">SteadXp</a> works like you describe, although it is still a product in development.  Also, it uses its own accelerometers, rather than using your cell phone's.  Presumably, this is simpler than relying on the myriad protocols of various cell phone manufacturers.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-10-16T19:53:50.013Z
Answers.Text: Data gathering on the phone wouldn't be that hard.  It's available to the OS on most phones and can be accessed in a standard way through that.  The bigger trick would be reliable mounting.
Answers.UserDisplayName: AJ Henderson
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2014-10-16T19:08:28.947Z
Answers.LastActivityDate: 2014-10-16T19:08:28.947Z
Answers.UserDisplayName: Jason Conrad
Answers.Body: <p>The Sony a7c, FX9, RX0, a7siii and ZV-1 put the gyro data in metadata.  Sony provides free software (Sony Catalyst) to use the gyro data to stabilize video in post where you can choose how much of the frame is cropped for stabilization.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-09-30T15:05:49.947Z
Answers.LastActivityDate: 2020-09-30T15:05:49.947Z
Answers.UserDisplayName: wooac
Body: <p>So I had an idea. Almost everyone has smartphone equipped with gyroscope and accelerometer. It wouldn't seem too far fetched to me to strap this smartphone to DSLR and have some app record data from these sensors while shooting video. Then you would load both the video from DSLR and data from your smartphone into computer and use the sensor data to stabilise your footage. Of course you would have to time-sync your data and video and you would probably get just 720 stabilised video out of 1080 original footage because of cropping, but you would get stable video for good coin (if you already have both smartphone and dslr) and in very portable package (unlike 3axis brushless gimbals and seadicams).</p>
  
  <p>Now I am pretty sure I am not first person to think of that. Is there any solution like this? or do you see some obstacle I've missed?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-10-16T16:35:44.843Z
Comments.Text: Cool idea! @AJHenderson's answer below seems to dispute it (he's a really smart guy!), but I like how you were thinking: use the stabilization information from one device and apply it to footage from another, provided they were locked together during the shot.
Comments.UserDisplayName: BrettFromLA
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-10-16T19:59:18.883Z
Comments.Text: @BrettFromLA - I made some additional updates to my answer.  My original post was a bit too harsh on the idea as it does provide some simplification to processing, it just doesn't provide anything you can't theoretically extract with sufficient software processing given the right kind of footage.
Comments.UserDisplayName: AJ Henderson
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-10-16T20:39:25.807Z
Comments.Text: @AJHenderson Right; that makes sense. It seems as though the OP's idea would gather the stabilization data BASED ON THE VISUALS during shooting, whereas the stabilization data could be generated BASED ON THE VISUALS during post-production and produce exactly the same results.
Comments.UserDisplayName: BrettFromLA
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-10-16T20:40:29.547Z
Comments.Text: Gyroscopes, gimbals and springs would eliminate the unavoidable (??) blur that comes from stabilizing in post.
Comments.UserDisplayName: BrettFromLA
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-10-16T20:49:00.847Z
Comments.Text: @BrettFromLA - not quite exact in every case, but in enough to make the value proposition dubious, yes.  As for the blur, yes, that blur is from two things, camera movement during a frame exposure and trying to warp to correct for the perspective of the camera in the wrong place.  Physical stabilization addresses both of those, which is why it is pretty much always preferable to stabilization in post unless space constraints prevent it from being an option.  Mesh clouds (3d models) like MS Research is doing help reduce warp with more accurate guesses, but are still not perfect.
Comments.UserDisplayName: AJ Henderson
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2014-10-16T21:54:36.127Z
Comments.Text: I forgot to mention that I would not expect such method to be ideal, it would obviously be no good to stabilise perspective etc. Stabilising footage by analyzing it in post is well known to me and fact, that in less than ideal conditions this method can be very unreliable led me to think about using phone to detect movement in the first place :-)
Comments.UserDisplayName: Delltar
Tags: <dslr>
Title: Using smartphone sensors to stabilise DSLR video in post
UserDisplayName: Delltar
ThreadText: # Using smartphone sensors to stabilise DSLR video in post
  
  (Asked by: username_0 on 2014-10-16)
  
  So I had an idea. Almost everyone has smartphone equipped with gyroscope and accelerometer. It wouldn't seem too far fetched to me to strap this smartphone to DSLR and have some app record data from these sensors while shooting video. Then you would load both the video from DSLR and data from your smartphone into computer and use the sensor data to stabilise your footage. Of course you would have to time-sync your data and video and you would probably get just 720 stabilised video out of 1080 original footage because of cropping, but you would get stable video for good coin (if you already have both smartphone and dslr) and in very portable package (unlike 3axis brushless gimbals and seadicams).
  
  Now I am pretty sure I am not first person to think of that. Is there any solution like this? or do you see some obstacle I've missed?
  
  **Tags:** dslr
  
  > **Comments:**
  > **username_1**: Cool idea! @username_2's answer below seems to dispute it (he's a really smart guy!), but I like how you were thinking: use the stabilization information from one device and apply it to footage from another, provided they were locked together during the shot.  
  > **username_2**: @username_1 - I made some additional updates to my answer.  My original post was a bit too harsh on the idea as it does provide some simplification to processing, it just doesn't provide anything you can't theoretically extract with sufficient software processing given the right kind of footage.  
  > **username_1**: @username_2 Right; that makes sense. It seems as though the OP's idea would gather the stabilization data BASED ON THE VISUALS during shooting, whereas the stabilization data could be generated BASED ON THE VISUALS during post-production and produce exactly the same results.  
  > **username_1**: Gyroscopes, gimbals and springs would eliminate the unavoidable (??) blur that comes from stabilizing in post.  
  > **username_2**: @username_1 - not quite exact in every case, but in enough to make the value proposition dubious, yes.  As for the blur, yes, that blur is from two things, camera movement during a frame exposure and trying to warp to correct for the perspective of the camera in the wrong place.  Physical stabilization addresses both of those, which is why it is pretty much always preferable to stabilization in post unless space constraints prevent it from being an option.  Mesh clouds (3d models) like MS Research is doing help reduce warp with more accurate guesses, but are still not perfect.  
  > **username_0**: I forgot to mention that I would not expect such method to be ideal, it would obviously be no good to stabilise perspective etc. Stabilising footage by analyzing it in post is well known to me and fact, that in less than ideal conditions this method can be very unreliable led me to think about using phone to detect movement in the first place :-)  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_2 on 2014-10-16. Score: 2
  
  This doesn't work, at least not exclusively (more on that after some background). With stabilization, there are two main ways to do it. You can either stabilize during shooting by keeping the camera steady or after by taking unstable video, counteracting some of the motion and adjusting the image to account for other types of motion.
  
  When you stabilize during shooting, gyroscopes, gimbals and springs are used to cancel out forces trying to move the camera to actually keep it stable. This works because the camera doesn't actually move. Things like optical image stabilization and steadicams perform this function.
  
  Correction after the fact is possible, but harder. When a camera moves, the problem is that it changes what the shot looks like. Both angular movement and perspective can vary based on the actual settings of the camera and not just the movement, so there isn't a direct mapping from movement to correction.
  
  There are two main types of camera movement, the camera can move through space, which results in near objects appearing to move more than far away objects as the relative change is greater. (Think of a block directly in front of the lens vs a mountain in the distance, the block can clear the screen entirely while the mountain doesn't appear to have moved.)
  
  The other form of movement is rotation. This makes the rate of change depend on how zoomed in we are (the focal length of our lens). If we have a very wide shot, then slight vibrations don't make much difference. If we zoom in a long ways to a tight shot however, even the smallest vibrations have a major impact on the shot.
  
  None of this information would be available to the data logging on the smartphone, so a lot still needs to be figured out by analyzing the footage. You additionally would need to somehow keep the smartphone and camera together which a) adds weight, b) adds bulk and c) decreases the ergonomics. The data really isn't that valuable as that information and all the other information we need can easily be gained simply from looking for high contrast areas of the image.
  
  Stabilizing after the fact works by doing edge detection on each frame and looking at how the edges move from frame to frame. These edges provide details about where things are in the scene and allow us to actually recreate the motions of the camera and the objects within the scene by looking at all the changes. If they are consistent overtime with a stationary scene, we know they are static objects, if they don't, we know they are moving objects.
  
  So, basically, the data the smartphone would gather isn't needed, isn't that helpful and adds too much disadvantage in trying to capture it to justify the gathering in the first place. It could possibly save on some post processing load to simplify calculations from edges or provide improvements in cases where the footage lacks good trackable subjects, but generally, I wouldn't expect it to be worth much. If it costs more than $100 or so, you are better off going with hardware stabilization that will provide far more benefit.
  
  For more of an idea about what can be determined about video just from analysing the footage itself, I suggest taking a look at Microsoft Research's [Hyperlapse Project](http://research.microsoft.com/en-us/um/redmond/projects/hyperlapse/). Pay particular attention to the path they are able to derive as that is the same thing a device lack this would provide the rotational and acceleration data would be helpful towards determining.
  
  
  ---
  
  ### Answer by username_3 on 2014-10-16. Score: 1
  
  [SteadXp](http://steadxp.com) works like you describe, although it is still a product in development. Also, it uses its own accelerometers, rather than using your cell phone's. Presumably, this is simpler than relying on the myriad protocols of various cell phone manufacturers.
  
  > **Comments:**
  > **username_2**: Data gathering on the phone wouldn't be that hard.  It's available to the OS on most phones and can be accessed in a standard way through that.  The bigger trick would be reliable mounting.  
  
  ---
  
  ### Answer by username_4 on 2020-09-30. Score: 0
  
  The Sony a7c, FX9, RX0, a7siii and ZV-1 put the gyro data in metadata. Sony provides free software (Sony Catalyst) to use the gyro data to stabilize video in post where you can choose how much of the frame is cropped for stabilization.
  
  
  ---
  
  