Yes, in nutshell, this is it :-)
But knowing depth information is not enough - you must also very precisely know the position of the camera the individual pictures were taken from. Computing both depth information and position of the camera is in fact a very challenging task.