Spaces:
Running
on
Zero
16-bit raw depth map output is practically 8-bit
The current demo outputs 16-bit raw depth map PNG. But you've probably noticed it's essentially black. If you open it in Photoshop and adjust levels / exposure you'll be able to get a proper white-to-black disparity map out of it, but it will feature strong gray banding artefacts.
If you open it with UPNG for example - you'll be able to see that for every 2-byte pixel the 1st byte is essentially zero in 99.9% of pixels - which means this is just a 8-bit grayscale image packaged as 16-bit with 1/255 multiplier. On some occasions it might output values more than 255 - but it happens only in particular images, and the largest value I was able to get was 537!
From exploring app.py - looks like raw depth isn't normalized to depth min - depth max range unlike color ramp is. May be it's like that by design, but then raw 16-bit depth has limited practical use.
would love to see this fixed, I've tried using these as displacement maps in blender and the results are stair-stepped due to the banding.
What kind of focal length and sensor size do you use to reproject this depth map? Or do you get this from the image file meta data? Does the model use this meta data?
@psoetens
I'm testing it with my custom internal tooling (see my demo for Leia Pix: https://twitter.com/antmatrosov/status/1749343136713752847).
Looks like depth coming from the model is still 0...1 float - it just needs to be properly normalized to:
depth = (depth - depth.min()) / (depth.max() - depth.min()) * 65535.0 before exporting it to uint16
I'll try to run a custom clone of this project and check it out
I tried normalized 8-bit depth map, it has similar results as normalized raw depth map.