  Schlieren imaging is an optical technique to observe the flow of transparent
media, such as air or water, without any particle seeding. However,
conventional frame-based techniques require both high spatial and temporal
resolution cameras, which impose bright illumination and expensive computation
limitations. Event cameras offer potential advantages (high dynamic range, high
temporal resolution, and data efficiency) to overcome such limitations due to
their bio-inspired sensing principle. This paper presents a novel technique for
perceiving air convection using events and frames by providing the first
theoretical analysis that connects event data and schlieren. We formulate the
problem as a variational optimization one combining the linearized event
generation model with a physically-motivated parameterization that estimates
the temporal derivative of the air density. The experiments with accurately
aligned frame- and event camera data reveal that the proposed method enables
event cameras to obtain on par results with existing frame-based optical flow
techniques. Moreover, the proposed method works under dark conditions where
frame-based schlieren fails, and also enables slow-motion analysis by
leveraging the event camera's advantages. Our work pioneers and opens a new
stack of event camera applications, as we publish the source code as well as
the first schlieren dataset with high-quality frame and event data.
https://github.com/tub-rip/event_based_bos

Computer Vision and Pattern Recognition;Image and Video Processing