PHYSICS BASED SIMULATION OF REALISTIC LIGHTING AND ILLUMINATION FOR NEXT GENERATION GROUND VEHICLE IMAGE GENERATORS
Over the past several years, the rate of advancements in modern computer hardware and graphics computing capabilities has increased exponentially and provided unprecedented opportunities within the Modeling and Simulation community to increase the visual fidelity and quality in new Image Generators (IGs) As a result, IG vendors are continuously reevaluating the best way to make use of these new performance improvements. Some vendors have chosen to increase the resolution of the environment by displaying higher resolution imagery from disk while other vendors have chosen to increase the number of polygons that are capable of being presented in the scene while maintaining 60Hz. While all of these approaches use the latest hardware technology to improve the quality of the simulated environment in the IG, the authors of this paper have chosen to focus on a different approach; to improve the accuracy and realism of the simulated environment. To accomplish this, the authors have developed capabilities that exploit the modern advances in hardware and graphics processing to mimic the human vision system and improve each pixel with enhanced atmospheric and illumination calculations improving the accuracy and realism of these pixels on the display's surface. INTRODUCTION Modern day computer hardware and graphics computing capabilities continue to increase at an exponential rate. As the hardware advances, Image Generator (IG) vendors are continuously reevaluating the best way to make use of these new performance improvements. Some vendors may choose to use hardware improvements to increase the number of polygons capable of being presented in the scene while maintaining 60Hz. Other vendors may choose to page more high resolution imagery from disk, and yet others, may use the hardware improvements to generate database content at run time. While all of these approaches to improving simulation fidelity and realism have their unique benefits (and drawbacks) the authors of this paper have chosen to focus on a different approach to improve realism and fidelity. The authors have chosen to focus on exploiting the modern advances in hardware and graphics processing to mimic the human system and improve the quality of each pixel with enhanced atmospheric and illumination calculations in order to improve the realism of these pixels and their final red, green, and blue luminance levels on the display's surface. The ultimate goal of the author's efforts is to improve the realism of visual and sensor simulations by trying to imitate the effects that our human visual system perceives in the real world environment and display them in a simulated environment. As such, the focus of this paper describes new capabilities that can be integrated into various vehicle driving simulations to immediately improve fidelity and realism. The goal of this paper is to present the improvements in accuracy and realism that can be obtained in visual and sensor simulations by imitating the effects our human visual system 'sees' in the real world and displaying them in a simulated environment. Our human vision system is capable of adapting to vastly different luminance levels (i.e. light levels) that can extend to a range of up to 14 orders of magnitude; yet, simulation display systems are only capable of producing an illumination range of approximately 3 orders of magnitude. Therefore, in order to mimic how our visual system Proceedings of the 2015 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS) PHYSICS BASED SIMULATION OF REALISTIC LIGHTING AND ILLUMINATION FOR NEXT GENERATION GROUND VEHICLE IMAGE GENERATORS Page 2 of 9 perceives the vast ranges of illumination levels in a simulation, physics based rendering techniques must be employed to emulate certain effects experienced in our human visual system (i.e. glare and adaption) while accounting for the limitations of the display. This involves the physics based modeling of light propagation in the ambient environment in High Dynamic Range (HDR) while also modeling the variations in human perception in order to view the results on Low Dynamic Range (LDR) displays. The prediction of visibility and appearance of scene features using our approach provides significant improvements in realism and enhances the overall quality of ground vehicle simulations. The resulting realism can significantly improve training when high contrast scene content is present and more accurately represent conditions experienced in the real world. This paper will show the results and improvements that can be obtained by mimicking various aspects of the human vision system in next generation IGs for ground simulations. In the real world, the human visual system is capable of adapting to vastly different luminance levels (i.e. light levels) that can extend to a range of up to 14 orders of magnitude [10] For instance, the sunlight on a clear summer day can be as much as 10 million times more intense than the illumination from the moon later that night [6] Figure 1 shows the large range of luminances in the environment and some associated visual parameters as an example. Therefore, in order to mimic how our visual system perceives the vast ranges of illumination levels in a simulation, it seems logical that we would replicate the exact same luminance levels in the simulation display system. However, while physics based rendering methods enable the accurate simulation and distribution of light energy rendered in day and night scenes, most simulation display systems are only capable of producing an illumination range of approximately 3 orders of magnitude (compared to the 14 orders of magnitude perceived by the human vision system) Therefore, accurate physics based rendering does not automatically guarantee the realistic visual appearance of a displayed scene. Additional rendering techniques are required to mimic certain effects experienced in our human visual system (i.e. glare and adaption) while accounting for the limitations of the display. Figure 1: Range of Luminances and Visual Parameters. After Ferwerda, et. al. [6] This paper discusses how the authors have improved the realism of visual simulation by emulating the adaptation of the human visual system to varying driving and vehicle conditions. This involves the physics based modeling of light propagation in the ambient environment in High Dynamic Range (HDR) while also modeling the variations in human perception in order to view the results on Low Dynamic Range (LDR) displays. Figure 2 below illustrates the night driving realism which can enhance the overall quality of training through the approach proposed here. Figure 2: Night driving. The proposed approach to physics based modeling of light will improve the realism and therefore training results of ground vehicle driving simulations under a wide variety of environments and conditions. BACKGROUND AND RELATED WORKS Understanding how the human visual system works and the mechanisms involved in the perception and adaptation to different illumination levels experienced in the real world lends support to the innovations presented here. The adaption of the human vision system is achieved through a combination of mechanical, photochemical, and neural processes in the visual system. The pupil, the rod and cone systems, and changes in neural processing play a major role in the adaptation mechanism. However, not all intensity levels are perceived equally well, and adaptation does not happen instantaneously, especially under the wide range of environments and conditions faced by ground vehicle assets. It is convenient to think of the human vision system as a complex 'biological sensor' comprised of rods and cones that adapts to ambient illumination with the three vision functions: Proceedings of the 2015 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS) PHYSICS BASED SIMULATION OF REALISTIC LIGHTING AND ILLUMINATION FOR NEXT GENERATION GROUND VEHICLE IMAGE GENERATORS Page 3 of 9 scotopic vision, that is active at a luminance range of approximately 10 6 to 10 2 cd/m2 mesopic vision, that is active at a range of ~10 2 to 1 cd/m2, and photopic vision, that is active at a range of ~1 to 106 cd/m2 [1] Figure 3 (below) illustrates how the spectral sensor response curves of rods and cones change for the scotopic, mesopic, and photopic vision functions. Knowing and simulating the state of this biological sensor is a key component in determining what an observer should be able to distinguish in the virtual scene under different simulated lighting conditions. Significant advantages in ground vehicle training can be achieved through emulating the scotopic, mesopic, and photopic vision functions, the differences in perception at different luminance levels, including the dynamic characteristics. One of the most important aspects in modern vehicle and driving simulations is the ability to accurately display realistic synthetic environments and scene content while maintaining real time rendering performance. For example, imagine being able to offer highly realistic weather or lighting conditions for different terrains, including non road terrains, faced by our ground vehicles in different areas of the world. Accurate modeling of natural illumination and how the atmosphere modulates and contributes to this has become increasingly possible. Some of the reasons why this is advantages are as follows: Realistic radiance values tend to produce expected, reproducible results. When 'tuning' a scene that starts with realworld values, it is easy to determine how much each "tweak" deviates from reality, which in turn helps determine the resulting numerical accuracy of the system. Less visual tricks are required to produce desired final results that appear and behave like the natural environment. Figure 3: Spectral Response Curves of Rods and Cones for a) Scotopic, b) Mesopic, and c) Photopic Vision [6] Co