MetaVR Sensor Simulation: Physics-Based IR and Notional Sensor Features
MetaVR Virtual Reality Scene Generator™ (VRSG™) provides thermal sensor imagery that portrays the environment of a simulated device with true perspective and real-time responses to the movement of a vehicle and its sensor. Thermal sensors portray relative temperature differences in a given scene and are therefore able to provide information that is not available in the visible spectrum (that is, not available in the "out-the-window" view). The contrast in temperatures aids users in identifying and classifying the features and entities depicted in the image.
VRSG features significant enhancements to the simulation of an infrared (IR) sensor. Working jointly with Technology Service Corporation (TSC), MetaVR developed an advanced physics-based IR sensor modeling capability, which is available in the domestic release of VRSG. (Note: International VRSG users require ITAR approval to use this feature.)
The VRSG sensor view is a user controllable image that models two levels of thermal views:
VRSG's physics-based IR
The key feature of VRSG's physics-based IR capability is the real-time computation of the IR sensor image directly from the visual scene, with no off-line precomputation or data storage requirements. This real-time model combines automatic material classification of visual RGB imagery, and a physics-based IR radiance and sensor model.
As the resolution of the imagery increases so does the resulting sensor simulation. The following image shows the thermal view of terrain compiled at 2.5 cm resolution using 2.5 cm imagery collected by MetaVR's imagery collection small UAV:
Next is a VRSG sensor image of a solid particle explosion and resulting crater on 2 cm per-pixel resolution geospecific terrain of Yuma Proving Ground, AZ.
VRSG is capable of rendering vast geographic expanses of high-resolution geospecific imagery (such as CONUS++ 3D terrain at 1-meter color resolution). Material classified representations of such large and high-resolution areas are generally not available nor are they practical to acquire. The technology developed by MetaVR and TSC solves the classification problem. The process uses pixel-shader technology to convert the per-pixel filtered visual spectrum RGB color into its component materials, which are then used by the physics-based IR model to compute IR radiance and sensor display intensity. The physics-based model takes full account of the local environment, time of day, and sensor characteristics. The result is a physically accurate sensor scene derived from a visual-spectrum database.
The VRSG images above show a scene on MetaVR's Afghan village 3D terrain at midnight and in the early afternoon. Notice in the scene on the left the warmer vegetation and the cooler asphalt road and concrete structures. The cooler 3D structures are difficult to discern as they have cooled to an even temperature and blend into the background. In contrast, the image on the right shows the same area in the afternoon, after the sun has heated up the asphalt road and the stucco sides of the buildings. The vegetation is cooler than the concrete and asphalt structures heated by the sun. This example illustrates thermal inversion.
The only offline processing requires the user to create a classification palette for supervised material classification by associating up to 5 RGB colors with physical materials from the VRSG/RealIR material database. Using the per-pixel material properties and the real-time parameters derived from environmental conditions, a per-pixel radiance raster image is computed for each video frame, as would be seen by an actual thermal sensor.
This radiance image is processed in real-time by including blurring, noise and Automatic Gain Control (AGC). The AGC process maps the radiance range into a display dynamic range, and can be disabled by the user if manual level and gain controls are to be used. The AGC process uses a histogram analysis of the radiance image to determine an appropriate display dynamic range for the scene. (A more efficient implementation of AGC has been implemented in VRSG version 6, which takes advantage of DirectX 11 compute shaders.) Sensor noise is automatically computed as a function of the display’s dynamic range.
Geotypical content, such as 3D culture and moving objects, can be directly mapped to material codes using an external file that associates texture names with material codes. The geospecific and geotypical content use the same underlying physics model, resulting in a consistent rendering of the scene.
VRSG’s notional sensor-view features
VRSG’s notional IR-based simulation requires no material code mapping and offers these capabilities:
The military vehicle models that are delivered in MetaVR’s 3D content libraries have built-in IR textures that represent heat signatures or hot spots that respond to some event or other stimulation. As each hot spot is unique, each can be stimulated independently for a thermal response to some vehicle activity.
VRSG's physics-based IR feature is available in the U.S. domestic release of VRSG. Please contact MetaVR for an updated license unlock code to enable this feature.