This weekend at Virtual Reality LA 2016 in Southern California, the company demonstrated a revision of its Light Field VR prototype unit attached to a desktop PC equipped with a Maxwell-based GPU. The algorithm Nvidia used to compute all the light field images in real-time is based on its CUDA parallel programming language for GPUs.
Nvidia Light Field Stereoscope Prototype (via Forbes.com)
With today’s VR headsets, a user’s eyes are more focused on the displays rather than the actual image. This results in terrible eye strain over long periods, wearer fatigue, and most commonly, motion sickness. As we mentioned in our HTC Vive hands-on demonstration, today’s head-mounted displays (HMDs) often require the use of headphones to help alleviate much of the “seasickness” effect that plagues any VR environment. We have been told this has something to do with the wearer’s visual movements not matching the audio signals that he or she is receiving in the real world, and wearing headphones usually compensates for this sensory signaling mismatch.
Nvidia Light Field Stereoscope Head-Mounted Display Schematic (via Forbes.com)
The Nvidia and Stanford project appears to take things to the next step, however, by completely eliminating the headphone requirement with a breakthrough VR headset design that produces no seasickness side effects. The prototype “Light Field stereoscope” unit uses two layered displays with back-to-back LCD panels separated by 5mm. These are sandwiched between a backlight on the far side of the viewer and the two circular viewing lenses near the viewer's eyes.
“The way we perceive the natural world is much more complex than stereoscopic,” Gordon Wetzstein, an assistant professor of electrical engineering at Stanford, said in an interview.” Our eyes can focus at different distances. Even one eye can see in 3D. It does that by focusing the eye.”
Wikipedia - A light field's radiance as measured by "L"
The Stanford and Nvidia prototype wearable unit creates a natural depth-of-field in each eye and combines it with a stereoscopic technique of showing images at slightly different angles in each eye. During SIGGRAPH 2013, Nvidia showed an early Light Field display prototype with a pair of Sony ECX332A OLED micro displays installed on a glass-like frame. Each panel measures 15.36mm x 8.64mm (0.60 x 0.34 inches), roughly equating to about 83 pixels per millimeter. The displays each have a resolution of 1024x720 with 24-bit color pixels with 97 percent NTSC color gamut, a brightness of 200 cd/m2, and have an ultra-low response time of just 0.01ms.
Nvidia Head-Mounted Near-Eye Light Field Display Prototype (November 2013)
To start, a microlens array splits each image into individual light rays, similar to the recently-launched Lytro ILLUM plenoptic camera. The array captures information about light intensity in a given scene along with the direction that light rays are traveling in space. The end result is that the depth-of-field distance captured in each image appears to be much more three-dimensional, making the virtual environment much more “natural” on the eyes and minimizing motion sickness.
Source: Stanford University at SIGGRAPH 2015
Perhaps the most fascinating part of Light Field VR is that it allows the wearer’s eyes to more naturally respond and rest at focal cues. This reduces fatigue and more naturally replicates how we perceive the world in real life.
Back in November 2013, Nvidia researchers Dr. Douglas Lanman and Dr. David Luebke published a paper illustrating the company’s prototype near-eye Light Field display complete with head tracking, so users can see their hands within virtual environments.
Nvidia Light Field Research
They concluded that current consumer VR headsets present two major restrictions – the first is that they have narrow field-of-view displays located in the periphery of a viewer’s visual field – and the second is that they have rather bulky designs with tight head straps. Nvidia wants to change all of this with its microlens display system, as it will simultaneously enable a wider field-of-view and provide more compact eyewear to the user for a more comfortable viewing experience.
Stanford University - The Light Field Stereoscope (SIGGRAPH 2015)
Light Field, or “plenoptic” virtual reality technology, is expected to be deployed in VR headsets in three to five years from now.