HTC Vive Pro Eye - DARBEE Visual Presence Benefits For VR Systems With Eye Tracking And Foveated Processing

January 23, 2019

DARBEE Visual Presence (DVP) helps VR a lot but is considered by some to be computationally expensive for application in VR/HMD systems. With the development of good eye tracking and foveated view processing, DVP can be as much as 5X less computationally expensive.

In the past, DVP had to process the entire field of view for VR imagery, because there was no way to leverage a valuable missing piece of human vision processing, "foveation."

"Foveated" means that the center of one's visual field is more focused (has greater resolution) than the surrounding region. The reality is, human eyes can only focus in at the center portion (foveated region) of a focal depth plane. It is impossible for a human visual system to focus on an area that is not in the central foveated region. 

All image capture, since the beginning photography, creates an image that has a focal region that is unlike how we see with our eyes. A camera focuses at a focus depth, with pretty much all of the spatial information at that plane being in focus. The result is that all parts of the image at that plane (including objects at the periphery) will always be in focus, which is fundamentally unnatural looking (we have gotten used to this appearance). This will feel unnatural particularly with VR applications. 

Now we can use computation to create a more realistic and dynamic real time foveated region of a digital image, and this form of processing is now being leveraged by VR/HMD systems.

In order to compute a dynamic foveated field of view, the system needs to know exactly where your eyes are pointing and focusing. If a system can track your eyes and know where you are focusing your eyes, then it can leverage significant benefits. [The video (link below) describes the benefits] Therefore, fast and accurate (real time) eye tracking is critical to foveated image processing and the success of VR/HMDs in general.

Since the introduction of the Oculus Rift, about 6 years ago, DarbeeVision has been waiting for these two features to be developed and integrated, to specifically benefit VR headset systems.

1. Eye Tracking - VR will never gain traction with the masses, if eye-tracking is not real time and accurate. It appears that the Vive system may have crossed a threshold in both of these areas.
2. Foveated Processing for the Field of View - Non-foveated viewports waste image processing and rendering resources, because the eyes can only fully focus on about 10-15% of the image (the center of the field of view). All other parts of the image do not need to be rendered with as good quality. With good eye-tracking, the fovea region can be rendered in full resolution (with DVP), while the surround region is rendered at a much lower quality (without DVP).

Short Video about New HTC Vive Pro Eye system: https://youtu.be/eyaI9l9ydEY

Some of the ramifications for DARBEE Visual Presence application are:

• DVP is applied ONLY to the most important part of the image, the foveated region, where the realism is fully appreciated.
• DVP can be ~80% less computationally expensive than it would traditionally be (if 100% of the frame is DVP processed).
• DVP can be embedded into hardware solutions with a much lighter computational resource requirement. 
• DVP for foveated VR applications may cost little enough that the processing could be implemented as software.

Another background resource.

https://www.arris.com/globalassets/resources/white-papers/exec-paper_virtual-and-augmented-reality_final.pdf

We look forward to being able to work with strategic partners to determine how eye-tracking and foveated-view processing will harmonize with how DVP may be applied to VR/HMD, with less expense and for greater harmony to how the human visual system works.   

Larry  Pace

President

DarbeeVision Inc.
Orange, California

Direct (714) 931-5941

www.DarbeeVision.com