Augmented reality and virtual reality headsets are proliferating, but now there’s a system where the same tiny displays can be used as cameras to track eye-movement. Fraunhofer FEP has cooked up a new headset building on its latest bidirectional OLED microdisplays which, in addition to OLED pixels, have photodiodes sandwiched in-between. With those, the tiny screens can not only show data to the wearer, but watch where that person is looking too.
It’s something Fraunhofer has been working on for some time. Indeed, the company showed off an early version of the technology all the way back in 2012, albeit with just 640 x 480 resolution and monochrome pixels. That would have been sufficient for basic head-up display purposes, like notifications and navigation instructions, but far short of what immersive AR and VR would require.
This new set hasn’t moved the needle on resolution massively, mind, though it does switch to color. The bidirectional micro displays each run at 800 x 600 resolution, in fact, though Fraunhofer says that it’s really a starting point for commercial brainstorming. The company will show the headset off at awe Europe, with plans to talk about more compact versions and higher resolution systems.
While there are certainly systems out there today which can add eye-tracking to a head-mounted display, they rely on separate cameras within the headset. Perhaps best-known is FOVE, a crowdfunding success subsequently getting the attention of Samsung in 2015, though Oculus acquired an eye-tracking startup of its own late last year.
There are several reasons why you might want eye-tracking in a headset, for both VR and AR. Most obvious is hands-free control: if you know where the user is looking, you can select on-screen graphics in both UIs and within games and experiences. That reduces the amount of scrolling and joystick-waving you have to do.
However, potentially more important is foveated rendering. That effectively prioritizing graphics potency for the areas that the wearer is looking at directly, whereas the periphery is allowed to run at a less system-intensive resolution. Given the nature of the human eye, it gives the impression that the whole scene is at a higher level of detail.
Foveated rendering has big implications for wearable displays, not least mobile ones. Driving high-resolution HMDs is graphics-intensive and, while tethered systems can rely on beefy desktop GPUs, wireless versions powered by smartphones – or the processors more commonly found inside smartphones – can’t afford the battery and heat impact of that. By concentrating what performance is available on just the areas the user is focusing on, a more power-efficient balance can be struck.
For the moment, Fraunhofer is talking more about how “data glasses” like these could be used in professional settings, rather than for gaming and entertainment. For instance, designers, surgeons, first-responders, and mechanics could all benefit from a totally hands-free way of interacting with a system that overlays digital information over their real-world perspective. The prototype headset, which for the first time supports both AR and VR use, hooks up via regular HDMI and USB connections for flexibility, and supports different graphics being delivered to each eye for 3D purposes.
This article was originally published by Slash Gear.