1920×1080 has been the standard resolution for gaming, media and productivity for years now – and even longer if you had the budget when HD displays were less commonplace. Likewise, 4K is poised to claim the throne of ubiquity.
At the moment, 4K screens are still outside of the price range of most users. What’s more, the technology to drive those displays can be even more expensive. The best graphics cards on today’s market struggle to maintain a constant 60fps at 4K resolutions, the entry-level benchmark for acceptable framerates on a modern PC.
That benchmark is raised further when considering virtual reality. To reduce the delay between frames and thus any sense of motion blur or potential nausea-inducing lag, one must increase the framerate. The panels inside both the HTC Vive and Oculus rift both have a refresh rate of 90 Hz, and demand your system be able to push framerates to match for the best possible experience. Those headsets currently only sport 1080p panels, however.
This is because HTC and Oculus both understand the points above: a high framerate is critical to an optimal experience. Cost is obviously the most significant factor, and 1080p panels here are a strategic decision: HTC and Oculus also both understand that potential customers are more likely to have that optimal experience and grow their products (and the industry as a whole) if their computers can run the games they’re trying to play, and they can afford the entry cost.
Software can also help bridge the gap between optimal experience and hardware capability. In virtual reality, the renderer can use technology called “Asynchronous Reprojection” (or Asynchronous Spacewarp to Rift users). When a game doesn’t hit that 90FPS requirement, the system halves it down to 45FPS and the frame that was previously rendered is injected in place of the dropped frame, reducing the overall judder and maintaining a smooth experience.
In some games, this works like a charm. Elite: Dangerous is a terrific example – as a seated game, you’re less prone to experiencing any artifacts or motion peculiarities if the system is periodically (or constantly) running at 45 FPS. With a lower acceptable benchmark, players can crank the in-game graphics and the supersampling (rendering the game at a higher resolution than the screen to sharpen the final image) a bit more without hugely compromising the experience elsewhere.
Virtual reality headsets themselves are the combination of several different technologies. At the top of the list are HD displays in small form factors (their creation and relatively low cost driven initially by the smartphone industry), 3D physical tracking and wireless motion controls (remember the Wii?).
As HMDs evolve, additional technologies are being integrated into them. The next generation of VR HMDs will look to feature both 4K displays over their 1080p predecessors, and include eye-tracking functionality to track what you’re focusing on. This has some interesting benefits.
Adaptive resolution has existed for some time now. Similar to asynchronous reprojection, adaptive resolution looks to maintain a lag-free experience by dynamically adjusting the output resolution of the rendered image to match a desired framerate, lowering it where the computer might otherwise drop frames and raising it when extra GPU resources are available. These types of optimizations are the kinds of things computers have been doing for ages, with things like level of detail scaling. GPUs now make these changes on the fly without significant hits to performance in other areas.
By effectively combining this idea with eye-tracking technology, we get to something called “Foveated Rendering.” Foveated rendering essentially sharpens the image at the point your individual eyeballs are focused on, and then gradually reduces the resolution outside of that point to reclaim GPU cycles. Since you can’t make out the finer details in what’s in your peripheral view – such as the difference in a 720p crate in the corner of a room and a 4K crate, you’re none the wiser to the loss in image quality.
Eye tracking, like all of the other technologies that made virtual reality possible, is now affordable enough to be integrated into HMDs without massive hikes in cost. They’re also fast enough to keep up with rapid eye movements and apply that information towards a better framerate. When considering the challenges behind maintaining 90fps at 4K, foveated rendering is another way to gain a performance advantage, and may become a bare-minimum requisite for future VR titles and the HMDs they will be played on.
