In May of each year SoftBank-owned Arm regularly announces a suite of intellectual property (IP) designed for premium mobile phones, tablets and laptops usually shipping in the early part of the following year. Last year's Cortex-A76 CPU, Mali-G76 GPU and Mali-V76 VPU took the headlines, with the HiSilicon Kirin 980 system-on-chip (SoC) - powering an array of high-end Huawei and Honor phones such as the P30/Pro, Mate 20/Pro, Magic 2 - moving quickly from Arm design to actual product.
Receiving lesser attention is the display processing IP that helps offload a bunch of work from the GPU. Its main purpose in premium smartphones is to handle multi-layer composition, scaling, and high-dynamic range support. The GPU can do all of these operations, of course, but having a fixed-function block with an optimised path to memory reduces total SoC power by as much as 30 per cent. Up until today, the best DPU IP rests with the Mali-D71 processor, capable of 4K120 display.
Which brings us on to today's announcement of the Mali-D77.
 |
 |
On first glance there's not a whole heap of difference between the Mali-D71 and Mali-D77. Both are based on the Komeda architecture and carry the same general composition and scaling specification. What's new here, and it's an important distinction, is that Mali-D77, though naturally capable, is primed for premium VR displays.
How? Notice the two yellow rectangles in the D77's high-level block diagram. These introduce two features - asynchronous time warp (ATW) and lens correction - that are pivotal for smooth VR displays. Let's take each in turn.
As you may know from desktop GPU architectures, ATW is a method of reducing the latency between when the GPU spits out the last rendered frame, usually at consistent V-Sync intervals, and the exact location of your head at the current time. The GPU frame is usually just a bit behind, because it is normally unable to compensate for post-frame head motion. This latency effect, where everything feels a mite out of sync, is exacerbated on a head-mounted VR unit, giving rise to a queasy feeling for some.
ATW runs on a parallel thread and calculates the small change in head position between the last V-Sync and the exact current time. The sum of these is added just before the new V-Sync, helping keep good correlation between head movement and what you see on the screen. Put simply, VR doesn't work well without ATW.
VR headsets use lenses rather than just the panel's display for a couple of reasons: increasing the field of view and, secondly, enabling the image to be closer to the eye. That raises a problem, however, because running a lens introduces a barrel distortion effect that, unless corrected, gives the user a pincushion-like image. A bunch of maths is needed to modify the panel's image so that it looks right after going through the lens, even though this causes the original panel image to look wrong, hence lens-correction support.
Running lens correction is painfully expensive on the GPU alone because of the sheer transformation required for high-resolution displays. The number of texture lookups for a 2,560x1,440 display, for each eye, at 90Hz, is staggering. But because the lens type is already known per HMD, insofar as it doesn't change, a significant amount of pre-distortion can be calculated to reduce the load.
Other types of lens-distortion processing for VR HMDs are needed, too, because by their very nature lenses cause what is known as chromatic aberration. Mali-D77 incorporates chromatic aberration control (CAC) to mitigate the effect and produce a proper-looking image.
 |
 |
The answer to why Arm has gone to this trouble is simple: power-saving. The GPU is less efficient at such work than a dedicated DPU, with memory bandwidth being of key concern. Having a VR-capable DPU takes a chunk of the additional load off, freeing up the GPU to do what it does best - churn out high framerates.
Let's be clear. Anyone developing a premium IP without the need for explicit VR capabilities doesn't need the Mali-D77; the present Mali-D71 is more than capable. Arm's decision to release this specific variant speaks volumes about its confidence in future VR HMDs. From what we have seen thus far, the market hasn't embraced VR to the level predicted a couple of years ago, likely the timeframe when Mali-D77 was being planned. Yet, if you don't build it, they won't come...
