In the cutting edge technology industry, you can never afford to stand still. Emerging requirements like complex visual content, increasing UI layers and 4K HDR video, are all pushing the bounds of what we can achieve in a Mainstream mobile form factor. This is why we’ve launched a brand new Mali Multimedia Suite featuring two graphics processors, a display processor and a video processor, expressly for the DTV and mainstream mobile markets.
Graphical complexities are largely the realm of the GPU, and you can read more about them in my GPU blog here, but the video and display processors have a huge part to play in overcoming the challenges of these emerging technologies and delivering a truly high quality visual experience to the user.
We’ve spoken a little about DTV previously, in the context of the GPU, but from a display perspective, today’s generations of Smart TV have really upped the ante. We used to simply watch the show we were watching and if a menu popped up, chances are it covered the whole picture and was plain text only; pretty simple really. That is not the case today. With a whole host of pre-integrated apps offering content of different types, complexities and viewing options, not to mention voice or gesture recognition user interfaces, things got a whole lot harder. Add in the need for picture in picture, and even the emerging requirements we’re seeing from the Chinese market in particular, where users expect to be able to see up to 16 video streams simultaneously in a 4x4 video wall, and we reach another level again.
Mali-D51 is the first Arm Mali display processor in the Mainstream roadmap to take advantage of the fantastic Komeda display architecture, launched last year with the high-performance Mali-D71. Designed expressly to deliver a high quality visual experience in a smaller silicon area, Mali-D51 is 30% smaller than Mali-D71. In Mainstream mobile devices it’s very unlikely you’d want to display a great deal of 4K content, which is really still the realm of the high-performance roadmap, but you still need to be able to handle it, so in the mainstream it’s all about making smart choices which might include downscaling the video stream from 4K for the target display resolution in question. Mali-D51 capitalises on this to provide superior quality at any resolution up to 2048x4096 pixels at 60fps, and across the same number of layers as the Mali-D71.
Mali-D51 achieves twice the scene complexity in the same area as a dual display implementation of the previous mainstream product, Mali-DP650. When driving a single display, it makes use of the otherwise underutilized resources of the secondary display to add a second set of four composition layers to the scene. In addition to which these layers can be rotated, alpha-blended and/or scaled. This constitutes double the number of layers as Mali-DP650 which means that we can support all that additional data required by UI overlays, picture in picture and more without interrupting the GPU.
Memory access is a constant requirement in display technology and Mali-D51 recognises this by achieving a 50% memory latency saving compared to its predecessor. Add in CoreLink MMU-600 and even greater benefits can be achieved across the entire system. In order to continuously drive pixels to the display panel you need to access the system bus regularly, halving the memory access latency means you only need to spend half the time on the bus in order to drive the images to the panel. This speeds up the entire display pipeline and vastly increases robustness against underrunning, providing seamless, real-time performance without any visible glitches or artefacts to compromise the user’s visual experience.
Our newly announced video processor, Mali-V52, is also set to address the growing requirements of the mainstream market. It’s predecessor, Mali-V61, was targeted at ultimate scalability, covering all device requirements from the smallest, lowest power security camera, to the latest, top of the line smartphone. With Mali-V52 we’ve got a little more specific. Rather than supporting such a vast array of use cases, the Mali-V52 hones in on specific efficiency gains most vital for the Mainstream market.
Scalable from 1-4 cores (as opposed to the previous 1-8), Mali-V52 specifically targets 4K content streaming for the more silicon restricted device and as such is almost 40% smaller than the Mali-V61 configuration required for the same performance. This is such a huge saving that it almost requires no further explanation, but I’ll give one anyway. Like Mali-D51, this new video processor achieves its incredible optimizations through a series of smart trade-offs, giving a huge range of options and choices for all of our partners. For starters, we’ve doubled decode performance per core, which means that in the same area you can perform 4k60 decode or 4k30 encode. With a single core of Mali-V61 you could decode 1080p60, whereas a single core of Mali-V52 can support 4k30 or 1080p120, as the core has been designed to perform twice as fast for the main codecs such as HEVC, H.264 and VP9.
Mali-V52 has been designed to ensure an optimized overall system cost and power. One such example is its ability to tolerate a longer system bus response time without dropping any performance. With Mali-V52 there is a guarantee that no frames will ever be dropped for a fixed delay of 400 cycles on the bus, double that of the Mali-V61. In addition, a huge amount of effort has been spent to ensure resilience to latency peaks of up to 5000 cycles which allows the video processor to go offline for many clock cycles giving way to higher priority masters such as the display.
Mali-V52 can also achieve 20% lower bitrate and still have same quality as before, which is very useful in making significant savings on storage and achieving the highest possible quality in the same bandwidth. In devices where we often pay more for more storage, this makes a big difference to the end user’s video capabilities. You can select specific quality levels and balance them against your priorities for storage and power consumption, which is why it’s so important to have a dedicated video processor in mobile devices rather than letting all that hard work fall to an already overworked CPU.
One of the specific use cases we’re seeing is driven, as so many up and coming technologies are, by the Chinese market. It’s not enough to provide a simple picture in picture anymore, so you can preview what you might like to watch, Chinese users expect to be able to select their programming content from up to 16 actively streaming channels. This 4x4 video wall approach is a requirement from many of our Chinese Set Top Box and DTV partners and is no longer a nice to have but a definitive future requirement and one we had to address with this range of IP.
This is where the individual processor takes a backseat though, and the entire Mali Multimedia Suite comes into play…
By combining two or more of the Mali Multimedia Suite processors our partners can attain much greater efficiencies than with standalone products. Arm supports a full software stack to enable hardware composer functionality on the display, allowing the Android OS to look at the hardware capabilities of the system and infer that it can send the composition there instead of the GPU – it spots the hardware and automatically engages with it via the software to save engineers even having to consider it.
For the video wall use case discussed above, Mali-V52 can decode the 16 videos simultaneously, all at HD and whilst writing them out to the display processor as a single frame. Mali-D51 is then able to take that single frame and send it direct to the panel without any additional bottlenecking or slow-down, due to its superior memory interface.
The real benefit this brings to the overall system is in the sheer amount of effort it saves the GPU. If you were to perform rotation, scaling, gamma correction etc., on the GPU, your system would be using 30% more power, a big problem in the constraints of the mobile device. This doesn’t prevent you from consuming content created in 4K though, the brilliance of the Mali Multimedia Suite of IP is that the new video processor, Mali-V52, can take 4K content and scale it down directly before sending it to the display. This means those great new 4K shows released by the likes of Netflix and Amazon are still attainable across all consumer devices. It’s not just 4K though, Mali-V52 supports 10bit HDR content and works perfectly with Mali-D51 and Assertive Display 5 to display this content optimally across any type of panel, be it HDR or SDR.
From DTV to smartphones, the Mali Multimedia Suite of Graphics, Video and Display works better together to attain the highest quality visual experience for the user, whatever their budget.
Learn more about the Mali-D51
Learn more about the Mali-V52