Face verification has long been considered one of the greatest challenges in the field of computer vision, so we're proud to have designed a face verification system that achieves 98.36% accuracy on the Labeled Faces in the Wild database. It’s a real-time proof of concept, running efficiently on Arm machine learning (ML)-optimized platforms.
Now, depending on how much you know about machine learning, you might have read that percentage with amazement, cynicism or incredulity. Whilst it’s undeniably accurate, it’s also undeniable that the kind of figures that can be achieved in lab-based experiments are often quite different from real-life scenarios. The successful deployment of each solution relies on much more than the accuracy of a single metric – which is why we set out to gain a better understanding of the challenges end users encounter as they develop their machine learning solutions on Arm platforms, moving from lab-based theory to real-world deployment. We also wanted to explore the practical limitations of neural networks after their initial training, once they’re deployed in the field.
We chose a face unlock (face verification) system, as it requires a surprising range of features from the software released by Arm and uses multiple hardware IPs already in production. We were pleased to find that there were already great hardware solutions available – for the problems of always-on, as well as for processing of the initial vision stages of the pipeline – and that the GPU on a mid- or high-end mobile platform was sufficient for our needs. We also found that the capacity of the Arm ML processor provided the throughput necessary to provide a premium experience on high-end phones.
As expected, there were a number of subtle issues that crept into the reality of deployment, from dealing with different lighting conditions to building reliable data sets for less common research problems such as spoof detection – not to mention wading through the large number of network versions that researchers have developed, each with different choices and training options.
Interestingly, for some issues we discovered that tried-and-tested solutions were more effective that shiny, new techniques: there were multiple instances where robust implementations of classic computer vision or pre-deep-learning ML techniques proved more than sufficient, and with lower complexity. While these stages will probably be overtaken by deep learning at some point, it’s interesting to observe the varying pace of progress in different areas of the problem space.
The pipeline design we selected – which combines classic and modern machine learning (deep learning) – supports key functionality multi-user verification, as well as an anti-spoofing stage, to address key security issues around spoofing a user's identity using a photo or video. It also uses multiple IPs, to discover how this approach can help the user experience.
Our white paper, Deploying Always-On Face Unlock: Integrating Face Identification, Anti-Spoofing, and Low-Power Wakeup – which you can download below – documents the project, exploring practical solutions to the issues we encountered as well as posing some interesting open questions to anyone looking to move this initial work forward. It also illustrates how straightforward and efficient the deployment of machine learning is using Project Trillium, Arm’s machine learning platform. I hope you enjoy reading it!
[CTAToken URL = "https://pages.arm.com/machine-learning-face-unlock-whitepaper.html" target="_blank" text="Download whitepaper: Deploying Always-On Face Unlock" class ="green"]