Implementing Power Efficient Facial Detection and Recognition with ARC Processors

Introduction

Facial recognition – the challenge of identifying and verifying faces in photographs – is a simple problem for humans but a complex problem for computers. After sixty years of research, recent deep learning advancements enable widespread use of face recognition technology across a broad range of applications. Access control for your battery powered device? Authenticating users in banking applications? Replacing the need for car keys in your automobile? More personalized patient care at hospitals? All use cases that could benefit from facial recognition. Implementing facial recognition in edge device applications will require efficient, low power processors with dedicated support for the latest artificial intelligence (AI) algorithms.

Because computerized facial recognition measures a human’s unique physical characteristics, facial recognition is a type of biometrics. Accuracy can be difficult because of the variability of human faces. Any image captured could include variations in expression, position, pose, orientation, or lighting conditions. Individual faces could also vary with skin color, differences in facial hair, or inclusion of eyewear. Still, facial recognition is easier to deploy and implement than other biometric techniques like iris or voice recognition. And unlike fingerprint recognition, facial recognition does not require any physical interaction by the end-user.

The Steps of Facial Recognition

For a computer, facial recognition can be broken down into different steps (Figure 1). Face detection locates one or more faces in the image. Face normalization aligns the captured face to be consistent with the faces already stored in the database. Feature extraction is used to extract and measure the distance between key features, such as the eyes or points on the lips, for pattern matching. Finally, face matching is the last step in recognition where the extracted features of the input face are compared against the extracted features of the faces in the database.

History of Facial Recognition

Facial recognition techniques appeared as early as the 1960s when Woodrow (Woody) Bledsoe, a mathematician and computer scientist, first applied computers to the challenge of recognizing human faces. Working with Helen Chan and Charles Bisson at Panoramic Research, Inc (PRI), Bledsoe helped develop a system to manually extract and record the coordinate locations of various facial features (eyes, nose, hairline, mouth, etc.) from a batch of photos (such as a book of mugshots). The metrics were then input into a computer database. Given a new photo, the database determined which original photo most closely resembled the new photo. This manual approach to capturing facial characteristics had its limitations, but Bledsoe continued his work in pattern matching and automated reasoning and is considered one of the founders of AI.

In 1991, Matthew Turk and Alex Petland, from MIT’s Media Laboratory Vision and Modeling Group, had a breakthrough when – applying algorithms based on linear algebra – they were the first to use computers to automatically extract facial feature from photos. By 2001, Paul Viola of Mitsubishi Electric Research Labs and Michael Jones of Compaq released a paper detailing their rapid object detection algorithm – dubbed Viola-Jones. Used as a real-time face detector, Viola-Jones required full view frontal upright faces but could detect faces in real time with high accuracy. It became the standard for facial detection for many years until it was overtaken by deep learning techniques.  

The advent of deep learning techniques was a game changer for facial recognition. Trained neural networks (NNs) provided a significant jump in accuracy over programmed algorithms like Viola-Jones. In 2014, Facebook released Deepface, a nine-layer neural network with over 120 million connection weights (or coefficients) that have been trained by Facebook users using four million uploaded images (Figure 2). Deepface had 97.25% accuracy. In 2015, Google researchers developed FaceNet with an accuracy of 99.63%. FaceNet is really more than one neural network graph. It is based on at least two convolutional neural networks (CNNs).

Facial Recognition Moving from the Cloud to the Edge

Both DeepFace and FaceNet, with their large coefficients, are best suited to online, non-real-time applications – like sorting pictures in the cloud. Large coefficients require large memories, a significant amount of computations and a lot of data movement – all making cloud-based real-time low power implementations difficult to impossible. As market interests for facial recognition shift to low power implementations on edge devices such as smart phones, consumer cameras or laptops, performance limitations required more efficient neural networks and power efficient neural network engines.

Another strong reason for the move from cloud based facial recognition to edge devices is individual privacy. Face recognition – especially in surveillance use cases – can passively capture faces. Recording faces of unsuspecting people and sending unique identifying information to a central/cloud database is causing privacy concerns for many consumers. One advantage of facial recognition done entirely at the network edge is that the computations are done offline instead of using cloud-based services.

Battery-Powered, Always-On Facial Recognition

Although there are many low-power edge device facial recognition applications, perhaps the most difficult to achieve is an always-on facial recognition system providing access control to a battery-operated device like a laptop, tablet or cellphone. A user expects hands free and instantaneous facial recognition. This means the electronics needs to be constantly looking for a face. One of the best solutions is to divide the facial recognition tasks across two processors.

Small microcontrollers with DSP capabilities – such as the Synopsys DesignWare® ARC® EM9D processor IP family – excel at always-on performance for AIoT applications. While small AI-enabled microcontrollers don’t have the horsepower to handle the full facial recognition neural network tasks, they can detect if a face is present and then wake up a dedicated, power-efficient neural network accelerator, such as in the ARC EV processor IP,  to complete the recognition steps.

Facial detection for access control where the input image is prominent and close is a fairly simple AI task that can be performed with minimal input image size and with a neural network graph with minimal computational requirements. Keeping the graph complexity and the number of coefficients low are extremely important to minimizing power and to fitting on a small microcontroller.

Facial Detection with ARC EM9D Processors

The facial detection graph in Figure 3 was developed in collaboration with Technical University of Eindhoven. Captured input images are reduced to 36x36 pixel gray scale images, and four layers and only 1k model parameters are needed. Facial detection applications like this can be run with minimal power consumption on the ARC EM9D processor. The ARC EM9D has a zero latency XY architecture for multiply-accumulator (MAC)-intensive CNN algorithms and can perform the facial detection graph while only consuming µWs of power. To accelerate design closure, the ARC EM9D processor is supported by a highly optimized embARC MLI library that supports a wide range of low/mid-end machine learning applications like always-on facial detection.