There are many ways to tell a person apart: fingerprints, iris scans, and Aadhar cards. Yet there is nothing like a face. During the Covid-19 lockdowns, India’s Defense Research and Development Organization (DRDO) developed a facial recognition system that could identify individuals even if they are wearing a mask. But with great power comes great responsibility. How does this system work and how should the state use it?

Facial recognition technology (FRT) is one of the foundational problems of artificial intelligence. Over time, there have been significant advancements in FRT hardware. Equipment like cameras and computers now pack more in the exact dimensions. This means that more powerful computers can be placed directly in the hands of a user with a powerful camera.

The natural beauty of FRT is its black magic software. Computer vision algorithms have been developed over time to digitize an image, recognize the face in that image, and then try to match it against a database of other fronts. While it took decades to design and test, today, the software is blazing fast. 

FRT works because every pixel of a photo corresponds to a number that conveys its color. What FRT algorithms do is take these values and try to find the contours of the face. The facial contours are consistent in color for the majority of people. Then, this outline is used to confine the next set of algorithms that generate “features.” These features are meant to be unique and notable clusters in an image. Given that a face will remain consistent over time, these features can be relied on to increase matching accuracy.

FRT is not a single algorithm. Instead, it is many different techniques and algorithms working to extract unique and identifiable information about your face. The best example is the phone that offers facial unlocking. The phone does not run your face across an extensive public database. Instead, it captures your face during setup to build a database and then uses those data points. The technique discussed above is one of many ways researchers have developed cutting-edge FRT. But advancements in FRT have not been restricted to images alone.

Live feeds from high-resolution public surveillance systems are being fed into sophisticated software in real-time. This software cannot only identify individuals in a dense crowd but also track their behavior. Such software claims to be able to issue an alert in case of unusual crowding or if an altercation occurs. There is a separate protest detection feature. Various private sector startups invest in advanced and nuanced analytics as part of their FRT suite. One example is Innefu Labs, based in New Delhi, which deploys gait and body analysis. Hence, it makes it harder for a malicious entity to escape the all-seeing eye because they may cover their face or disguise themselves in another way but cannot change how they walk. Even if they do that, more nuanced parameters will build every individual's unique profile.

It is not that these analyses work in isolation. Quite the contrary, they work in tandem to isolate a match. If someone goes through the list of website visitors, there will be millions of people. But they can reduce the number by looking for users from a specific region. Still, it is impossible to track anyone specific. However, suppose the system also follows which device the user uses when they log on to the website. In that case, it creates a unique set of data points that can be used to identify a specific person next time they are on the website. Similar strategies are now being used to build an FRT with such advanced features.

Not just a dystopian tool, the FRT industry is worth almost $4 billion as of 2020. And it will most likely grow over fifteen percent by 2028. Many governments use FRT on a large scale and private entities have even implemented such surveillance tactics, with over 70 percent of hotels likely to have some form of FRT software by 2025.

Protecting Personal Privacy

We have maintained that the face, as a biometric identification marker, is unique compared to other biometrics. The most crucial differentiation, however, is the power of consent. Usually, the power of approval to provide an iris scan and fingerprints is actionable. Unless under duress, consent can be freely exercised. But as far as FRT is concerned, consent is abstract and moot. If you have been at an airport, your face has been run through an FRT system. Because we cannot fly without our ID, your face and personal identification information are now part of an extensive database. You may not consent to it but the system has done the deed.

This is the problem of the policy concerning FRT. Another peculiarity is the use of FRT in phones and laptops. Many new applications use this to increase their attractiveness to the consumer. One example is LensKart, which allows users to scan their faces and try digital renders of a frame they might want to buy. Their privacy policy underlines how they store photos and scans to map and measure your facial features. They maintain that no third party can lay their hands on this data. In the private sector, consent concerning facial recognition is more prevalent as they are not forcing one to use such features. But public sector deployment is much more intrusive.

In a 2020 study, about 70 percent of Americans felt that FRT makes society safer. There is some truth to this sentiment. In 2018, a person was arrested using FRT at Dulles International Airport. During the anti-Citizenship Amendment Act protests in New Delhi, the police used a facial recognition system to track suspects. There was much furor in Indian courts recently, given that the original purpose of the FRT in the hands of the New Delhi police was to follow missing children. Consequently, various protests that have erupted in the capital since then have involved arrests using FRT systems.

Outside of urban settings, FRT is also crucial for border security and military bases to detect intrusions, control access based on clearance, and monitor delinquents.

Currently, India is bullish on deploying FRT. Outside of China, it is fast becoming the country with the most surveilled cities. In addition to developing a national database, many states have announced plans to develop a comprehensive database of biometrics.

FRT is a double-edged sword in the hands of the state. It is a favored tool but the reputation of the police shapes the trust of the public. It would be a gross miscalculation to rule out the use of such technologies entirely. And given its role in promoting national security, it is unlikely to happen. However, responsible policing will ensure greater public trust.

Girish Linganna is a Defense & Aerospace analyst and is the Director of ADD Engineering Components (India) Pvt Ltd, a subsidiary of ADD Engineering GmbH.

Image: Reuters.