All around us we are seeing a surge in the adoption of facial recognition technologies (FRTs) -- biometric systems that can be used to verify or identify a person based on their facial patterns. Examples of this range from National Crime Records Bureau's (NCRB) proposal to create a nation wide automated facial recognition system for law enforcement purposes to the Digi Yatra scheme that promotes the use of facial recognition at airports; from Facebook's auto tagging of photographs to Chaayos's use for receiving payments and recording reward points.
The inalienability of a person's face and the convenience with which it can be captured, make it an easy choice for satisfying the ever expanding demands of identifiability in the digital era. This is supplemented by the increased availability of digital images, videos and widespread use of closed circuit television (CCTV) systems, all of which become the fodder for the training and deployment of facial recognition systems. However, this is also the reason why the rapid adoption of FRTs, without any accompanying checks and balances, becomes worrying at many different levels.
In a recent Data Governance Network paper we discuss the growing use cases of FRTs in India and the legal and ethical concerns around it. These concerns include the lack of transparency around the use of FRTs; the threats to privacy and other civil liberties; problems of accuracy and effectiveness; and evidence of biased outcomes. While all of this holds true for the use of FRTs by the government as well as private entities, the imbalance of power between the citizen and the state and the likely consequences from its abuse make it particularly relevant to question the use of FRTs for law enforcement purposes.
Functions and use cases of FRTs
Most of the well known use cases of FRTs can be classified into four buckets based on their underlying functions.
The first function is that of identity verification -- checking if a person really is who they claim to be. For instance, in January, 2018, the Unique Identification Authority of India (UIDAI) had announced that it would allow the use of FRT as one of the modes of authentication under the Aadhaar Act. Through subsequent circulars the UIDAI had also mandated telecom service providers to start undertaking face authentication of their subscribers. While, following the Supreme Court's verdict in the Puttaswamy case, it is no longer possible for the government to mandate Aadhaar based face authentication by private entities like banks and telecom companies, the possibility of it being used by the government for distribution of welfare benefits remains very true.
The use of FRTs for purposes like voter identification, conducting know your customer (KYC) verifications and attendance in schools and offices are some of the other use cases that would fall under this head. For example, Delhi's Indian Institute of Technology has a home-grown solution called Timble that is used to mark student attendance. Proposals are also underway to roll out similar systems to mark the attendance of young school going students in Tamil Nadu's government schools and for all government teachers in the state of Gujarat.
The next function is that of access control, which basically builds on the identity verification function to assess whether a person is an authorised user of a particular space or service. Applications that pursue this function include biometric unlocking of mobile devices, entry into airports, homes or other premises and authorising withdrawals from ATM machines. For instance, in 2018, the Ministry of Civil Aviation launched the Digi Yatra project to create a facial biometrics based boarding system to be launched at various Indian airports. Testing under the project, which is currently voluntary, has already been going on at the Hyderabad, Bengaluru and Delhi airports. Similar systems have already been adopted at airports in many other parts of the world.
The third broad category, which also evokes the strongest concerns, is that of security and surveillance, including use of FRTs for law enforcement purposes. As per the AI Global Surveillance Index released by the Carnegie Endowment for International Peace, 85 percent of the countries that they studied (64 out of 75) were found to be using facial recognition systems for surveillance purposes (Feldstein, 2019). Examples of this include the Skynet and the Sharp Eyes projects in China, live facial recognition systems being tested by the London Metropolitan Police and NCRB's proposed National Automated Facial Recognition System (NAFRS).
As per the tender document released by NCRB in June, 2019, NAFRS is meant to be used for a range of purposes, including the identification of criminals, missing children and persons and unidentified dead bodies. The images that may be used for these purposes may come from the Crime and Criminal Tracking Network System (CCTNS), passport authorities, the Central Finger Print Bureau or the government's missing children tracking portal. The list also contains a sweeping category for "any other image database available with police / other entity". This seems to suggest that virtually each and every database in the country could potentially be linked to this system.
A clarification issued by the NCRB in response to a legal notice sent by the Internet Freedom Foundation (IFF) suggests that the scope of the project may be slightly narrower than what is indicated in the tender document (IFF, 2019). However, even if this were to be believed to be true, the design and scale of the project signal the clear likelihood of a gradual mission creep once such a system is put in place.
In addition to NCRB's proposed system, several state police departments are already deploying facial recognition systems. This includes reports about the use of FRTs by the Delhi Police, the Hyderabad police and under the Punjab Artificial Intelligence System.
Finally, FRTs also serve a number of commercial and business efficiency related functions. This includes photo tagging on social media apps, photo filter functions on chat apps and various uses in the retail and hospitality sectors. For instance, digital signage systems can predict a gazer's age and gender and accordingly display suitable advertisements and content for them. Facial detection and analysis also serves as the building block for other tools like emotion or sentiment analysis, which can offer useful applications in the marketing and entertainment sectors.
What are the main concerns?
Most of the use cases of FRTs, in India as well as globally, can be tied down to the pursuit of greater convenience (contactless payments and shorter queues at airports), efficiency (reduced airport staff), security (scanning crowds for "suspicious" persons), or accountability (checking for teacher absenteeism). While the technology could possibly help in achieving some of these objectives, this is often not established through rigorous and transparent testing. Moreover, the use of FRTs comes at a significant cost, which is not being accounted for by the developers and adopters of such systems.
The primary focus of most of the technical research on face recognition has been on improving the accuracy and efficiency of the technology. In other words, to minimise the false negatives and false positives. While both these metrics are useful indicators for evaluating the effectiveness of machine learning systems, their actual relevance has to be seen in light of the context in which such technologies are being deployed. For instance, false negatives in a system like Aadhaar would lead to the exclusion of legitimate beneficiaries while a false positive in the surveillance and law enforcement context can subject individuals to unwarranted investigation, embarrassment and harassment (Marda, 2019).
However, even if a facial recognition system were to achieve perfect accuracy, that would not make an obvious case for its adoption. This is because the use of FRTs has many other far reaching implications, from a legal, ethical and societal perspective, which need to be taken to account while determining whether and to what extent this technology should be deployed. Following are some of the main areas of concern.
Transparency -- In most situations there is a complete lack of information about when, or the specific purposes for which, FRTs are being deployed. Individuals affected by these systems also do not have access to meaningful information about the sources of training data that were used to develop the system, the sources of gallery images, the criteria for the selection of a particular vendor or technology partner, the accuracy rates of the system and the privacy and security protocols being followed. Transparency about these aspects is a necessary step for enabling independent testing and audits of facial recognition systems.
Information of this sort can become particularly necessary when facial analysis tools are being used to determine whether a person's face matches with someone who is suspected of committing an offence. Civil society groups in the United States are currently contesting a claim before the Florida Supreme Court in a case where a person was convicted for illegal sale of drugs based on the results of a facial recognition algorithm. The accused was the first among a list of probable matches identified by the algorithm with a "one star of confidence" that it had generated the correct match. The person was however not given access to the basis on which this determination was made or the details of the other individuals who were identified as potential matches.
Privacy and civil liberties -- The permanence of one's face and its intrinsic link with personal identity makes facial recognition a powerful tool for identification. The fact that in a large number of cases a person's face is exposed at all times or their images are available in various government and private databases makes it particularly difficult to exercise agency over the use of one's facial data. Some examples of privacy invasive uses of FRTs include its adoption by the Chinese Government for the profiling and tracking of Uighur Muslims and integration of FRTs in body worn cameras used by police forces in many parts of the world.
Widespread use of FRTs can also create a chilling effect on other rights, like the right to free movement, assembly and speech. Visuals of masked protesters in Hong Kong taking down smart lamp posts and surveillance cameras are symbolic of this tussle between the state's use of surveillance technologies and counter-measures being resorted to by protesters. As governments chose to crack down on such forms of resistance through "anti-mask initiatives" this not only affects the rights of the protesters but also those who may adopt facial coverings for various religious, cultural or practical reasons.
Concerns about the overreach of FRTs are however not just limited to autocratic regimes or even to government related uses. Private sector use of facial recognition also poses many significant threats to privacy and security. For instance, researchers have demonstrated how a person's face can easily be used as a personal identifier for pooling together information about them from multiple online sources -- like dating websites and social media portals (Acquisti, Gross, and Stutzman, 2014). Therefore, once a person's images are available online, whether voluntarily or as the result of someone else's actions, FRTs can make it almost impossible for the person to exercise the option of revealing their true identity in one context but remain anonymous in others.
The security of devices that rely on facial unlocking features can become another point of vulnerability for user privacy. The relevance of the differential facial security standards available on different smartphones was brought to light in a study where the researchers found that 26 of the 60 smartphones that they tested were vulnerable to a "photo hack" -- the device could be unlocked using the phone owner's photograph instead of the real person (Kulche, 2019). This illustrates how, given the user profile and characteristics of the Indian market, reliance on facial unlocking techniques on low-end devices could create increased vulnerabilities for consumers.
Accuracy and reliability -- It has been a well acknowledged problem in the field of facial recognition that the results of the system are only as good as the quality of the images that are being run through it. The results are therefore prone to errors on account of differences in the conditions of the images being compared, in terms of appearance, expression, age, lighting, camera angle, etc. This is particularly true in cases where the technology is applied in non-cooperative settings, for instance, using images gathered from a CCTV camera or for real-time biometric processing. For instance, a study on the live facial recognition system being tested by the London Metropolitan Police found that out of the 46 potential matches identified by the system only 8 matches could eventually be verified correctly, indicating a success rate of just about 19 percent (Fussey and Murray, 2019).
Having said that, it is also important to acknowledge that the technical capabilities of facial recognition systems have been improving over time. For instance, 3D facial recognition systems have already managed to overcome many of the technical issues faced by prevalent 2D systems. As per the National Institute of Standards and Technology, the "best performing algorithms" in its 2018 Face Recognition Vendor Testing Program showed significant improvements over the 2015 test results, offering "close to perfect recognition" (Grother, Ngan, and Hanaoka, 2019). Yet, there still remain significant variations in the results among different algorithms and developers, with recognition error rates in a particular scenario ranging from "a few tenths of one percent up to beyond fifty percent".
Bias and discrimination -- The training data being used for FRTs also plays a major role in determining the effectiveness of their outcomes. Buolamwin and Gebru, 2018 have demonstrated how the commercially available facial recognition tools offered by companies like Microsoft, IBM and Face++ showed much higher error rates for women with darker skin tones. This difference arose primarily on account of the under-representation of data belonging to this group in the training dataset. Similarly, a study done by the American Civil Liberties Union using Amazon Rekognition found that nearly 40 percent of the false face matches between members of the US Congress and a database of arrested persons were of people of colour although only about 20 percent of the Congress members actually belonged to this demographic group (Snow, 2018). While most of this research has emanated in the US context, it is easy to draw some parallels with the challenges that would arise in the deployment of similar systems in the context of India's multi-racial, multi-ethic set up.
Research of this nature is valuable in that it can nudge appropriate fixes to the training data and algorithms. However, it has also been rightly pointed out that ensuring better demographic representation in data sets does not do much to solve the larger issues of injustice in the institutional contexts within which facial recognition is being employed (Hoffmann, 2019). For instance, Keyes, 2018 challenges the very premise of deploying automated gender recognition systems, which tend to reflect the traditional models of gender as being binary, physiologically based, and immutable. This works to the specific detriment of transgendered persons, who may not fit into these traditionally defined gender constructs.
Limitations of the supporting ecosystem -- Another important factor, particularly in the Indian context, comes from the realities of the surrounding ecosystem within which technologies like FRTs are sought to be introduced. For instance, the mandatory use of FRTs for marking attendance in rural schools would have to account for real world factors like power outages, network down time, availability of devices and prevailing power structures in the local community.
While these issues go beyond the technical capabilities of FRTs, or even the legal and ethical implications around them, it would be dangerous to adopt such technological solutions without accounting for these realities. Similar concerns have also come up in the context of biometric authentication using Aadhaar, and would continue to remain relevant if facial recognition were to be deployed in this context.
FRTs under the draft PDP Bill
Given the variety of concerns being raised by the deployment of FRTs, it becomes particularly problematic that all of these applications are taking place in the absence of a robust data protection law in India. While the current Information Technology Act, 2000 and the rules under it do classify biometric data as "sensitive personal data" and afford certain protections to it, it is widely acknowledged that the scope and enforcement of the law remain grossly inadequate. Moreover, the obligations under the present law are applicable only to "body corporates", hence excluding most instances where government agencies interact with biometric facial data. It is also worrying to note that there has been no public consultation on the adoption of FRTs in any of the different contexts discussed here nor any systematic evaluation of the costs and benefits of using this technology.
The current draft of the PDP Bill that was recently introduced in the Lok Sabha seeks to take care of some of these concerns by bringing the State along with other private actors who deal with the personal data of individuals within the scope of the proposed law, labelling them as "data fiduciaries". The bill requires that the "explicit consent" of the individual is required for any processing of sensitive personal information, including biometric data. However, it also allows for such processing to take place under other grounds such as an authorisation under law or a court order or judgment.
We have seen an example of such an order from the Delhi High Court which had in April, 2018 directed the Delhi Police to deploy FRTs for tracing missing children. This action reportedly resulted in the identification of close to 3,000 missing children by matching the images of missing children with a photo database of over 45,000 children living in various children's homes. While this was certainly a positive outcome, the episode also leaves us with several unanswered questions. For instance, what happens to the data of the children who were part of this exercise but whose data did not match with the missing children? Will their data be retained and used for other purposes? Could this include use for future investigation of criminal cases?
The other provisions of the draft Bill that are specifically applicable to biometric and sensitive data include a requirement of data protection impact assessment for large scale processing of biometric data by significant data fiduciaries and a requirement that a copy of all sensitive data needs to be localised on data servers in India. Further, the Bill also authorises the government to ban the use of certain forms of biometric data, except as permitted by law. However, there is no guidance on the actors against whom, and the circumstances in which, this power could be exercised.
While many parts of the current draft Bill retain the recommendations made by the Srikrishna Committee's draft that was submitted to the Government in July 2018, we see a sweeping departure from the Committee's recommendations when it comes to the processing of personal data for surveillance and law enforcement purposes.
The current draft of the Bill contains a fairly broad set of exemptions for the processing of personal data for the purposes of prevention or investigation of any offence or contravention of any law. Unlike the earlier version of the Bill, this exemption is not subject to the requirement of fair and reasonable processing of the data by the authorities. It also does not provide that such processing should be "necessary and proportionate" for achieving the intended purpose.
Another important safeguard that was suggested by the Srikrishna Committee was that any data processing involving the victim or a witness would ordinarily have to be done in accordance with the provisions of the law, including requirements like consent, purpose and use limitation, etc, unless this may prejudicially affect the case. By removing this requirement the current draft now offers a much broader canvas to law enforcement agencies. In addition to the exemption of certain types of processing, the PDP Bill also allows the government to completely exempt particular agencies from the applicability of the law on grounds such as security of the state, public order, etc.
To put these exemptions in context, suppose that an order under Section 144 of the Criminal Procedure Code, 1973 (CrPC) is imposed in a particular area directing individuals not to assemble in groups. Any person engaging in a peaceful protest could therefore find themselves acting in violation of the order and therefore the police may invoke the exemption under the PDP Bill to deploy facial recognition tools in order to identify the protestors. Given the wide scope of the facial recognition system being developed by the NCRB and the sweeping powers that are already available to the police to call for any "document or other thing" for investigation purposes, under Section 91 of the CrPC, the PDP Bill could effectively provide a free pass to the authorities to conduct mass deployment of FRTs on the protestors. This may include comparing the available images against the records gathered from a range of sources like CCTVs, student IDs, driving licenses, passport records, etc. This creates new barriers to the exercise of people's democratic right to protest.
In sum, the present draft of the PDP Bill offers wide ranging exemptions to law enforcement agencies, and can be regarded as effectively strengthening rather than checking the use of FRTs by the state.
Way forward
Facial biometric data is one of the most sensitive categories of personal data and therefore any adoption of this technology, either by state agencies or by the private sector, necessarily has to be preceded by the adoption of a robust data protection law. Assuming that a data protection law is brought about along the lines of the PDP Bill, it would determine the basic level of protection for the use of facial biometrics, including requirements relating to explicit consent, transparency obligations, purpose limitation and other usage restrictions.
However, the proposed data protection framework will not secure the degree of accountability that we need from the range of stakeholders participating in the implementation of FRTs. Firstly, a data protection law is not designed to compel the developers and vendors of facial recognition systems (as opposed to its users) to ensure transparency about their underlying models, training data being used, false positive and negative rates and other more granular information. Yet, information of this sort is necessary for there to be any independent checks and analysis on the accuracy, reliability and biases in the systems. We therefore need to look beyond data protection laws to find meaningful ways of ensuring transparency and public disclosure on the development and use of facial recognition systems.
Secondly, it must be noted that the PDP Bill only speaks to a few of the concerns posed by the use of FRTs, namely issues of data privacy and, to some extent, transparency. However, the broader privacy concerns posed by the technology, its accuracy limitations and biased outcomes still remain. Here it is useful to reiterate that with ongoing advances in technology, it is likely that many of the accuracy and reliability related concerns around FRTs might be overcome. However, satisfactory technical performance of such systems is only a necessary, but not sufficient, condition for their deployment. The use of FRTs has to be supported, in all cases, by a robust framework for gauging the suitability and proportionality of applying the technology in any given context and measuring the accompanying risks.
Finally, the wide ranging exemptions available to state agencies under the PDP Bill pose many specific concerns when it comes to the use of intrusive technologies like FRTs. In allowing for the sweeping application of FRTs for law enforcement purposes, the PDP Bill essentially condones the most pervasive and worrying use cases of FRTs. To be clear, such a use would still fall foul of the tests laid down by the Supreme Court in the Puttaswamy right to privacy decision. However, the language in the Bill lifts the statutory burden that should have been placed on law enforcement agencies to ensure proportionate application in each and every case and places the burden on petitioners to challenge the constitutionality of the application before a court of law.
References
Acquisti, Gross, and Stutzman, 2014: Alessandro Acquisti, Ralph Gross and Fred Stutzman, Face recognition and privacy in the age of augmented reality, Journal of Privacy and Confidentiality, 6(2), 2014.
Buolamwin and Gebru, 2018: Joy Buolamwin and Timnit Gebru, Gender shades: Intersectional accuracy disparities in commercial gender classification, Proceedings of Machine Learning Research, 81:1–15, 2018.
Feldstein, 2019: Steven Feldstien, The global expansion of AI surveillance, Carnegie Endowment for International Peace, 17 September, 2019.
Fussey and Murray, 2019: Pete Fussey and Daragh Murray, Independent report on the London Metropolitan Police Service’s trial of live facial recognition technology, The Human Rights, Big Data and Technology Project, July, 2019.
Grother, Ngan, and Hanaoka, 2018: Patrick Grother, Mei Ngan and Kayee Hanaoka, Ongoing face recognition vendor test (FRVT) Part 2: Identification, National Institute of Standards and Technology, November, 2018.
Hoffmann, 2019: Anna Lauren Hoffman, Where fairness fails: Data, algorithms, and the limits of anti discrimination discourse, Information, Communication & Society, 22(7), 2019.
IFF, 2019: Internet Freedom Foundation, NCRB finally responds to legal notice on facial recognition, we promptly send a rejoinder, 8 November, 2019.
Keyes 2019: Os Keyes, The misgendering machines: Trans/HCI implications of automatic gender recognition, Proceedings of the ACM on Human-Computer Interaction, November 2018.
Kulche, 2019: Peter Kulche, Facial recognition on smartphone is not always safe, Consumentenbond, 15 April 2019.
Marda, 2019: Vidushi Marda, Facial recognition is an invasive and inefficient tool, The Hindu, 22 July, 2019.
Snow, 2018: Jacob Snow, Amazon’s face recognition falsely matched 28 members of congress with mugshots, American Civil Liberties Union, 28 July, 2018.
The author is a Fellow at the National Institute of Public Finance and Policy, New Delhi. She would like to thank Ajay Shah, Ambuj Sagar, Apar Gupta, Christopher Slobogin, Elizabeth Coombs, Salil Tripathi, and an anonymous peer reviewer for valuable inputs and comments on the Data Governance Network paper titled Adoption and regulation of facial recognition technologies in India: Why and why not?, which forms the basis for this blog post.
