Hill, Kashmir (2023). Your Face Belongs to Us: A Secretive Startup’s Quest To End Privacy As We Know It. New York: Random House, 352 pages.
By Melanie Kolbe-Guyot
Admittedly, it took me some time to finally pick up and read investigative journalist Kashmir Hill’s book “Your Face Belongs to Us: A Secretive Startup’s Quest To End Privacy As We Know It.” But once I did, it was quite hard to put it down again. Unlike many other books we review as part of our ongoing “Digital Governance Book Review” series, “Your Face Belongs to Us” is not an academic book rather an investigative journalistic one, representing the culmination of Hill’s painstaking research on Clearview AI, a hitherto little-known face-recognition technology start-up.
First published in an article in The New York Times on November 2, 2021, “The Secretive Company That Might End Privacy As We Know It,” Hill publicly disclosed shocking information for many: Hundreds of police and intelligence units in the U.S. secretly contracted Clearview AI services to run facial-recognition software on online pictures of U.S. (and foreign) citizens, without their knowledge or consent. In this book, Hill traces the way this company came into being. She also discusses the future of privacy that the commercialization of facial-recognition technology in conjunction with the availability of countless online pictures to feed it with, will bring.
But more specifically, this book is about a company that broke a taboo. As Hill justly summarizes in her prologue, the prior cross-sectoral status quo was that “no one should roll out an application to identify strangers. It was too dangerous, they said. A weirdo at a bar could snap your photo and within seconds know who your friends were and where you lived. It could be used to identify antigovernment protesters or women who walked into Planned Parenthood clinics. It would be a weapon for harassment and intimidation. Accurate facial recognition, on the scale of hundreds of millions or billions of people, was the third rail of technology. And now Clearview, an unknown player in the field, claimed to have built it” (p. ix).
Hill’s book reads like a true crime story, seamlessly blending biography, history of facial-recognition technology, and real-world developments and controversies. She introduces a cast of diverse actors, from Clearview’s founders—in particular the key figure, Hoan Ton-That—to those who recognized and resisted the looming danger of extensive facial-data collection and processing by private companies without citizens’ knowledge and with minimal governmental regulation.
The book’s greatest strengths are its extensive evidence, meticulous reconstruction of key events, and Hill’s compelling storytelling skills. These elements enable her to carefully trace the emergence of Clearview AI and offer insights into the motivations, thoughts, and reactions of its protagonists and those who employ the company’s services. In particular, she gives us a glimpse into the consequences that a singular focus on technological progress, devoid of ethical considerations, would be for society.
Although Hill’s book makes no claim to be academic, it nonetheless contains numerous highly pertinent observations that make it appealing to academics and anyone keenly interested in privacy.
Ethical Arbitrage and Surveillance Roots
The book is structured into three parts, roughly divided as follows: (1) the early years of the company’s founding and development, including the scientific and policy antecedents of the underlying technology; (2) the gradual breakthrough of the company and facial recognition technology, at large, including the first public use-cases; and (3) the reactions and fallout from the implementation of said technology, particularly in law enforcement contexts.
At the center of the story is the somewhat eccentric persona of Hoan Ton-That, the founder and creator of Clearview AI, and his journey in building, developing, and pitching the app. Two other frequently featured characters include Richard Schwartz and Charles Johnson, whose main contributions were to provide connections and to raise capital for the start-up. Especially regarding Ton-That, the book reads partially like a biography, detailing his early connections to the far right and the Trump camp, leading up to his dealings, in recent years, with the increased media and regulatory push-back against Clearview AI. I found it most interesting that the book offers a glimpse into his quintessential Silicon Valley mindset that prioritizes progress and technological capabilities over the implications of their highly problematic ends (i.e. their potential use to repress, discriminate, and persecute people).
This mindset stands out against, as Hill shows in Chapters 10 and 14, the considerations and actions of other academic researchers and industry leaders at Google and Meta. They had also developed facial-recognition technology and were aware of its risky potential in identifying anyone, based on their face. However, unlike Ton-That, they ultimately decided to refrain from breaking the taboo of commercializing the technology (Initially, Meta did use it to suggest tagging friends recognized in users’ uploaded pictures, but they phased out this feature in 2021).
As Hill aptly points out, it was this taboo-breaking that differentiated Ton-That and his collaborators from their contemporaries: “What Clearview had done was astounding, but not for technological reasons. Companies such as Google and Facebook were capable of building the technology that Clearview had built, but they had regarded it as too taboo to release to the world. The significance of what Clearview had done was not a scientific breakthrough, it was ethical arbitrage. Ton-That and his colleagues had been willing to cross a line that other technology companies feared, for good reason” (p. 163).
One of the key themes that stand out to me in the book is the implicit danger of technological progress that lacks ethical considerations. Throughout the narrative, I kept wondering whether Ton-That and his collaborators truly did not understand or simply did not care about the mass surveillance implications enabled by their technology. At various points during the description of the company’s development—from its antecedents to its earlier iteration as SmartCheckr.com, and finally to its incarnation as Clearview AI—Hill revisits this unspoken question: Were the men behind Clearview AI aware of the potential nefarious uses of facial-recognition technology and the accompanying ethical concerns?
Through interviews and reconstructed e-mail exchanges, the answer ultimately emerges as “yes,” but with a caveat. They interpreted these potential uses differently. The astonishing achievement that highly accurate mass facial-recognition technology represents was seen by them as “the future.” Meanwhile, the consistent response from people trying out their app was initially amazement, followed by being “creeped out” by the sheer amount of information that could be revealed, from a simple picture search, about a random person. This response, however, was interpreted as “future shock” rather than a clear indication of the ethical and privacy concerns that were causing discomfort.
The removal of privacy appears to have been a key objective of the endeavor. In Chapter 5, Hill details the inception of the subsequent Clearview AI, by beginning with its predecessor, SmartCheckr.com. This app was designed to find people by linking their e-mail addresses to pictures, their social-media footprints, political leanings, and personal information online. The very first use case—for a right-wing public-event organizer wanting to identify and prevent political opponents from entering their event—reveals that the technology’s core purpose was always to strip away individuals’ privacy by de-masking and identifying them in the real world.
SmartCheckr was essentially a background screening tool, a characteristic that Clearview AI inherited, despite the founders promoting other applications, such as identifying criminals or aiding in CSAM (Child Sexual Abuse Material) investigations. After unsuccessful attempts to establish SmartCheckr as a political intelligence tool and the mounting bad publicity it received, the tool was dropped, only to reemerge as Clearview AI under a new corporate identity in 2017.
Nonetheless, as chronicled in Chapters 11 and 12, the fundamental aim of linking real-world identities to online photos and to information found its eventual key market in law enforcement and homeland security units. These customers inherently had both an interest and a practical use for such comprehensive facial-recognition technology.
Public Images and Unintended Consequences
“Your Face Belongs to Us” is not just a book about the ethical implications of facial-recognition technology; it also delves deeply into the development, science, and technology behind it. In several chapters, Hill expertly weaves in early (now discredited) scientific theories that sought to connect human traits to biological markers through the study of the human body. Notably, both, these early theories on facial markers and social deviance, and the current applications of sophisticated facial recognition technology, found their most relevant use cases in law enforcement contexts.
In Chapter 4, Hill skillfully details the various failures and successes in the development of facial-recognition technology, highlighting researchers’ insights occurring in parallel with Ton-That and his collaborators’ efforts. The narrative also explores the growing unease among policymakers, such as James Ferg-Cadima and Alvaro Bedoya, regarding the collection and analysis of biometric data and photos by private companies. These stories highlight the legal blind spots in protecting citizens from privacy incursions by private companies and the mixed success of efforts to block these invasions.
This growing unease becomes especially relevant in the changing political context and increasing public awareness, from the first “in the wild” mass use-case of facial recognition during the Super Bowl in 2001 to the aftermath of 9/11 and the efforts of NIST (the US National Institute of Standards and Technology) in measuring and benchmarking the performance of facial-recognition programs.
In tracing the development of this technology, Hill not only problematizes the accuracy of algorithms but also highlights the importance of the availability of publicly accessible photos used to train these algorithms. The sheer abundance of billions of public images, often easily scraped, collected, and stored by third-party actors like Clearview AI, emerges as a key theme and a particularly contentious point in the book’s later chapters.
As Hill demonstrates, Ton-That, his collaborators, and numerous subcontractors spent years amassing photos from publicly accessible websites such as LinkedIn, Flickr, Venmo, Facebook, and others. This extensive collection of information contributes significantly to the troubling power of Clearview AI’s technology.
In Chapter 19, Hill reveals that the collection and processing of pictures of EU citizens, without their knowledge and consent, violated GDPR rules, thus leading several countries to declare Clearview AI’s operations illegal. Conversely, in jurisdictions without such entrenched protections, Clearview’s defense rested on the argument that these photos existed in the public domain and that the company merely organized publicly available information.
Even more troublesome is that Clearview AI’s actions could ultimately be replicated by many other entities; Hill illustrates this through the replication experiment conducted by two Swiss data-journalists. They built their own facial-recognition technology by using freely available photos and facial-recognition software. “It was a shocking demonstration. The building blocks for a face search engine were just too easy to come by: online photos, more and more of them every day, just waiting to be scraped. The facial recognition algorithms to sort through them were increasingly becoming plug-and-play technology. It meant that even if Clearview were smacked down, copycats would be easy to create” (p. 195-196).
The issue of publicly available photos also taps into a broader problem that transcends facial recognition: the inability of users to predict how any data they put online can later be used, potentially against them. “This is the challenge of protecting privacy in the modern world. How can you fully comprehend what will become possible as technology improves? Information that you give up freely now, in ways that seem harmless, might come back to haunt you when computers get better at mining it” (p. 110). This insight resonates with contemporary discussions about companies that, such as OpenAI, purchase chat data to train their language models, a development few users of public forums and chat platforms could have foreseen before the advent of ChatGPT.[1]
Resisting a “Perpetual Police Lineup”
The fallout and responses to Clearview AI are the subject of the book’s final chapters. The most powerful sections, in my opinion, are those illustrating the negative impact and specific harmful use-cases of facial-recognition technology. In Chapter 17, for instance, Hill powerfully narrates in detail how a facial recognition mismatch led to the wrongful arrest of a Black father, resulting in traumatizing after-effects for his entire family. This incident highlights not only the technology’s failure but also human error, as law enforcement deemed the software match to be sufficient evidence to charge and arrest the suspect.
Beyond issues of technological inaccuracy and overreliance on machine decision-making, Hill also warns that the use of facial-recognition technology—particularly Clearview’s brand—alters both the subjects and the methods of discrimination. She notes, “Most of our antidiscrimination laws are predicated on categories that are typically visible at a glance: race, gender, disability. Technology such as Clearview’s will make possible a new era of discrimination, allowing businesses to turn away customers based on what is knowable about them from an internet search of their faces” (p. 188). This development also challenges existing protections for citizens. As Hill correctly observes, “Police didn’t need a warrant; they just needed to pay for the intel. Centuries-old rights protecting American citizens from government abuses were growing increasingly irrelevant as private companies took over the work of surveilling the country’s inhabitants” (p. 204).
Although Hill includes facial-recognition technology use-cases from Russia, China, and the UK, her primary focus is on its deployment in the United States. In Chapters 21 and 24, she devotes special attention to the efforts to combat Clearview AI. One of the most enlightening insights from these chapters is the realization that challenging the use of facial-recognition technology, based solely on arguments of algorithmic accuracy and racial bias, is largely ineffective. Hill cites an activist who observed, “The advocacy community had ‘led with its chin’ by focusing so much attention on a fixable problem with the technology, as opposed to the broader threat it would pose when perfected. That had provided facial recognition purveyors the opportunity to use greater accuracy across diverse groups as a justification to deploy the technology more widely” (p. 239). When called to justify itself before Congress, Clearview AI capitalized precisely on this line of defense.
In the somewhat sobering Chapter 21, Hill details how the ACLU (American Civil Liberties Union), a historic US civil-rights organization, grappled with restricting Clearview AI’s practices. They ultimately settled their lawsuit, securing an agreement from the company to not sell its database to private individuals and corporations. However, law enforcement agencies can still use Clearview’s services. This might seem like a victory, but it does not resolve the underlying issue: As critics have argued, law enforcement’s use of Clearview’s technology places everyone in a “perpetual police lineup.”[2]
Although only briefly addressed in the book, the legal responses to the collection and processing of images from the public domain constitute another important issue. Several complaints in France, Greece, and Austria, citing violations of GDPR rules regarding the collection and processing of data on individuals located in these countries, led to fines for Clearview AI in 2021 and 2022. However, the company does not sell its services in Europe (although U.S. law enforcement still has access to European citizens’ data) and does not cooperate with EU regulators, rendering these fines largely ineffective.[3]
In some instances, Clearview AI successfully appealed such penalties. For example, in 2022, the company was fined over £7.5 million by the UK’s Information Commissioner’s Office (ICO) for illegally storing facial images. However, in 2023, Clearview AI successfully appealed the ICO’s fine and enforcement actions, arguing that its services were used exclusively by law enforcement agencies outside the UK, where the ICO has no jurisdiction.[4]
The book ends somewhat abruptly, with a final interview with its founder, Ton-That, who continues to interpret the legal and political backlash to the app as “future shock” that will eventually resolve itself. With this, Hill returns to one of the major tensions of technological progress: the clash between innovation and ethical considerations. This underscores the ongoing battle between the relentless push for technological advancement and the societal need for checks and balances to ensure that such progress does not come at the expense of fundamental rights and privacy.
Appraisal
The book’s greatest strengths lie in its comprehensive documentation, detailed depiction of crucial events, and Hill’s engaging narrative techniques. As noted throughout this review, she adeptly pinpoints the critical issues and tensions at the heart of the battle between technological progress and privacy. However, the mosaic-like stitching of different stories, topics, and character chapters can sometimes be dizzying. Although the initial shifting between storylines helped build interest and engagement, toward the book’s end, it became more distracting.
Furthermore, I find that the reader needs a concluding chapter that would weave together the various threads, especially those in the third part of the book, into a coherent conclusion. Although it is clear that the story of Clearview AI is far from over and that the book can only capture a fraction of it, the end of the narrative leaves the reader with frayed threads rather than a cohesive global message that Hill (in my opinion) had been building up throughout the book
As a US journalist writing for a US audience, Hill focused predominantly on American developments; which is understandable. I appreciated the excursions to the cases of Russia and China, however, I found them to be too brief. Both cases provide ample evidence of a future marked by political repression and social control through facial-recognition technology and extensive tracking of digital footprints. These examples deserved greater attention to underscore her point and to serve as a stark warning for future US developments.
In summary, however, this book and Hill’s research and perseverance constitute an important read about the ways technology companies continue to profoundly shape our societies and thus should not be beyond scrutiny or accountability.
[1] Controversies in 2024 surrounded in particular the partnership between OpenAI and, respectively, StackOverflow and Reddit, to train their LLMs with user these platforms’ user data.
[2] Chris Vallance, BBC (October 18, 2023), “Face search company Clearview AI overturns UK privacy fine.” https://www.bbc.com/news/technology-67133157
[3] Natasha Lomas, TechCrunch (May 10, 2023), “Clearview fined again in France for failing to comply with privacy orders” https://techcrunch.com/2023/05/10/clearview-ai-another-cnil-gspr-fine/
[4] Chris Vallance, BBC (October 18, 2023), “Face search company Clearview AI overturns UK privacy fine.” https://www.bbc.com/news/technology-67133157
This edition of the Digital Governance Book Review was authored by: Melanie Kolbe-Guyot, C4DT.
Image credit: Cover of Your Face Belongs to Us: A Secretive Startup’s Quest To End Privacy As We Know It by Kashmir Hill, published by Random House.