Deepfakes
Audio, video, or images generated or altered by machine learning to show a real person saying or doing something they did not, or to fabricate an event, with a realism that defeats casual inspection.
Recorded sound and moving pictures were the last evidence most people trusted on sight. Deepfakes take that trust and forge the evidence. A model trained on a person's face or voice can produce them saying anything, and the result is persuasive because the audience's test, "I saw it, I heard it," is the thing being faked. The device is testimonial and fabricated evidence at once.
Its second effect is the more durable. Once forgeries are known to be possible, genuine recordings can be dismissed as forgeries, a move Chesney and Citron called the "liar's dividend." A propagandist need not produce a convincing fake; the existence of fakes is enough to discount any inconvenient truth. Exploits Seeing Is Believing and, in reverse, Ambiguity Effect.
Historical examples
Chesney and Citron's "Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security" surveyed the technology and its uses and introduced the "liar's dividend": as the public learns that recordings can be faked, those caught on genuine recordings gain a new defense. The paper set the terms of the policy debate that followed.
Chesney, Robert, and Danielle Keats Citron. "Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security." California Law Review 107, no. 6 (2019): 1753–1820. Source
When Gabon's government released a New Year's address by President Ali Bongo after months of his absence for medical treatment, opponents declared the video a deepfake, and the claim became part of the justification for an attempted coup days later. Analysts found no evidence of forgery. The episode is the standard case of the liar's dividend: the possibility of a fake did the political work.
Hao, Karen. "The Biggest Threat of Deepfakes Isn't the Deepfakes Themselves." MIT Technology Review, 10 October 2019; Chesney and Citron (2019).
In advertising and commerce
In 2019 the chief executive of a British energy firm transferred €220,000 after a call from what he believed was his parent company's chief, a voice clone. In 2024 a finance employee at the engineering firm Arup's Hong Kong office paid out about $25 million after a video call in which every other participant, including the chief financial officer, was synthetic. The device's commercial use is theft.
Stupp, Catherine. "Fraudsters Used AI to Mimic CEO's Voice in Unusual Cybercrime Case." The Wall Street Journal, 30 August 2019; "Arup Confirms Deepfake Fraud," CNN, 17 May 2024.
Find a clip of a public figure that seems surprising. Can you find where it was first published, and by whom? Are there other recordings of the same event?
In the wild
Short examples of the technique at work, each with what to notice. The same examples are questions in the quiz, where the answer is not given away.
A video shows the candidate slurring an insult at voters; it exists only as a re-upload, and the campaign says the event never happened.
What to notice. A fabricated recording with no provenance; the realism is the device.
Confronted with a recording of his own remarks, the executive said, "Anyone can fake anything now," and offered no analysis.
What to notice. The liar's dividend: the existence of fakes is used to discount a genuine recording.
"Your CFO is on the video call and has approved the transfer." Every face on the call was generated.
What to notice. Synthetic media impersonating real authority to command an action.
Sources
- Chesney, Robert, and Danielle Keats Citron. "Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security." California Law Review 107, no. 6 (2019): 1753–1820. doi:10.15779/Z38RV0D15J
- Paris, Britt, and Joan Donovan. Deepfakes and Cheap Fakes: The Manipulation of Audio and Visual Evidence. New York: Data & Society Research Institute, 2019.
- Stupp, Catherine. "Fraudsters Used AI to Mimic CEO's Voice in Unusual Cybercrime Case." The Wall Street Journal, 30 August 2019.