Synthetic personas
Fabricated people, with generated faces, invented biographies, and machine-written output, deployed as commentators, journalists, or experts, so that a message arrives from someone who does not exist.
A sock puppet is an existing person's second identity; a synthetic persona is nobody's first. Generated portraits that match no living face, biographies assembled from plausible fragments, profiles that publish articles, columns, and comments written by a model: the result is a speaker who cannot be traced, exposed, or embarrassed, because there is no one behind the name. Where the audience once checked a byline against a person, the check now returns a face and a résumé and nothing else.
The device's first documented cases used generated faces to populate influence networks in 2019; a fabricated freelance journalist placed opinion pieces in real publications in 2020; language models since have made the persona's output as easy to fabricate as its face. Exploits Authority Bias and the trust in a face and a name.
Historical examples
Facebook's removal in December 2019 of a network of over 600 accounts linked to a media company was the first major takedown to report the use of machine-generated profile photographs at scale; the analysis by Graphika and the Atlantic Council's Digital Forensic Research Lab documented the faces' telltale artifacts. Generated faces have appeared in nearly every network takedown since.
Nimmo, Ben, et al. #OperationFFS: Fake Face Swarm. Graphika and DFRLab, December 2019; Meta. Coordinated Inauthentic Behavior Report, December 2019.
"Oliver Taylor," a supposed University of Birmingham student and freelance writer, placed opinion pieces in the Jerusalem Post, the Times of Israel, and other outlets in 2019 and 2020 attacking activists. A Reuters investigation found no such student, a profile photograph that experts identified as generated, and a persona that vanished when exposed. The editors had never spoken to him.
Satter, Raphael. "Deepfake Used to Attack Activist Couple Shows New Disinformation Frontier." Reuters, 15 July 2020.
In advertising and commerce
Websites staffed by invented writers with generated portraits and machine-written articles, sometimes numbering in the hundreds, have been documented by researchers tracking "unreliable AI-generated news," alongside generated "experts" quoted in sponsored content and generated influencers with product endorsements and follower counts in the millions.
NewsGuard. "Tracking AI-Enabled Misinformation." Reports, 2023–2024; Sadeghi, McKenzie, and Lorenzo Arvanitis. "Rise of the Newsbots." NewsGuard, May 2023.
Pick a commentator whose work you've seen but who you know nothing about. Try to establish that they exist: employer, colleagues, events, a photograph with someone else. How far did you get?
In the wild
Short examples of the technique at work, each with what to notice. The same examples are questions in the quiz, where the answer is not given away.
The columnist has a headshot, a degree, and forty published pieces, and no colleague, editor, or event can be found that confirms she exists.
What to notice. A speaker fabricated whole; the byline is the only evidence of the person, and it was made.
The influencer endorsing the supplement has two million followers and a face that a reverse search matches to nothing at all.
What to notice. A generated endorser; testimonial from someone who does not exist.
Two hundred "local news" sites list the same twelve staff writers, whose portraits share a blurred background and a centerd gaze.
What to notice. Synthetic staff populating spoofed outlets; the faces are the tell.
Sources
- Nimmo, Ben, C. Shawn Eib, L. Tamora, et al. #OperationFFS: Fake Face Swarm. Graphika and DFRLab, 2019.
- Satter, Raphael. "Deepfake Used to Attack Activist Couple Shows New Disinformation Frontier." Reuters, 15 July 2020.
- Goldstein, Josh A., et al. Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations. Stanford Internet Observatory, Georgetown CSET, and OpenAI, 2023.