Last month during ESPN’s hit documentary series The Last Dance, State Farm debuted a TV commercial that has become one of the most widely discussed ads in recent memory. It appeared to show footage from 1998 of an ESPN analyst making shockingly accurate predictions about the year 2020.

As it turned out, the clip was not genuine: it was generated using cutting-edge AI. The commercial surprised, amused and delighted viewers.

What viewers should have felt, though, was deep concern.

The State Farm ad was a benign example of an important and dangerous new phenomenon in AI: deepfakes. Deepfake technology enables anyone with a computer and an Internet connection to create realistic-looking photos and videos of people saying and doing things that they did not actually say or do.

A combination of the phrases “deep learning” and “fake”, deepfakes first emerged on the Internet in late 2017, powered by an innovative new deep learning method known as generative adversarial networks (GANs).

Several deepfake videos have gone viral recently, giving millions around the world their first taste of this new technology: President Obama using an expletive to describe President Trump, Mark Zuckerberg admitting that Facebook’s true goal is to manipulate and exploit its users, Bill Hader morphing into Al Pacino on a late-night talk show.

The amount of deepfake content online is growing at a rapid rate. At the beginning of 2019 there were 7,964 deepfake videos online, according to a report from startup Deeptrace; just nine months later, that figure had jumped to 14,678. It has no doubt continued to balloon since then.

While impressive, today’s deepfake technology is still not quite to parity with authentic video footage—by looking closely, it is typically possible to tell that a video is a deepfake. But the technology is improving at a breathtaking pace. Experts predict that deepfakes will be indistinguishable from real images before long.

“In January 2019, deep fakes were buggy and flickery,” said Hany Farid, a UC Berkeley professor and deepfake expert. “Nine months later, I’ve never seen anything like how fast they’re going. This is the tip of the iceberg.”

Today we stand at an inflection point. In the months and years ahead, deepfakes threaten to grow from an Internet oddity to a widely destructive political and social force. Society needs to act now to prepare itself.

When Seeing Is Not Believing

The first use case to which deepfake technology has been widely applied—as is often the case with new technologies—is pornography. As of September 2019, 96% of deepfake videos online were pornographic, according to the Deeptrace report.

A handful of websites dedicated specifically to deepfake pornography have emerged, collectively garnering hundreds of millions of views over the past two years. Deepfake pornography is almost always non-consensual, involving the artificial synthesis of explicit videos that feature famous celebrities or personal contacts.

From these dark corners of the web, the use of deepfakes has begun to spread to the political sphere, where the potential for mayhem is even greater.

It does not require much imagination to grasp the harm that could be done if entire populations can be shown fabricated videos that they believe are real. Imagine deepfake footage of a politician engaging in bribery or sexual assault right before an election; or of U.S. soldiers committing atrocities against civilians overseas; or of President Trump declaring the launch of nuclear weapons against North Korea. In a world where even some uncertainty exists as to whether such clips are authentic, the consequences could be catastrophic.

Because of the technology’s widespread accessibility, such footage could be created by anyone: state-sponsored actors, political groups, lone individuals.

In a recent report, The Brookings Institution grimly summed up the range of political and social dangers that deepfakes pose: “distorting democratic discourse; manipulating elections; eroding trust in institutions; weakening journalism; exacerbating social divisions; undermining public safety; and inflicting hard-to-repair damage on the reputation of prominent individuals, including elected officials and candidates for office.”

Given the stakes, U.S. lawmakers have begun to pay attention.

“In the old days, if you wanted to threaten the United States, you needed 10 aircraft carriers, and nuclear weapons, and long-range missiles,” U.S. Senator Marco Rubio said recently. “Today….all you need is the ability to produce a very realistic fake video that could undermine our elections, that could throw our country into tremendous crisis internally and weaken us deeply.”

Technologists agree. In the words of Hani Farid, one of the world’s leading experts on deepfakes: “If we can’t believe the videos, the audios, the image, the information that is gleaned from around the world, that is a serious national security risk.”

This risk is no longer just hypothetical: there are early examples of deepfakes influencing politics in the real world. Experts warn that these incidents are canaries in a coal mine.

Last month, a political group in Belgium released a deepfake video of the Belgian prime minister giving a speech that linked the COVID-19 outbreak to environmental damage and called for drastic action on climate change. At least some viewers believed the speech was real.

Even more insidiously, the mere possibility that a video could be a deepfake can stir confusion and facilitate political deception regardless of whether deepfake technology has actually been used. The most dramatic example of this comes from Gabon, a small country in central Africa.

In late 2018, Gabon’s president Ali Bongo had not been seen in public for months. Rumors were swirling that he was no longer healthy enough for office or even that he had died. In an attempt to allay these concerns and reassert Bongo’s leadership over the country, his administration announced that he would give a nationwide televised address on New Years Day.

In the video address (which is worth examining firsthand yourself), Bongo appears stiff and stilted, with unnatural speech and facial mannerisms. The video immediately inflamed suspicions that the government was concealing something from the public. Bongo’s political opponents declared that the footage was a deepfake and that the president was incapacitated or dead. Rumors of a deepfake conspiracy spread quickly on social media.

The political situation in Gabon rapidly destabilized. Within a week, the military had launched a coup—the first in the country since 1964—citing the New Years video as proof that something was amiss with the president.

To this day experts cannot definitively say whether the New Years video was authentic, though most believe that it was. (The coup proved unsuccessful; Bongo has since appeared in public and remains in office today).

But whether the video was real is almost beside the point. The larger lesson is that the emergence of deepfakes will make it increasingly difficult for the public to distinguish between what is real and what is fake, a situation that political actors will inevitably exploit—with potentially devastating consequences.

“People are already using the fact that deepfakes exist to discredit genuine video evidence,” said USC professor Hao Li. “Even though there’s footage of you doing or saying something, you can say it was a deepfake and it’s very hard to prove otherwise.”

In two recent incidents, politicians in Malaysia and in Brazil have sought to evade the consequences of compromising video footage by claiming that the videos were deepfakes. In both cases, no one has been able to definitively establish otherwise—and public opinion has remained divided.

Researcher Aviv Ovadya warns of what she terms “reality apathy”: “It’s too much effort to figure out what’s real and what’s not, so you’re more willing to just go with whatever your previous affiliations are.”

In a world in which seeing is no longer believing, the ability for a large community to agree on what is true—much less to engage in constructive dialogue about it—suddenly seems precarious.

A Game of Technological Cat-And-Mouse