Deepfakes: Identifying and Addressing Synthetic Media

We could think of deepfakes as the modern-day equivalent of Photoshopping. It is basically linked to the creation of photographs of fictitious events using certain types of artificial intelligence known as deep learning; hence, the term “deepfake”. 

You have most likely already seen some deepfakes floating around the internet, having a celebrity or politician as a target. However, not much has been said about them, so keep reading to find out what is exactly behind deepfakes.

1. Deepfakes: What’s behind synthetic media?

The term deepfakes comes as a result of a combination of the words “deep learning” and “fake”. 

Deepfakes are a cutting-edge technology that manipulates or creates totally synthetic audio and video content using deep learning algorithms, frequently by superimposing one person’s voice or likeness onto another’s. 

This approach makes it possible to produce wholly fictional but incredibly convincing media, raising serious problems for privacy, identity, and misinformation in the digital era.

You may read: An Approach to the Triarchic Theory of Intelligence

2. Uses of deepfakes

When looking at the world of deepfakes one can notice many are often used in the pornographic industry.

Back in September 2019, the AI company Deeptrace discovered around 15,000 deepfake videos online—a nearly tripling in just nine months. 

A startling 96% of them were pornographic, and 99% of them morphed the faces of women in the public eye onto porn stars. Fake movies are likely to become more popular outside the world of celebrities as new technologies enable unskilled people to create deepfakes with just a few images.

In the words of Boston University law professor Danielle Citron, “Deepfake technology is being weaponized against women”. There is a lot of spoof, satire, and mayhem in addition to the porn.

3. Are deepfakes limited to videos?

Deepfakes are definitely not exclusive to videos. The deepfake technology also allows for the creation of convincing but completely fictitious images.

Besides that, deepfakes can also be created through audio only. You can manipulate this technology to produce “voice skins” or “voice clones” of famous people. 

There have been some previous scams that involve, one way or another, deepfakes. Such was the case of the head of a UK division of a German energy company, as they transferred approximately £200,000 into a Hungarian bank account in March of last year after being called by a con artist who impersonated the German CEO.

Although the evidence surrounding this particular case remains ambiguous, the findings have led to believe the voice as a product of a deepfake.

4. How are deepfakes made?

Audiovisual technologies have for sure captivated most of us. With time, the capabilities of both video and picture manipulation have become a subject of study for researchers and special effects studios. 

When it comes to deepfakes and the tools and technologies behind their creation, the research has still a long way to go.

The term “deepfake” was first used in 2017 by a Reddit user under the same name who uploaded edited pornographic videos on the website. In the videos, porn actors were given celebrity faces such as those of Gal Gadot, Taylor Swift, Scarlett Johansson, and others.

The truth is current technologies make it quite simple for users to create deepfakes, for example, a face-swap video. 

Let’s say the process behind deepfakes requires hundreds of face images in order to achieve a positive outcome, usually, they are first put through an encoder, an AI program. 

Once there, and in order to compress the photos, the encoder discovers and learns commonalities between the two faces, reducing them to their shared features.

The faces in the compressed photos are then taught to be recovered by a second AI system known as a decoder. 

You may also read: Yakutia 50 degrees below zero: How is life in the coldest city in the world?

You train one decoder to recover the first person’s face and another decoder to recover the second person’s face since the faces are different. You simply send encoded photos into the “wrong” decoder to carry out the face swap. 

For instance, the decoder trained on person B is fed a compressed image of person A¡s face. Then, using the features and positioning of face A, the decoder reconstructs the face of person B. This must be carried out on every frame for the video to be believable.

5. Who is behind deepfakes?

The truth is there are many different groups behind deepfakes— everyone, including experts from academia and industry, amateur enthusiasts, visual effects studios, and porn makers. 

Governments may also be experimenting with the technology as part of their online strategy to, for example, reach specific individuals or disrupt and delegitimize extremist organizations.

6. The technology behind deepfakes

On a typical computer, creating good deepfakes is definitely challenging. 

The vast majority of deepfakes are produced using cutting-edge desktop computers with potent graphics cards, or even better, cloud computing resources. The processing time is cut from days to weeks to hours as a result.

Of course creating deepfakes requires a wide set of skills. To edit finished videos to remove flicker and other visual flaws are great examples. 

Nevertheless, a variety of technologies are now accessible to aid in the creation of deepfakes. You can get them made by a number of businesses, who will handle all the processing on the cloud. Even a smartphone app called Zao allows users to add their faces to a database of TV and movie stars that the system has been trained on.

7. How to identify deepfakes?

Although it might sound like an easy task, the truth is the difficulty increases as technology advances. 

A few years ago, back in 2018, the world saw the discovery by US researchers that deepfake faces don’t blink regularly. It makes sense, right? The algorithms never really learn about blinking because the majority of photographs feature people with their eyes open. 

It initially appeared to be the perfect solution to the detection issue. But as soon as the research was publicized, deepfakes started to appear. The game is designed in such a way that any weaknesses are immediately rectified.

Deepfakes of lower quality are simpler to detect, though. The skin tone may be uneven, or the lip synching may be poor, some faces may even have flickering around their edges. 

Besides that, deepfakes find it particularly challenging to reproduce small details like hair, especially when individual strands are visible on the fringe. Poorly produced jewelry, teeth, and unusual lighting effects like iris reflections and irregular illumination can all be telltale signs.

Of course the matter has already captured the attention of governments, academic institutions, and tech companies. At the time some companies have even ran deepfakes detection challenges as a way to cope with this new technology.

8. Now what?

Well, AI, ironically, might have the key. Artificial intelligence has already made it easier to identify fraudulent videos, but many of the detection tools currently in use have a critical flaw: they tend to favor celebrities because they can train on endless amounts of publicly accessible data. 

Tech companies are currently developing detecting systems to alert users if deepfakes are detected. A different approach is based on the media’s lineage. Although a blockchain online ledger system might store a tamper-proof record of films, photographs, and audio so their origins and any manipulations can always be confirmed, digital watermarks are not failsafe.

In no way are deepfakes evil or should be considered malicious. Many are actually beneficial. 

Deepfakes can bring voice back for those who have lost it due to a health condition, they can also animate museums and art galleries, and of course, they can also be widely used in the entertainment industry. These are all positive outcomes this new technology could potentially have. Yet, it is key to continue to understand what’s behind deepfakes to move towards a safer landscape. 

Further research and regulations are still needed.

Keep reading: What Is the Dark Web?