In the past year, the term “deepfake” has entered into news headlines and the realm of popular culture. If you’ve run into articles like “Deepfakes are getting better. Should we be worried?” and “Internet Companies Prepare to Fight the ‘Deepfake’ Future” you might be feeling apprehensive or even wondering what, exactly, a deepfake is. In the era of fake news where content can spread through the internet like wildlife, this emerging technology is sparking new debates and driving new experiments.
Well, what are deepfakes? According to CSO Online, on a basic level, deepfakes “are fake videos or audio recordings that look and sound just like the real thing.” The term “deepfake” first emerged on Reddit in 2017, but it has been widely adopted to describe eminently realistic video content that is, in fact, falsified. Now, deepfake serves as “a catchall for manipulated video.”
What’s behind the creation of these videos? Artificial intelligence. Deepfakes are created using increasingly sophisticated deep learning technology that allows you to edit faces in videos.
The creation of deepfakes is no longer limited to advanced technologists and computer scientists. Recently, consumer apps that employ artificial intelligence have made deepfakes accessible to the masses. In the past few months, a convincing deepfake app has gone viral, prompting a wide range of predictions about its imminent use cases and risks.
Much of the news following the rise of deepfakes has focused on their more sinister implications, especially as a widely-available consumer product. Because of its potential to be harnessed to produce false propaganda and defame political opponents, deepfake technology has recently been banned in California during election season. Additionally, this week, Facebook has countered the abundance of deepfakes by launching a challenge for researchers to produce the most accurate deepfake detection solution.
While recent headlines have focused on problematic deployments of deepfakes, like all technology, it has benefits as well as risks. If you’ve been getting yourself into a frenzy about the less-than-well-intentioned use cases of deepfakes, it might help to know that deepfake technology has enormous potential for good.
In the realm of arts and culture, deepfakes are being used to produce art, engage audiences, and provide personalized experiences to visitors in a way that has never been done before. We’re here to explore some of the compelling, positive implications of this technology for artists, museums, cultural attractions, and more.
Here are three things you need to know about deepfakes in arts and culture:
1. Technologists Are Already Deepfaking Famous Artworks
There is no shortage of inspiration at the intersection of art and technology, and art is often used in early concepts and experiments to provide a humanistic touch. In this vein, it comes as no surprise that technologists are deploying deepfake technology to reimagine famous artworks.
Take, for example, the Mona Lisa. This year, researchers from Moscow’s Samsung AI Center and Skolkovo Institute of Science and Technology have taken portraits such as da Vinci’s “Mona Lisa” and manipulated them to talk and move as though they are real people. Far more advanced than simple animation, the process of bringing the Mona Lisa to life has involved identifying “facial landmarks in the painting” with an advanced algorithm.
Source: Moscow’s Samsung AI Center and Skolkovo Institute of Science and Technology
This early prototype of Mona Lisa brought to life has gone viral, and for good reason. Paintings like da Vinci’s are shrouded in mystery and popular intrigue that has fueled fascination, examination, and debates for centuries.
Source: © Johan De Smet via BOZAR
The same technology was used to animate other famous works, including Gustav Klimt’s “Portrait Of Johanna Staude,” which exists in the collection of Vienna’s Belvedere Palace.
Source: Moscow’s Samsung AI Center and Skolkovo Institute of Science and Technology
Other works include as Vermeer’s “Girl with a Pearl Earring,” and Kramskoy’s “Portrait of Unknown Woman.”
Source: Moscow’s Samsung AI Center and Skolkovo Institute of Science and Technology
As these researchers have illuminated, deepfake technology offers new potential ways to explore and entertain in the world of art and popular culture. If technologists are leveraging famous cultural imagery as a part of their nascent experiments, imagine what will happen when the next wave of fully intentional culture-centric projects start to take form.
2. Deepfakes Offer a New Tool for Audience Engagement
There is growing competition for attention, dollars, and loyalty from today’s consumer, and that goes for museums too. According to La Placa Cohen’s 2017 Culture Track study, a leading resource on trends in the cultural sector, there has been a flat or downward trend in cultural memberships and subscriptions. As this competition grows, cultural organizations have to think outside of the box to keep audiences coming through their doors and engaged when they get there.
How much more likely would you be to visit a museum if you could have a conversation with a legendary artist? Or be greeted by a long-gone historical figure upon entering the gallery? That’s where deepfakes can lend a helpful hand.
Now, the Dalí Museum in St. Petersburg, Florida offers this opportunity to their visitors. In the exhibition Dalí Lives, an AI-driven video editing and rendering technique was used to produce a life-sized recreation of Dali out of thousands of hours of archival video footage.
Source: Dali Museum + Goodby Silverstein & Partners
According to the Verge, “Dalí appears before visitors when they press the doorbell on the kiosk where he lives, and he tells them stories about his life.”
Source: Dali Museum
According to Nathan Shipley, the technical director for the project, Dalí Lives “may be the first time a cultural institution has used deepfakes for artistic purposes,” but this certainly won’t be the last! The realistic Dali has delighted visitors across the board, highlighting the incredible potential of deepfakes to help the public engage with art, history, culture and more.
In the competition to draw visitors through your doors and attract new patrons, one of the most important things museums and attractions can offer is immersive experiences. For example, in recent years, we have uncovered the power of 3D printing to recreate artifacts for audiences to touch and observe, as well as witnessed the emergence of pop-up museums and the new era of “Instagrammable” interactive experiences like the Meow Wolf, Museum of Ice Cream, and teamLab.
Deepfakes are rapidly emerging as another tool in a suite of technologies that can make the museum experience more interactive, relevant, and engaging for audiences, especially those who crave something new.
3. Deepfakes Enable Contemporary Artists to Create New Forms of Media
Within the past year, deepfake technology has not only become a tool to engage with existing art and past artists, but to create new and innovative forms of art and expression. Several artists have pioneered the use of deepfakes for creative promotion, including Turner Prize-winning British artist Gillian Wearing. This year, Wearing used a deepfake video to create a new extension of her exhibition at the Cincinnati Art Museum, “Life: Gillian Wearing.”
Source: Gillian Wearing
She’s not the only one. This year, Hao Li, an associate professor at the University of Southern California, has emerged as the world’s top deepfake artist. He now runs a startup called Pinscreen, focused on business and entertainment uses of deepfake technology.
Deepfake technology is also being used by artists to offer up a powerful social commentary. Over the summer, artists Bill Posters and Daniel Howe created a deepfake video of Mark Zuckerberg, as part of a conceptual art installation called Spectre, which was included in an exhibition called “Alternate Realities” at Site Gallery in Sheffield, England. The video offered up a critique of data security in the world of big data through the strikingly realistic likeness of Mark Zuckerberg.
Just like other emerging technologies like artificial intelligence and augmented reality, the use of deepfake technology as a tool for artistic creation is still in its infancy, and will more than likely bloom in the coming years.
Preparing yourself for the Deepfake future
In the age of fake news and internet hoaxes, it is important to stay informed and vigilant about the media you consume. Thankfully, there are countless efforts and tools to help with detecting fake or altered media as well as to preempt the misuse of deepfake technology. Knowing that it’s nearly impossible to run from these new technologies, we should be both open and informed about their potential, not just in the world of arts and culture, but in the world of engagement, entertainment, and nonprofit causes.
If you are still feeling anxious about the possible risks and dangers of deepfake technology, you’ll be happy to know that there are significant efforts to protect against their negative use. For one thing, they’re increasing easy to flag and detection algorithms are rapidly becoming available.
At the same time, their beneficial use cases keep expanding. When it comes to arts and culture, deep fake technology can be used by artists, museums, and organizations to engage the public in exciting new ways. Imagine if deepfakes weren’t just limited to one exhibit, but transformed the entire museum-going experience. What if all the paintings in the galleries or portraits of historical figures came to life? What if deepfake technology was coupled with AI-powered chatbots to deliver personal conversations with influential figures who died centuries ago? In the years to come, you might start seeing these types of experiences thanks to the underlying technology behind deepfakes.
Additionally, there is great potential to harness deepfakes or other forms of synthesized media for fundraising purposes. Just imagine receiving a customized video request for donation from your favorite Impressionist artist from your local art museum or a prominent figure from a history museum – how much more likely would you be to support that museum than if you simply received an email? Examples like these may take form in the near future and add to the ways in which organizations fundraise and communicate with their audiences.
In the years to come, deepfakes could emerge as an essential tool in arts and culture marketing, community engagement, and nonprofit development. This technology has the capacity to deliver enormous good – we look forward to watching its evolution and seeing the exciting ways museums and attractions leverage this new tool to engage their visitors and patrons.
Interested in reading Cuseum’s latest research? Download our 2021 Membership Insights Report.