Impact case study database

The ability to model 3D faces in speech from video alone (described in R1-R3) eliminated the need for complex capture studios and 3D scanners, and became the cornerstone for generating ‘synthetic media’ such as AI-generated realistic, human-like avatars. Agapito co-founded Synthesia with other researchers and entrepreneurs to provide commercial solutions for a range of applications for this new technology, from lip-sync dubbing for content localisation to personalised video messages and corporate training. Synthesia technology allows users to create professional looking videos by simply typing a message, using an automated, 3D-driven AI process to synthesize photorealistic results that are indistinguishable from real video but without the need for cameras, actors or expensive film studios.

Agapito’s research has 1) enabled the commercial viability and rapid growth of Synthesia; 2) benefited Synthesia’s customers by providing new and more cost-effective services; and 3) increased public understanding of synthetic media through its high-profile work.

Impact on enabling the commercial viability and rapid growth of Synthesia

Regarding the impact of Agapito’s research on founding Synthesia, CEO and co-founder, says: “The algorithms proposed in her research for accurate 3D non-rigid shape estimation from video based on photo-consistency, were pivotal to the creation of Synthesia and now photometric tracking lies at the core of our technology. The ability to capture the 3D geometry and appearance of a human face in speech from a single video with algorithms building on Agapito’s research has been transformational in allowing us to build a low-cost solution to create high fidelity 3D avatars of humans for animation and synthesis” ( S1). It is this breakthrough in capturing 3D geometry from short video clips or even a single image which allows the low-cost, controllable synthesis that is so valuable in applications.

3D understanding means that the actions of the synthesized faces can be fully decoupled from the input clip, and controlled by smart software, to open up the new applications which are at the heart of Synthesia’s rapid growth. As the CEO continues: “By making it easier to reconstruct 3D photorealistic faces, the algorithms permitted Synthesia to provide new services to clients” ( S1). Synthesia has transitioned from offering high profile video-to-video services towards offering a “Software as a Service” platform where users can create videos simply by writing the speech of the digital ‘actor’. This technology has a wide range of applications, from corporate training to in-house communication to sales. As such, Synthesia’s services have been used by diverse clients, including Reuters, WPP, Dixa, Just Eat, Tesco, FedEx, Facebook and Google.

With this technology in place, Synthesia was extremely well placed to grow rapidly during the recent boom in the use of online services. Synthetic media has become a cheaper and quicker way to produce video, an advantage amplified by the Covid-19 pandemic. With studios and other facilities out of action, companies with needs in corporate communication, training or advertising used synthetic video generation for the first time. In an interview with TechRepublic, the CEO said, “Using AI, we've digitized the video production process and enabled our customers to create a video in 5 to 10 minutes, without the need for any cameras, actors or studios" ( S1).

During 2020, [TEXT REMOVED FOR PUBLICATION]. As a result of its world-leading technology, Forbes magazine named Synthesia one of its “fearless five” Tech companies ( S3).

Reshaping famous faces for global commercials and charity campaigns

Synthesia has provided the technology behind many of the most high-profile uses of photorealistic video face synthesis, as the ease of use and low cost of this technology were combined with the need for extremely high professional standards of quality and ethical use.

For example, the company completed a highly successful project for the food delivery company Just Eat. After filming a campaign with the rapper Snoop Dogg in 2020, the company wanted to extend it to its Australian subsidiary MenuLog. However, rerecording the advert with the new name would have been prohibitively expensive. By using Synthesia’s rendering, they were able to simply edit the original, reaching an entirely new audience without making a new advert, which created substantial savings for the firm. In addition to saving significant costs for the firm, this advert was crucial in helping MenuLog reach new audiences, and the campaign went on to receive over 10,000,000 views ( S4).

The technology was also crucial in a 2019 Malaria No More campaign that raised USD14,000,000,000 to help end the world’s three biggest preventable killer diseases: AIDS, Tuberculosis and malaria. It was used to make David Beckham speak 9 languages as part of a 2019 campaign video, and because this video localised the campaign to suit specific global audiences, it created 700,000,000 online impressions and resulted in the disease’s peak awareness in almost 3 years. The campaign attracted over 1,800 pieces of media coverage, resulting in internet searches for malaria reaching an all-time high ( S5). It won marketing awards, as well as an award for social good in AI, and was instrumental in winning commitments from world leaders to increase funding for fighting malaria. As one of the Malaria No More team said, “ This magic wouldn’t have been possible without the talented team at Synthesia” ( S5).

Cutting costs and increasing engagement for training videos and online assistants

Synthesia’s development of a highly automatic and scalable SaaS platform, which is key to its current growth, has enabled easy-to-use Text to Video services which customers can use to generate video of synthetic actors as easily as typing text.

For example, the multinational communications company WPP used Synthesia in 2020 to provide their corporate training videos in multiple languages, without having to reshoot using different actors and scripts. As WPP’s chief technology officer told Wired magazine, this saved them a considerable amount of money: “ A company-wide internal education campaign might require 20 different scripts for WPP’s global workforce, each costing tens of thousands of dollars to produce. With Synthesia we can have avatars that are diverse and speak your name and your agency and in your language and the whole thing can cost USD100,000” ( S6). The former Global Director of WPP said that “Synthesia allowed us to transform the way we think of training materials” ( S6).

Life Extension Europe, a nutritional supplement supplier, utilised Synthesia to improve its online sales. The success in this approach was reflected in the increased time that visitors spent on their webpages; for instance, with the new videos, the average session duration in the UK increased to 9 minutes 37 seconds from the previous average of 3 minutes and 36 seconds, an increase of +167%. Similarly, the average number of page views per session increased to 6.16 (versus 4.24 previously), an increase of 45.3%. The campaign recorded an additional 101 transactions ( S7).

Improving public understanding of and engagement with synthetic media

The final impact of Agapito’s research, via Synthesia, has been increased awareness of the positive potential of synthetic media. While there has been widespread concern about ‘deepfakes’ and the potential for misinformation, Synthesia has used Agapito’s research to demonstrate the many beneficial applications of synthetic media. Their work, and the code of ethics they have developed for the use of their work, has been widely covered in news media in 2019 - 2020, illuminating ways in which the changed landscape of AI-generated video can be regulated and turned to socially responsible ends. For instance, the MIT Technology Review praised Synthesia for only working with vetted clients in its 2019 article, “Making deepfake tools doesn’t have to be irresponsible. Here’s how.” Similarly, TechCrunch expressed excitement that Synthesia’s products “ could also be used to expand the reach of creators around the world” ( S8).

Synthesia’s code of ethics includes a commitment only to create synthetic video of people who have given their explicit permission. This debate has also been promoted by other industry leaders such as Samsung, who have highlighted Synthesia’s work in their 2020 list of “5 companies leading the creation of AI-enabled images & videos” ( S9). As the Managing Director of Samsung Next (Samsung’s synthetic media initiative) said in relation to the Synthesia’s work, “it looks like AI will actually democratize creativity” ( S9).

In addition to raising awareness of the potentially positive impact of AI, Synthesia created engaging AI videos for public consumption, such as the Synthesia Santa, which allowed users to easily create a video from Santa speaking their text to friends and family. The site made 90,000 cards in the first three weeks of launch, and in September 2020, Synthesia was voted the #2 product by Product Hunt ( S10).