AI company releases deepfakes, can it control them?

AI company releases deepfakes, can it control them?

The Rise of Deepfakes: Impersonating, Disinforming, and Beyond

Synthesia software

Erica, Dr. Dass, Jason, and Gary. These seemingly ordinary individuals have something peculiar in common – they’re not real. Well, at least not in the traditional sense. They are deepfakes, the creations of Victor Riparbelli, CEO of Synthesia, a London-based generative AI company. With around 150 digital humans at their disposal, Synthesia allows its clients to bring their scripts to life by simply typing in the text and pressing “generate”. The applications for these avatars are vast, ranging from corporate training to spurring disinformation campaigns or crypto scams. However, as the commercialization of synthetic media takes off, the ethical responsibilities of these AI creators and the platforms that distribute their content become paramount.

Synthesia’s journey to becoming a pioneer in the generative AI industry was not without challenges. For six years, Riparbelli and his team worked diligently outside the limelight, striving to create video content without actual camera equipment. Initially, not many investors saw the potential in their vision. But when ChatGPT emerged, an AI language model developed by OpenAI, the company received a major boost. In June, Synthesia announced a funding round that valued it at $1 billion, demonstrating a significant increase from its previous valuation of $300 million just two years prior. Riparbelli, now 31, attributes his fascination with technology to his upbringing in Copenhagen, where he delved into gaming and electronic music production.

The allure of Synthesia for investors lies not only in its cutting-edge technology but also in its vast library of avatars. These digital entities come in different genders, skin tones, and professions, including everything from hipsters to call center workers. The level of customization is astonishing, allowing clients to fine-tune their avatars’ language, accents, and expressions. However, the company’s ability to train its algorithms on footage of real actors remains a crucial proprietary asset. By understanding how humans move and talk, Synthesia seeks to create the most lifelike avatars possible.

As impressive as Synthesia’s technology may be, it is still a work in progress. Presently, the avatars are restricted in their movements, unable to express themselves fully. Moreover, the company has faced challenges in preventing these deepfakes from being misused. Over the years, some of Synthesia’s avatars, particularly one known as Jason, have been found impersonating news anchors on social media. These instances of spreading disinformation have sparked concern and raised questions about moderation and accountability.

In response to these issues, Synthesia has taken responsibility for the problematic videos and made adjustments to its practices. For instance, the company restricts news content to enterprise accounts, which undergo verification. To bolster content moderation efforts, Synthesia has increased the number of content moderators, employing a mix of humans and algorithms to identify and prevent the dissemination of harmful scripts. However, this approach highlights a crucial shift in the content moderation paradigm. Traditionally, moderation has occurred at the point of distribution, but the power of AI is propelling it to the point of creation.

While Synthesia’s terms of service may be more restrictive than some would prefer, they aim to strike a balance between allowing freedom of speech and preventing the misuse of technology for disinformation campaigns. As technologies like AI continue to evolve, content moderation must adapt to the changing landscape. For Riparbelli, the ultimate vision is to democratize video creation, allowing anyone to describe a video scene through text, with AI generating the desired outcome. He envisions a future where video creation becomes as customizable and interactive as the evolution of text-based content on the internet.

Achieving this vision hinges on getting content moderation right. Riparbelli wants synthetic video to follow the path of websites, which evolved from replicating print media to embracing multimedia capabilities. Personalization and interactivity are key aspects he envisions for the future of video content. Perhaps, instead of passive viewing experiences, video interactions akin to Zoom calls with AI companions could become the norm.

As the generative AI industry continues to expand, companies like Synthesia must grapple with the ethical implications and responsibilities that accompany their creations. While there may be challenges along the way, Riparbelli and his team remain committed to evolving their technology, always aiming for utility over novelty. As Sam Gregory, executive director of Witness, aptly puts it, “We’re reaching this moment where we need to decide what the responsibility is through the whole pipeline of image, video, audio creation.” The choices made today will shape the future of deepfakes and synthetic media, ensuring they are used responsibly, ethically, and for the betterment of society.