Last week, I watched one of the most inadvertently hilarious documentaries I have ever seen on YouTube. But before I get to it and expose you to the full horrors, some background on one of the trends in AI vocals — using artificial intelligence to replace the voiceovers in a foreign documentary with an English-speaking narrator using an American or British-sounding accent. It’s becoming cheaper to employ AI than a professional voiceover artist, as this 2021 Washington Post article titled, “Every movie and TV show could soon be dubbed into any language you want” hints:
“We have the technology to fill a big gap,” said Oz Krakowski, chief marketing officer of deepdub, a Dallas- and Tel Aviv-based start-up that employs essentially this process. “We can give studios what they want and give consumers a totally unique experience.” The company is about to put its claim to the test. It is releasing “Every Time I Die,” a 2019 thriller whose English-language version is on Netflix, in Spanish and Portuguese versions dubbed entirely by AI.
Auto-dubbing companies take a range of approaches. Deepdub focuses on the audio, digitally redeploying the original actor’s voice off a machine translation, but leaving the video unchanged. Another firm, London-based Papercup, doubles down on this tack, using so-called synthetic voices.
Flawless goes the other way, relying on live (and labor-intensive) voice actors but editing on-screen lips and faces so they look like they’re actually speaking the language. All three companies bring humans into the process at various points for quality control.
* * * * * * * *
Venture capital firms have started to bet on auto-dubbing. Papercup’s executives said that in December they raised $10.5 million from a group of investors that included Arlington, Va.’s Sands Capital Ventures and media companies like The Guardian on top of several million raised previously. Flawless recently concluded its undisclosed Series A financing; deepdub is in the middle of such a round.
It’s easy to understand their interest. Foreign-language content is a vast unmined frontier for Hollywood. Netflix’s “Squid Game” has become the service’s No. 1 show in many countries, including the United States. If the Korean survival drama can do this largely with subtitles, the auto-dubbers say, imagine what would happen with on-demand dialogue? An endless parade of foreign-language Stateside smashes is not hard to conceive.
(There are already some dubbed versions of “Squid Game,” but they’re…not so well-received.)
As a kind of new spin on the Tower of Babel — everyone speaks different languages but still understands each other — auto-dubbing also means non-English speakers won’t need to learn English to grasp the dialogue in a Hollywood movie. (Subtitles, presumably, would fade away.)
But these rich possibilities also come with concerns.
And how. As Mark Knopfler sang in Dire Straits’ 1982 song “Industrial Disease,” “Warning lights are flashing down at quality control.” Or no quality control whatsoever, in the case of a documentary recently translated by AI into English that the YouTube algorithm served up to me last week.
Titled “1940-1944, Paris During the Occupation: The Untold Story of the German Soldiers,” it’s a one-hour-and-forty-minute look, originally shot for French TV, at how German soldiers spent their days in occupied Paris until it was liberated by the Allies. Ordinarily, this would make for some harrowing viewing, but the end product is rendered absolutely ridiculous by the number of gaffes and mispronunciations the AI narrator makes.
It’s a shame, because at first, “his” synthesized voice sounds very assured; similar to the narrators I remember on numerous Discovery channel documentaries in the 1980s and ‘90s. But because no one checked the AI voiceover audio before the video was uploaded (or worse, no one cared), it’s only a matter of time before “he” is saying phrases such as “Folly-Burger” instead of Folies Bergère. Rommel’s feared Afrika Korps are dubbed “the Africa Corpse.” (Obama no doubt sympathizes.) The Protestant Choir becomes the Pro-Test-Tent Choir. The Boche become the Bo-chaaays. The Arc de Triomphe in the Champs-Élysées is suddenly the “Arch de Triumph-eeee” in the “Chaaamps Elessseees.”
I could go on and on (I spent a fun-filled hour and a half typing up all of the gaffes I spotted, and those who can actually speak French and/or German will likely find even more than I did), but you get the general impression. Skynet may one day take over the world, but at least for now, if this lost-in-translation documentary is any guide, flesh-and-blood human narrators are still best:
Join the conversation as a VIP Member