December 8, 2021

I QUESTION THE PREMISE: The Beatles: Get Back shows that deepfake tech isn’t always evil.

What happened next is quite astonishing. Jackson had the grainy, desaturated film fed through a computer algorithm. Suddenly, the resulting video had bright, vibrant colors, with sharp images that looked like it was filmed yesterday, not in 1969. But even more impressive than the film restoration, which is a technical triumph in and of itself, is what they did with the audio. Per Jackson, in his Variety interview:

“To me, the sound restoration is the most exciting thing. We made some huge breakthroughs in audio. We developed a machine learning system that we taught what a guitar sounds like, what a bass sounds like, what a voice sounds like. In fact, we taught the computer what John sounds like and what Paul sounds like. So we can take these mono tracks and split up all the instruments we can just hear the vocals, the guitars. You see Ringo thumping the drums in the background but you don’t hear the drums at all. That that allows us to remix it really cleanly.”

The machine learning technology used here is very similar (if not identical) to what has been used in the past for deepfakes, making fake video look and sound real. A prime example of this is the Emmy Award-winning demonstration video produced by MIT’s Center for Advanced Virtuality, “In Event of Moon Disaster,” which depicts then-president Nixon reading a prepared statement that the Apollo 11 astronauts had perished in a catastrophe. To create it, MIT used Nixon’s likeness and speech from television appearances and fed it into a machine learning system to synthesize the audio and video and produce the uncanny film.

Uncanny is the right word — the video of deepfake Nixon reading William Safire’s proposed speech has that uncanny valley feel that Admiral Tarkin and Princess Leia had in the 2017 Star Wars prequel, Rogue One. The technology is making remarkable strides, yet you still know you’re watching a “synthespian” — but going forward, this technology is only going to get more and more realistic:

(Granted, simulating the fuzzy “Never Twice the Same Color” look of a live 1969 American TV broadcast helps to hide some of the flaws, unlike seeing Rogue One on a 25 foot tall movie screen.)

However, I’m not sure if the restoration techniques used to restore the audio and 16mm film from the 1969 Let It Be sessions counts as deep fakery. Jackson used even more powerful digital techniques to restore the scratchy, wiped out 1918-era WWI footage for his 2018 documentary, They Shall Not Grow Old. For Get Back, he had well-preserved 16mm color footage to begin with, giving the restoration techniques used on his WWI documentary that much more of a head start.

For the sound, once the band moves into their basement Apple studio, Jackson had a mix of the mono audio the filmmakers were recording of the group between songs, and the band audio was recorded on eight track reel to reel by Glyn Johns, one of rock’s best engineers (and in 1969, one of its wildest dressers).

To remix the mono audio of the band rehearsing in Twickenham Film Studios, Jackson used a version of the remix applet that’s included in several audio restoration programs, such as Izotope’s RX9:

Jackson was lucky that Billy Preston hadn’t arrived yet, since the current platform doesn’t separate keyboards from guitars. Perhaps the most powerful audio restoration was done in the scene where John and Paul discuss George quitting in the Twickenham cafeteria. But even with all of the sound restoration technology available to both Disney, and the director of the Lord of the Rings movies, he still had to subtitle the conversation for the viewer to understand what’s going on:

So it’s all very powerful digital audio and film restoration technology, but nothing in Get Back seems like we’ve entered into deep fake territory. (Unless you were a “Paul is dead” believer in 1969, that is…)

InstaPundit is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to