Generative “Eno” Documentary Reshapes the Film for Every Viewing

NAB

A randomized documentary of the career of legendary techno-music pioneer Brian Eno, in which every screening is potentially and infinitely different, is the latest project to be served up by generative AI.

Eno is billed as the world’s first generative cinematic documentary. “Like a musical performance that’s different every night, the film creates a unique viewing experience for each audience that takes it in,” explains Matt Grobar at Deadline.

article here

The 75-year old British music producer and visual artist who has worked with David Bowie, U2, Grace Jones and Talking Heads, and who birthed the ambient music genre and frequently mixes technology with art, is ripe for a video retrospective.

“I usually can’t stand docu-bios of artists because they are so hagiographic,” Eno told Variety’s Todd Gilchrist.

So, rather than charting a chronological path through Eno’s career, documentarian Gary Hustwit proposed using a generative system to create a film that would literally be different for every audience that screened it.

“The use of randomness to pattern the layout of the film seemed likely to override any hagiographic impulses,” Eno said.

If that was enough to pique Eno’s interest in the project, for Hustwit the approach was about provoking new ways of creating and experiencing a film.

“I like movies where you learn different things about the subject, but you, as the viewer, make the connections… I always think that’s a lot more rewarding, as a viewer. It’s a different kind of filmmaking, but it’s also a different kind of film watching.”

It helps that the first and last scenes of the 85-minute doc are always the same. Plus, there are certain scenes pinned to the same timeslot in each version, including a scene where Eno discusses generative art.

“We thought that was probably a good scene that everybody should see,” Hustwit told Lauren Forristal at TechCrunch.

Everything else, however, can be different, depending on the material the generative program decides to insert.

“It’s kind of a modular approach,” Hustwit explained to Forbes’ David Bloom. “You can learn different facts about that person at different times in the film. In the end, you make the connections as a viewer.”

Like one of media artist Refik Anadol’s AI creations, Eno is going to be different each time it is screened. That poses a problem for film critics, Bloom points out.

To DeadlineHustwit explained, “There are billions of different combinations that could possibly exist of this movie, and every time you watch it, you’ll never see that version again. So, it’s an interesting experiment. We can change the way that the form of film works [so] let’s talk about the possibilities.’”

Hustwit had another reason for making the film this way too. It’s a showcase for the generative tool (cutely dubbed Brian One) that he has built along with digital artist Brendan Dawes by their startup company Anamorph.

The tech was trained to select scenes from over 500 hours of archival footage and new interviews of Eno as well as animated visuals and music to produce the unique iterations of the doc.

Anamorph spent five years building the software, combining patent-pending techniques with the team’s own knowledge of storytelling. The company says it’s not trained on anyone else’s data, IP or other films.

“The main challenge was creating a system that could process potentially hundreds of 4K video files, each with its own 5.1 audio tracks, in real time,” Dawes tells TechCrunch. “The platform selects and sequences edited scene files, but it also builds its own pure generative scenes and transitions, creating video and original 5.1 audio elements dynamically. The platform also needed to be robust in a live situation, it wasn’t an option to have it crash. So, we did a crazy amount of testing. We can create a unique version of a film live in a theater, or we can render out a ProRes file with its own 5.1 audio mix and make a DCP from that.”

He also stresses, “This is a generative system, not generative AI. I just need to make that clear, because pretty much everything that’s been said about Eno uses the word AI.”

Advertising agencies have apparently expressed their interest, Hustwit reveals to TechCrunch, with one company wanting to make 10,000 versions of a one-minute commercial.

Rather than make its tools publicly accessible, the company wants to collaborate on projects so it can “consider the source material and the overall story goals,” says Hustwit.

“Our main goal is to get the idea out about this new kind of cinema and hook up with great collaborators to help explore this idea.”

Hustwit ponders what an experimental form-pushing director like Jonathan Glazer (The Zone of Interest) could do with something like this.

“You could make a movie that’s always on, always evolving, always changing,” Hustwit told Forbes. “I feel like Eno, it’s really kind of an opening conversation. What’s next? What can we do with this?

A streaming service such as Netflix — which has played with interactive forms of video — could easily generate a different version of the documentary every day, Hustwit added.

However, to TechCrunch he poured cold water on the idea, saying that streaming networks aren’t equipped to dynamically generate unique video files and stream them to thousands of viewers so that each viewer is getting their own version of a movie.

“When we premiered Eno at Sundance, all the big streaming companies loved it, but they also admitted that their systems can’t handle the tech involved… These streamers need to differentiate, and I think enabling the films and shows they’re releasing with generative technology is a way to do that,” says Hustwit.

It’ll likely take years before streaming services adapt to the technology. Until that happens, Anamorph is sticking to live events and theatrical releases.

“Something that the theater industry badly needs right now is a reason to get people to come in, and if there is a uniqueness about the live cinema experience, that’s one way that can be achieved,” he adds.

Six versions of the film will be shown at Sundance, with additional screenings across 50 cities to be presented later this year.