AI for M&E: Things Are Going to Get Complicated

NAB

Few industries will be more directly impacted by generative AI than Media & Entertainment and as it evolves in its second year, the battle lines are being drawn.

article here

Broadly, the battle lines will be fought in three areas: the legal right to use AI; between open source and proprietary AI tool developers in IT; and Hollywood Studios versus the legion of employees from A-list talent to production crew.

The resolution of the strikes both for actors and writers has only punted the issue a couple of years down the road. Ultimately, it would seem, M&E is going to be shaken up for better and for worse.

“The last decade in film and TV was defined by the disruption of content distribution and the next decade will be defined by the disruption of content creation,” summed up industry analyst Doug Shapiro in an extensive post on Medium explaining how all aspects of production would impacted.

In its special report, “Generative AI in Film & TV,” Variety found the tech already beginning to disrupt traditional methods, with generative AI tools currently used to automate some creative tasks. Its impact stands to be positive, Variety concluded, “as it eliminates rote work, speeds project timelines and allows productions to pursue previously impossible creative paths prohibited by constraints on cost, time and even physical reality.”

At the same time, Variety notes that its use promises to reduce the need for certain processes and workers to achieve the same level of output. Spelled out: That’s job losses.

Shaprio breaks down the costs of production costs for movies including the 50% of “below-the-line” crew and production costs of which 25-30% is post-production (and of this percentage, mostly VFX). All in all, roughly two-thirds of these costs are labor, he says. “It is a sensitive topic for good reason, but over time GenAI-enabled tools promise (and threaten) to replace large proportions of this labor.”

Practical use cases are already cropping up across all stages of the TV and film production process. These include story development, storyboarding/animatics, pre-visualization, B-roll, editing, VFX and localization services.

How far will this all go?  Even making the relatively conservative assumption that TV and film projects will always require both human creative teams and human actors, Shapiro says future potential use cases include: the elimination of soundstages and locations, the elimination of costumes and makeup and even “first pass editing.”

“In the future, it is likely that editing software will make a first pass at an edit, which can then be reviewed by a human editor,” he suggests. “Similarly, it’s easy to envision an editing co-pilot or a VFX co-pilot that could create and adjust VFX in response to natural language prompts.”

You can argue, as Shapiro does, that we have a “visceral negative reaction” to anything that’s supposed to look human but doesn’t, the so-called ‘uncanny valley.

“In which case we will still need human actors, possibly for a long time — but it would also mean that every other part of the physical production process would be subject to being replaced synthetically.”

All of this will likely have a profound effect on production costs. “Over time, the cost curve for all non-Above The Line costs may converge with the cost curve of compute,” is Shapiro’s possibly true if disheartening conclusion.

The potential for lower production costs would seem a silver lining for Studios but it also presents a daunting change management challenge.

“Studios should start either by experimenting with non-core processes or developing skunkworks studios to develop ‘AI-first’ content from scratch,” Shapiro says.

Peter Csathy in TheWrap thinks the major studios, faced with mounting Wall Street pressure to transform their business models, will begin to focus on generative AI “to increase output and cut costs.” Early experiments he suggests will include hyper-automation in visualization and initial uses of “Synthetic Performers.” 

Streamers like Netflix, “with Big Tech DNA coursing through their veins,” will lead the way, he says.

Legislation to Tackle and Protect

The EU and the US Congress as well as individual states at the federal level will pass significant AI legislation that directly impacts the M&E industry in the next 12 to 18 months. President Joe Biden’s recent Executive Order points the way.

“Congress will demand that the Big Tech companies behind GenAI give some basic level of transparency about the material on which their large language models are trained,” says Csathy. “Regulators will also try to get ahead of the game — a stark contrast to when they were largely absent when social media rose in popularity and importance (and caused significant harm).”

Csathy expects the creative community to do its best to keep AI companies honest by implementing so-called “forensic AI tech” like watermarking to identify whether relevant creative works were “scraped” or not. That in turn will promote “opt in” solutions for AI training.

Startups to Rival Big Tech

The battle between proprietary AI and open source AI is at its fiercest. Broadly speaking this is the battle between Big Tech and smaller start-ups and the battle is being fought in the market. Perhaps OpenAI/ChatGPT’s lasting legacy will be in opening up the first bona fide market for AI. In fact, AI — as the moving chairs at OpenAI have shown — is no longer controlled by scientists in the lab but by Wall Street.

CambrianAI analyst Alberto Romero, in his blog The Algorithmic Bridge on Substack, characterizes the debate like this: “The open-source scene is vibrant, full of enthusiasts who firmly believe AI shouldn’t be in the hands of the few and are working relentlessly to make their vision of a better, democratized world through AI a reality. They have detractors who think AI, as a (potentially) very powerful (and thus dangerous) technology, shouldn’t be available for anyone to use.”

He adds, “If the open-source community wasn’t pushing as hard as it is, closed businesses would capture all the value.”

Open source-based startups are also growing in number and in quality of output.

“They’re catching up with the best models, such as GPT-4,” says Romero. “While closed-source LLMs [like ChatGPT] generally outperform their open-source counterparts, the progress on the latter has been rapid with claims of achieving parity or even better on certain tasks. This has crucial implications not only on research but also on business.”

He thinks that the era of extremely large models dominating AI was just a phase and it’s coming to an end.

“Small and cheap is the future,” he says. “Open-source AI is becoming a powerful counterforce to Big AI as more people realize that this tech shouldn’t be in the hands of a few — it’s catching up.”

Csathy thinks Big Tech companies like Alphabet will try to have it both ways. “Desperate to keep up with OpenAI (and Microsoft) Alphabet will relentlessly march on with its AI development while trotting out its new SynthID watermarking solution to quell the creative masses,” he predicts.

“Alphabet throws these bones to the creative community, while its stock price rockets upward and the entertainment industry struggles to monetize amidst its continuing transfer of wealth to the Big Tech players that disrupt it.”