IBC
article here
Google’s generative filmmaking tool Veo 3 sparks Hollywood exploration but creative ownership remains in flux
Barely a year after initial release Google DeepMind’s AI text-to-video technology Veo has reached version 3 and by all accounts it’s a gamechanger in auto-generating cinematic content. So much so that studios Sony Pictures, Warner Brothers, Disney, Paramount and Universal are clubbing together to make a short AI film to learn what the tech can do.
The experimental project, titled The Bends, is produced by the Entertainment Technology Center, a think tank and research body based at the University of Southern California, which is sponsored by the studios and tech companies including Epic Games and Google.
“We really see the momentum changing in AI right now,” says Eric Weaver, Head of Virtual & Adaptive Production at ETC. “It's gone from being a feared thing to ‘how can it actually help me as a tool?’”
The Bends is a purely generative AI film (about a blobfish) written and directed by young freelance filmmaker Tiffany Lin and is one of two AI driven projects funded by ETC. The other, called Pathways, explores a hybrid workflow and probes “how you film a live action performance and then use AI to change everything in the background?” Weaver explains.
Roberto Schaefer, ASC the Bafta-nominated cinematographer of Finding Neverland is a consultant on The Bends. “We got early access to Veo 3 and it is leaps and bounds ahead of what we were struggling with before which was to get coherent imaging that we could use in a cut,” he reports.
“The studios want to learn what the limits are, what they can do, how it can work, and they don't have to worry about the copyright on this because these are not shows which are made for commercial distribution just for education.”
Veo 3 is said to excel at translating complex prompts into realistic videos outputting 1080p resolution. It can generate synthetic humans with lifelike features, such as with five fingers per hand. New camera controls help to define precise camera movements, including rotations, dollies and zooms. And unlike Sora, OpenAI's video generator, Google’s software can simultaneously generate audio including dialogue with accurate lip-syncing from a single text prompt.
“The prompts are much more accurate,” Schaefer confirms. “I haven't seen any real hallucinations nor any real issues with continuity of image or of faces. You can create more structured sequences from a single prompt up to 8 seconds long which is good but I believe it’s going to advance to two minutes long per prompt. That’s like a Steadicam shot. These are long takes that you don't use on most shows so it's a luxury but fascinating.”
What’s more, the fidelity of human simulated characters leads Schaefer to believe the tech is on the verge of crossing the Uncanny Valley. “We're quite there yet, but almost. For TV commercials I think they already pass muster.”
A YouTuber called Revid recently posted a commercial made in Veo 3 for a fake pharmaceutical company to demonstrate the new economics of production. He says he used to shoot commercials with $500k budgets and 50 crew taking two months for pre-production, shooting, and post.
“Today, I just created the same quality commercial for $500. In under 24 hours. By myself. Using Veo 3, I went from concept to finished commercial faster than it used to take me just to get permits approved.”
Revid explains that ChatGPT helped brainstorm and refine the script ideas, then he generated 13 different shots using nothing but text prompts.
“This isn't just a cost reduction - this is a complete transformation of how content gets made. We're witnessing the democratisation of high-end video production in real time. If you can create broadcast-quality commercials for $500 in a day, why would any company still spend half a million dollars and wait months for the same result?”
Shortform is one thing but surely it’s only a matter of time before AI longform films hit the internet. Where does that leave the craft of the cinematographer which is built on practical, skilled appreciation of light?
“That's why I'm doing these AI projects with the ETC to understand how to work with it because AI is inevitable,” Schaefer says. “They are still going to be making movies with real sets, actors and cameras but DPs like me need to get to grips with AI.”
Just as the multi-Oscar nominated Roger Deakins (No Country for Old Men) was hired as a visual consultant to Pixar’s WALL-E (2008), Schaefer (who also shot Monster's Ball, The Paperboy, Quantum of Solace, Amazon series The Peripheral and Geostorm) is lending his expert eye to help render the imagery and storytelling more cinematic.
“An AI film is basically an animated film, whether it looks photoreal or looks like a cartoon,” he says. “There is a role for DPs like me to help craft the types of shots and the lighting and colour. Basically, to help AI filmmakers to tell a story with images.”
To that effect he has compiled a glossary of terms for camera movement, lens type, focal length and other technical descriptions. “All the different terms and knowledge that cinematographers use commonly on set, I'm giving to the writer director on The Bends to use for the prompts.”
One idea is train the AI (Veo 3 or others) to ‘understand’ the filmmaking vocabulary to aid AI filmmakers to better create cinematic imagery “rather than have the AI come up with some crazy terms based on computer logic.”
Schaefer acknowledges that by feeding his expertise into the machine it may eliminate any further need for directors of photography down the line.
“The machine is going to get better and better at understanding light and dark and shade and framing and all the rest of it,” he says. “Sooner not later we're going to get a completely AI generated long form feature.”
There’s a school of thought that low budget ‘kitchen sink’ indie drama from Ken Loach, Mike Leigh or Sean Baker will remain untouched by AI. While those filmmakers may eschew the technology, there’s nothing to stop a machine from mimicking their work.
“I think people are going to start making fake gritty documentaries and docu-style films,” Schaefer says. “I’m not saying we have to bow down to it, just that the visual proficiency of the software is that extraordinary [it will be hard to distinguish factual shooting of real people from fiction].”
The biggest stumbling block for the studios to use AI for image generation on commercial unionised production is copyright.
Google says it has partnered closely with the creative industries to “help shape” its models and announced one such partnership with director Darren Aronofsky. No stranger to fronting new tech, Aronofsky also made a promo film to showcase the Las Vegas Sphere.
Google claims “responsible creation and collaboration with the creative community” basing this on SynthID a watermarking system to help identify over 10 billion assets as AI-generated. It has just launched SynthID Detector, a verification portal to help people identify content watermarked by SynthID. While this may reduce the chances of misinformation and misattribution it is not clear how this resolves the issue of plagiarism on models which have presumably scraped the internet with no traceable link or reward back to the original author.
“The only company with any kind of conscience is Adobe,” Schaefer says. “Certainly, studios like Disney are developing their own in-house AI models based on their own archive but that maybe just too limited. Obviously, they can still hire artists to generate a new face, a new background, a new something to create a whole universe using only elements they own.”
Speaking at AI filmmaking conference AI On The Lot in the heart of Hollywood this week, Pinar Demirdag, Co-founder & CEO of AI platform Cuebric argued that innovating with AI and preserving human centric art are not mutually exclusive. “After photography was invented, painting didn't go away. After colour was introduced, black and white didn't go away,” she said. “It became a choice. So our choices of expanding our possibility for creation will be immensely vast. But I didn't drink the Kool-Aid. I'm very aware of the dangers of normalisation of mediocrity that AI can bring us. At the end of the day, we will be pushing a button and some options will be given to us, but we somehow collectively agree about what is good content, what is good art, and what is not.”
The Bends is planned to be finished at the ETC-USC by the end of June with feedback to follow.
No comments:
Post a Comment