He doesn’t draw or paint nor animates in the traditional sense but give Joe Letteri a computer and he can figure out how to create any image in the world – or a world yet unknown. The four times visual effects Oscar and BAFTA winner has shaped three decades of visual effects and become a master of creature and character creation.
Working with Steven Spielberg, Peter Jackson and James Cameron,
Joe Letteri has built the astonishing photoreal worlds and iconic CG characters
of Jurassic Park, Lord of the Rings, King Kong,
the Planet of the Apes series and Avatar and is currently working on the highly
anticipated Avatar sequels.
“Understanding how and
why something happens, moves or works in the real-world gives you a deeper
understanding of what make something look real in a virtual world,” he
explains. “That has been the driving force behind the way I look at things. I
want to see how far you can stretch the techniques and the technology in
pursuit of realism.”
He grew up outside Pittsburgh, Pennsylvania, and excelled at
science, maths, astronomy and physics. Attending the University of California,
Berkeley, he gravitated toward media and film.
“I didn’t see a computer until I was at university,” he says.
“But when I began using them to evaluate complex maths problems I became
interested in graphics and how you could use computers to create organic shapes
like clouds and mountains.”
Around that time mathematician Benoit Mandelbrot began using
computer graphics to create and display fractal geometric images. “The infinite
possibilities of visualising things with equations and making pictures out of
data fascinated me.”
Letteri’s first job, at LA post house Metrolight, was designing
logos and graphics for commercials and TV stations. It was enough to attract
legendary VFX house ILM to hire him in 1990.
“This was my breakthrough. ILM were working on the opening shot
for Star Trek VI: The Undiscovered Country and
needed an explosion of the Klingon moon Praxis. They’d tried to do it as a
practical effect but couldn’t get it to work.”
Letteri deployed his knowledge of fractals and the simulation of
natural phenomena to design and animate an exploding ring of fire; “I was a
huge Star Trek fan. I couldn’t believe my luck that this was the first shot I
ever produced.”
Up next was Jurassic Park, the seminal
picture that ushered in a new age of CGI. Spielberg had intended to use stop
motion animation for the dinosaurs but changed his mind when shown a test of a
CG T-Rex for which Letteri, working under ILM’s VFX supervisor Dennis Muren,
helped establish a new level of realism.
“The idea that
you could actually write code [in Pixar’s 3D software RenderMan] to get the
skin to look exactly how you wanted and how it reflects light was a revelation.
But it wasn’t just about how real we could make [the dinosaurs] it was also
about fitting them into the photography. We now had questions to answer about
how these CG characters work dramatically within the context of the story.”
Computer graphics was emerging as a tool which combined
photography and optics alongside with disciplines like biology for artists to
construct increasingly more complex 3D creations. Letteri learnt
cinematographic lighting techniques, drawing on his understanding of the
physics of light transport, to make characters “believable for a story or
setting that never been told or seen before.”
His credits at ILM include Casper, Mission:
Impossible and the 1997 special edition of Star Wars.
Pixel by pixel
“Unlike special effects, where you built a model and moved it and photographed it, now you could manipulate the pixels directly,” he says. “It opened the door to a whole different way of thinking about the art, in that anything you could describe you could capture pixel by pixel.”
“Unlike special effects, where you built a model and moved it and photographed it, now you could manipulate the pixels directly,” he says. “It opened the door to a whole different way of thinking about the art, in that anything you could describe you could capture pixel by pixel.”
Jurassic Park’s success in making CG characters integral to the
story had far reaching effects, ultimately inspiring director Peter Jackson to
begin planning The Lord of the Rings series.
In pre-production for the trilogy in 2001, Jackson hired Letteri to join his
fledgling studio Weta Digital in Wellington and create Gollum.
“Gollum was the big attraction for me,” says Letteri. “He is one
of literature’s great characters. What we needed to do was to make his skin
look soft and translucent like human skin, something that hadn’t been done
before.”
He devised a subsurface scattering technique to create the
effect and also helped pioneer advanced performance capture with actor Andy
Serkis, making Gollum the first character to appear alongside live-action
characters in a way that didn’t differentiate between the two.
“Creating digital characters with that emotional range and
subtlety has been the basis for all of the characters we’ve created since,” he
says.
Performance capture
evolution
The
original contract with Weta Digital was for two years but Letteri never left.
Now 61, he is its director, a shareholder, and responsible for all the work
that goes that the studio generates including X-Men: The Last
Stand, The Day the Earth Stood Still and I Robot.
Under his supervision, performance capture has evolved from a
post-production process to a real-time one, allowing a director to shoot actors
and digital characters and environments as if they are filming a live-action
movie.
A highpoint for this was Avatar(2009),
the landmark stereo film of the modern era. “Jim [Cameron] gave us the
opportunity to take stock of everything we we’re doing and question all the
assumptions of what worked and what didn’t,” Letteri recalls. “We built digital
assets and environments, then combined these with the digital puppets being
driven by the motion capture. It wasn’t just making believable CG characters it
was the scale, detail, and complexity. We had to create an entire planetary
ecosystem, with waterfalls and animals so that anywhere Jim wanted to point the
camera he would see Pandora come to life in the viewfinder.”
Weta’s virtual production pipeline was refined over successive
films. It was used extensively on The Adventures of
Tintin: The Secret of the Unicorn and The BFG, both for Spielberg, and advanced again with The Planet of the Apes films which managed to take
performance capture out of the studio into a forest and then even harsher
conditions of snow and rain.
Under Letteri’s direction, Weta wrote in-house renderers, Manuka
and Gazebo, which take their cue from physics to calculate how light interacts
with each surface – down to the level of calculating wavelengths of light
separately.
The facility’s artists and programmers have built on this to
create a suite of tools that aim to exactly match CG renders with the actual
lighting on set.
“On Avatar we used pretty
unsophisticated lighting which was just good enough to allow Jim to sketch out
his intent for composition,” Letteri explains. “What we really wanted was for
it to be physically correct so that what the director sees on set translates in
sync with the virtual lighting.”
PhysLight uses measurements of lumens and nits, colour
temperature and ISOs taken from the camera. “We know the precise ND filters,
the specific iris, shutter angle, sensor, framerate, even the serial number of
the camera and lens. All this data is used to simulate and synchronise with the
cinematography.”
The Avatar sequels
All of these cutting-edge developments are being trained on Avatar 2, 3, 4 and 5 which are in production in New Zealand and LA for release beginning in 2020 until 2025. Everyone expects the sequels to break ground – and in multiple ways. Cameron has reportedly test shot sequences underwater. Yet with technology moving so fast, striking a balance between taking risks on innovation and using tried and tested routes is one the VFX super must weigh.
All of these cutting-edge developments are being trained on Avatar 2, 3, 4 and 5 which are in production in New Zealand and LA for release beginning in 2020 until 2025. Everyone expects the sequels to break ground – and in multiple ways. Cameron has reportedly test shot sequences underwater. Yet with technology moving so fast, striking a balance between taking risks on innovation and using tried and tested routes is one the VFX super must weigh.
“The risk lies in not
doing it,” declares Letteri. “We’ve never been conservative about what’s
required of us to figure out a way of doing what hasn’t been done before. If
you don’t try you are never going to get there.”
While not strictly a visual effects technique, high frame rates
can be another tool to accentuate realism. Letteri worked on The Hobbit which was filmed at 48 frames a second
although the hyper-vivid quality was not to everyone’s taste.
“We are testing [HFR] for Avatar,”
Letteri confirms. “Audiences are used to seeing 24 frames per second where the
intermediate frames go blank and their mind fills the space with an image. It
engages the imagination in a certain way. There’s a case for high frame rates
with stereo 3D and lots of fast motion and you want to erase motion blur. It
depends on the intent of the filmmaker. They might experiment and see if it can
be used it to the advantage of certain scenes within a film.”
He feels that audiences are more likely to suspend their
disbelief seeing some physical effects like miniatures and stop motion “because
there’s a physical reality to it even though they know it’s not real” but that
CGI is unforgiving. “We have to get everything absolutely right and not slip up
otherwise the fact that it’s not real will strike a more discordant note in the
audience’s mind.”
That said, he says it’s often easier to solve a problem in a
computer “because you quickly run into limitations when you do it physically.
“I think we’ll be able to reach a point where we won’t be able
to tell what’s real and what’s not,” he says. “You will hit the limit of what
you need every pixel to look like to do exactly what it would have done if you
photographed that character. For some kinds of effects, I think we are already
there.”
A keen astronomer who has long harboured fantasies about
travelling to outer space, Avatar is a dream come true. “When Jim said ‘let’s go to Alpha Centauri and make a whole new
world” I couldn’t wait,” he says. “That’s where I’m going.”
No comments:
Post a Comment