Remote control

AV Magazine

In a changing media landscape, live sports and events remain a huge draw across every platform and screen. Broadcasters and content producers are covering more live events than ever, however, tightening production budgets are driving a need for more flexible and efficient alternatives to expensive outside broadcast (OB) operations.

https://www.avinteractive.com/features/av-live/remote-control-18-10-2019/

“Remote production is the next step change, opening up new ways of working by allowing a production team back at base to have all the resources they need to deliver high quality, captivating content,” says Mark Hilton, vice-president, live production at Grass Valley.

The idea of ‘at-home’ production or REMI (REMote Integration) in which large parts of the OB functionality is decoupled from the event location to edit and switch back at base, is far from new.

Indeed, the FIFA World Cup in Brazil 2014 was the large-scale trailblazer for the technology when, faced with the sheer geographic spread of the Brazilian venues, host broadcaster HBS used EVS’s C-Cast technology to pump video back to edit bays located at the International Broadcast Centre (IBC) in Rio. Since then, every major sporting event has featured broadcasters that have chosen not to take space at the IBC, instead taking multiple camera feeds and creating the live broadcast back at base.

With fewer personnel needing to travel to the actual event, this has the advantage of also allowing the same production crew to handle more productions by having centralised staff and technical resources.

Practical, large-scale implementations are becoming increasingly common, led out of the US and Australia where large distances can be economically countered by REMI, and in Scandinavia where severe winters mean moving expensive trucks around is a challenge.

Swedish broadcaster SVT’s remote IP production for the broadcast of the FIS Alpine World Ski Championships in Åre, Sweden last February boasted the largest number of cameras deployed (80) and the highest volume of remote signals transmitted to date from a live location over IP to a production hub 600 km away in Stockholm. Partners included Net Insight and Grass Valley.

“This was not only about sports but about transforming our entire outside broadcast workflow for live events,” explains Adde Granberg, CTO at SVT. “Our reason to centralise our production facilities was about saving costs and also to really focus on making programmes, not on how to make programmes from a technical perspective.”

For SVT in other words, remote is a means of taking our cost and complexity from the operation. While some aspects of live production, such as switching, have been successfully hived off to remote, others are more resistant. Although automated robotic cameras are being introduced to lower profile sports events, for the most part you still need camera operators and engineers to look after the cameras; the OB compounds of major events are still full of vehicles.

The connectivity costs are a barrier to deployment as well, meaning that scale is important in achieving cost savings.

“If you want to run a match by remote the big saving is meant to be in not travelling people, but you’re only not travelling somewhere between four to six production people and does that pay for your increased bandwidth costs? Not necessarily,” calculates Hamish Greig, CEO of CTV Outside Broadcasts. “However, if you have more than the single event, if you’re going to do thirty events with a dedicated EVS replay centre and schedule matches so the same home-based team can also do two games, then the economies of scale start to work.”

What’s changing now is that remote production is not just about pure cost-efficiency but increasingly about delivering on the audience’s expectation of more content and more engagement.

“The amount of live content surrounding the event is continuing to grow,” says Yvonne Monterroso, director of product management at wireless links developer, Dejero. “Whether it is a live stream from a fanzone, reports from the team base camps or athletes’ village, or post-event interviews, reporters and production teams are continually finding creative ways to provide top quality content to audiences.”

It’s one of the main reasons that SVT is choosing remote for all of its sports and entertainment live events.

“From a viewer’s perspective they don’t care or know if it’s remote or OB,” says Grandberg. “Actually, it’s the reverse. With remote you can collect more camera feeds and make more of them to give the viewer more action and understanding of an event than you ever could before.”

To attract and retain viewers, broadcasters are required to deliver increasing amounts of content and ensure the most immersive fan experience for viewers.
“This means delivering more camera angles and making content available whenever and wherever, via a wide range of platforms,” says Hilton.

Vendors are following suit by refining their equipment to replicate more of the nuances of production. Grass Valley, for example, has updated its Live Touch replay system so that the feel of working with a remote server more closely resembles that of a conventional jog shuttle. Mobile video specialist LiveU has a Video Return service to let remote crews remain aware of the programme currently on air and even receive teleprompt information.

Live audio mixing is following the same trends with software interfaces replacing physical console desks. Two mixing systems with no physical control surface recently made their European debuts. Calrec’s VP2 can be accessed from multiple locations via a web-browser. Wheatstone’s Dimension Three is a multi-touch virtual mixer that interfaces to all the major production automation systems.

Crucial connectivity
Connectivity is of course essential for any form of remote production. While live feeds can be contributed to the central facility over the public internet, producers will be concerned about the quality of the signal and its susceptibility to delay and packet drop out especially over vast distances (even between continents). The return feed – the ability for technical and production staff to communicate as live with the video being produced in a remote studio, is another essential.

There are a number of options for this. One might be to process the video at the venue using a technology like Secure Reliable Transport which is designed to manage and smooth jitter. Other solutions include dedicated IP traffic shaping and monitoring solutions offered by companies, such as Net Insight and Nevion (in which Sony recently took a 45 per cent stake). The video here might be compressed using J2K, which is fine if you’re outputting HD.

For 4K quality productions, dedicated fibre or satellite might still be the best option, albeit an expensive one. Very few venues though will be fibre-linked and the cost of sending satellite uplinks is one of the drivers for remote in the first place.

In reality, most premium live events will combine a mix of transport technologies in order to be on the safe side, with one complete circuit held in redundancy.

“Our provision of connectivity from an event back to the broadcaster is based around both the use of traditional fibre connectivity on an ad-hoc low commitment basis and also the use of evolving 5G platforms,” says Chris Pearman, Remote Production Strategy and Architecture, Red Bee Media. “We believe these technologies will enable the dream of full remote production as a ‘pop-up’ service (temporary channel for the duration of the event).”

As the cost of connectivity continues to fall and bandwidth continues to increase, then remote production will become even more cost effective and appealing, especially with the advent of 5G.

The multi-gigabit speed, ultra-low latency network will bring more capabilities to the live events market, such as returning multiple channels of audio, enabling multicamera productions from a single portable transmission solution, 4K contribution and onward streaming, and live Virtual Reality experiences.

“With the capabilities of 5G, we believe that remote production will dominate most sports production globally because of its flexibility and cost-effectiveness and this is already being demonstrated,” says Ronen Atman, vice-president, marketing, LiveU.

It’s a trend likely to continue at pace beyond IBC and into Tokyo 2020. Eurosport, one of the rights holders for the Olympics, has ambitious plans to build three permanent content hubs in Paris, Amsterdam and London which will be fully operational by next Summer.

From there it will produce and distribute live sports programming from live feeds brought in from anywhere around the world.

Has the Holy Grail of long term storage finally been discovered?

RedShark News

Stone, papyrus, paper... the history of recorded information suggests that a physical medium has the best chance of long term survival. It’s a problem that Hollywood studios have long grappled with by retaining archival film prints of movies in the knowledge that, contrary to many digital, optical or cloud-based formats, it will safely last a century.

https://www.redsharknews.com/technology/item/6767-has-the-holy-grail-of-long-term-storage-finally-been-discovered

While LTO tape could feasibly last several decades, work is afoot to find a longer lasting format which doesn’t require manual migration every few years or run the risk of read/write obsolescence and data decay.

A new proof of concept seemingly made of Krytonite could provide a lasting solution.

Microsoft and Warner Bros. have collaborated to store and retrieve the entire 1978 Christopher Reeve Superman movie on piece of glass roughly the size of a drink coaster, 75 x 75 by 2 mm thick.

It was the first public test for Project Silica, a Microsoft Research project led out of Microsoft’s Cambridge, UK lab in conjunction with a team researching optoelectronics at Southampton University.

According to Microsoft, the process works by laser encoding data within quartz glass. This creates ‘voxels’ or layers of three-dimensional gratings and deformations at various depths and angles. Machine learning algorithms read the data back by decoding images and patterns that are created as polarized light shines through the glass. It’s somewhat like creating upside down icebergs at a nanoscale level, with different depths and sizes and grooves that make them unique. A 2-mm-thick piece of glass can contain more than 100 layers of voxels.

Indestructible?

Here’s the Krytonite bit. You can boil it, bake it, microwave it, flood and scour it, even demagnetise it but this is one tough glass. Data intact.

Calling it a major milestone but also a work in progress, Mark Russinovich, Azure’s chief technology officer, said the question wasn’t ‘Can we do it?’ but one of refinement and experimentation.

Important to note that this is not going be something that pops-up in a consumer device for playing back movies. This is about building storage that operates at the cloud scale to archive what’s known as ‘cold’ data —stuff that doesn’t need to be frequently accessed.

That includes a film studio’s master copies for example. Warner Bros. says Project Silica could create a permanent physical asset to store digital content and provide durable backup copies. Right now, for theatrical releases that are shot digitally, the company creates an archival third copy by converting it back to analogue film. It splits the final footage into three colour components —cyan, magenta and yellow — and transfers each onto black-and-white film neg that won’t fade like colour film.

Those negatives are put into temperature and humidity controlled vaults and if Warner needs the film back, they must reverse the entire process. With one eye on the dwindling supply of film labs in the world and another on the cost, studios need a way out.

“When we shoot something digitally and print that to an analogue medium, you destroy the original pixel values,” says Brad Collar, Warner Bros’ SVP of global archives and media engineering. “And, sure, it looks pretty good, but it’s not reversible.”

Glass storage has the potential to become a lower-cost option because you only write the data onto the glass once. Femtosecond lasers — ones that emit ultrashort optical pulses and that are commonly used in eye surgery — permanently change the structure of the glass, so the data can be preserved for centuries.

There are alternatives. DOTS (Digital Optical Technology System) stores digital data onto metal alloy tape and can be kept at room temperature. Microfiche-scale human readable text at the beginning of each tape contains instructions on how the data is encoded and on how to actually construct a reader so as long as cameras and imaging devices are available, the information will always be recoverable.

Originated by Kodak, US-firm Group 47 has developed the technology since 2008. “No other storage medium is stable for 100+ years under conditions of benign neglect,” it states.

A rival preservation system from Norwegian developer Piql is claimed to store digital files on high resolution micrographic film for up to 500 years. The polyester-based media is more chemically stable than nitrate.

Like DOTS, the source code for the decoder is stored on the film at the beginning of the reel in both human readable and digital form so only a light source, a camera/scanner and a computer would be needed to restore the information in event of disaster.

Microsoft itself has a parallel project to Silica using DNA molecules to develop archival storage. Using DNA to archive data is attractive because it is extremely dense (up to about 1 exabyte per cubic millimeter) and durable (half-life of over 500 years).

Of course, what is not known for any of these technologies is just how long their shelf life actually is. Come back in 500 years and we’ll have an update for you.

Craft Leaders: Edward Norton, filmmaker

IBC

In 1999, actor Edward Norton read a novel that changed his life.

“I had just wrapped on Fight Club and was getting ready to direct Keeping the Faith when I read this book that made a really strong impression on me,” he says.

https://www.ibc.org/create-and-produce/craft-leaders-edward-norton-filmmaker/5178.article

Twenty years later Motherless Brooklyn comes to the screen with Norton as lead actor, producer, screenwriter and director.

“The narrative of taking 20 years to make sounds like I’ve been working on it for that long but it was more a question of finding the time to concentrate on it,” he tells IBC365.

Norton acquired the rights to Jonathan Lethem’s story in 1999 but such was his schedule – segueing from his directorial debut to performing alongside Robert DeNiro and Marlon Brando in The Score then as FBI agent Will Graham to Anthony Hopkins’ Hannibal Lecter in Red Dragon - that development took a back seat.

“I knew I had a stretch of a couple years when I wasn’t able to look at it, but the initial hook was and remains the uniqueness of the central character,” Norton explains. “Jonathan had written a voice for the character where you are inside his head, like Holden Caulfield in Catcher in the Rye. I knew that this could be a great emotional hook for a character in a film, but it wasn’t a film itself. I felt, even as I read it, that the character is a home run, but the challenge would lie in transposing it into cinema.”

Motherless Brooklyn – which was released in the US last week but doesn’t release worldwide until December - is a hard-boiled detective tale with a difference. It follows Lionel Essrog, a private eye with Tourette’s Syndrome who works for a mobster in New York. What “sounds like a ludicrous gimmick” has been turned into “one of fiction’s most memorable narrators”, critiqued The Guardian at the time.

Norton made two crucial changes to the novel in his screenplay. He relocated the period setting from the 1990s to the 1950s and layered the story with a political dimension.

“The book is noir and has a very 1950’s gumshoe vibe to it. Everything about the writing is postmodern down to the Raymond Chandler-esque dialogue to its old-fashioned tone, framed against contemporary references. So, the decision to shift back in time was not as exotic as it sounds.”

Norton adds: “When I told Jon about my idea it didn’t really bump him that much. Film noir was an inspiration for the novel he wrote.”

The actor recalled advice from the director Sydney Pollack who made Out of Africa from the books of Karen Blixen into an Oscar winning triumph.

“He talked about how they identified an essence of her character and built a story around it. Transposing something for cinema is about having a sense of the essence of character.

“I felt the core of the whole thing was the empathy you feel for Lionel. The higher-minded ambition and social history of New York came much later.”

In the film, the ruthlessness of the city’s authorities embodied by Moses Randolph (Alex Baldwin) echoes the lawless swamp of the street. The character, which does not appear in the book, is a cypher for Robert Moses the controversial official who dictated much of the city’s building projects at the expense of community housing during the early half of the twentieth century.

Deeper themes
The real estate kingpin can also be read as a proxy Donald Trump

“I was interested in power brokers and social issues before 2016 [when Trump was elected President],” Norton says. “They are part of the dark truth of New York, the story that people don’t talk about.

New York is held up as this melting pot of racial integration and the place where democracy works when in truth it has the incredibly dark history of this autocratic anti-democratic figure who controlled its transition from the 19th century into a modern city.”

After receiving a B.A. in history from Yale University, the Boston born actor moved to New York to refine his skills in off-Broadway productions. Before hitting the big time, he worked at various jobs, including at The Enterprise Foundation, an organisation committed to building affordable housing for low-income families. His big break in Primal Fear (1996) earned Norton an Oscar nomination, followed two years later by a second as the neo-Nazi in Tony Kaye’s American History X – all before the age of 30.

He has since maintained an A-list acting career with social and political responsibilities including as UN Goodwill Ambassador for Biodiversity and producer of progressive documentaries such as One October, a celebration of New York, and By the People: The Election of Barack Obama.

Norton finished Motherless Brooklyn’s script in 2012 but arguably its themes resonate deeper today.

“There’s nothing explicit about what is going on today but there is a timelessness to the idea that in America if we get complacent at what goes on in the shadows then the damage will last for a long time,” he says.

To convey this sense of alienation, Norton turned to British cinematographer Dick Pope, BSC with whom he worked on The Illusionist (which earned Pope an Oscar nomination in 2006).

“Dick’s talent for making the past seem visceral is what made me want to go to him,” says the director. “I used Robert Frank and Vivian Maier photos to convey the path I wanted to go down and Dick brought the idea of Edward Hopper paintings.

“There’s the sense of people feeling small in Hopper’s work and this idea of people feeling intense loneliness in urban places was very central for this story.”

Norton says they referred to the classic noir Sweet Smell of Success and Taxi Driver for their acerbic take on the dark underbelly of New York as well as The Godfather lensed by the great cinematographer Gordon Willis.

The movie investigates the gentrification of NYC in the way that Chinatown did for water in California, and goes from the highest ranks of government to the smokiest jazz clubs of Harlem.

“Ultimately, Motherless Brooklyn is like the dark yang to the yin of Chinatown,” he says. “What strikes me about Chinatown is how adult it is in rendering the dark things taking place underneath the sunlit white, cream and tan narrative of American life. But it is an LA movie and we needed to have the grit and intensity of New York.”

At no point, he says, did he consider not playing the Lionel although he gave a lot of thought to not directing.

He credits current Warner Bros’ chairman Toby Emmerich, in 2000 an executive at producer New Line, for encouraging him to take the reins. The film’s budget is a modest $26 million.

“How studios became places that don’t make original adult drama couldn’t be further from the truth,” Norton says. “Toby urged me to direct this. We shared a similar taste in film like Warren Beatty’s Reds and Spike Lee’s Do the Right Thing and he pretty much talked me into it.”

Norton adds that one of the perks of being a director is that “you get to call up the people you admire and ask them to work with you.”

That goes for actors Willem Dafoe, Bruce Willis and Baldwin on Motherless Brooklyn, Oscar nominated editor Joe Klotz (Precious); Production designer Beth Mickle (Drive) as well as Radiohead’s Thom Yorke, whom Norton knew just enough to ask him to write an original track for the movie.

“I was nervous calling him up,” he admits. “Wouldn’t it be horrible if it all falls flat and I end up in a creative argument with one of my artistic heroes?

“In fact, I wrote him a letter and sent him the script. Said I was looking for a ballad of Lionel’s experience. Something in his reply made me think he was distracted and too busy. Then, two weeks later, I woke at 5am to an email from Thom saying he’d being giving it a lot of thought and here’s the demo. It was immediately striking.”

Yorke penned the song ‘Daily Battles’, which is also covered in the film by jazz musician Wynton Marsalis.

“There was something embedded in Thom’s song that, in all honesty, was not a very focussed theme in my script. It was about waking up to your broken heart and the idea that you gotta get your head up out of your own problems. That really moved me and changed something in my whole approach. Instead of the cynicism that characterises a lot of noir, this affected my sense of where I wanted the story to go. I made more script changes off the back of his song that I did as a result of the election of Trump.”

Behind the scenes: Diego Maradona

IBC

To a heart-pumping electro-synth soundtrack and in grainy handheld video you are jumped straight into the front seat of car riding at pace with Diego Maradona. The city is Naples, the destination the Stadio San Paolo and this opening sequence to the biopic of the footballer – which comes to DVD this month - is noisy, chaotic and cut like a chase scene.

https://www.ibc.org/create-and-produce/behind-the-scenes-diego-maradona/5171.article

“This crazy drive seemed to sum up his life and the footage is all jump cut in this crazy French Connection style,” says director Asif Kapadia.

Diego Maradona is the subject of Kapadia’s latest archive driven study of larger than life personalities which follows the hugely successful Senna (2010) and Amy (2015) and is once again a collaboration with editor Chris King.

“We had researchers make a huge sweep of existing material from broadcasters in Italy and from Boca Juniors [Diego’s first club] and we also had access to a large number of tapes filmed by Diego’s own camera crew,” King says. “That was the hook that got us into the project.”

This personal archive had been commissioned by Jorge Cyterszpiler, Maradona’s agent, who shot hundreds of hours of candid footage on U-matic video between 1981 and 1987 of his charge relaxed at home, partying in clubs, travelling in planes, training and on the pitch.

“This was material that even hardcore Neopolitan fans hadn’t seen,” Kapadia says. “It’s not like it was all kept together and neatly archived. There was a hard drive full of footage we found in Naples and someone else had tapes in a trunk in Buenos Aires that hadn’t been opened for 30 years.”

The best example of this is a scene showing the moment Maradona walked into the Napoli stadium for first time. “The shots taken from behind him as he walks up the stairs to the pitch were found in Naples; the reverse shot was on a in tape in Buenos Aires,” Kapadia reveals.

King boarded the project in early 2017 after much of the initial research has been done. “We had an optimistic plan to get it ready for just after the World Cup 2018 but it got mired and we got caught up in the drama in the cutting room.”

Golden boyDiego has lived his life in the media gaze. The pressure to perform in public is part of the film’s thesis. In the film, his little boy lost ‘Diego’ persona is contrasted with the pompous drug-fuelled Hyde of ‘Maradona’ who must show no weakness.

“He’d lived five lives before he even set foot in Naples,” King says.

Their first step was “to bolt together a rough timeline of his life, make some intuitive guesses, eliminate obvious duplication, and react to the material.”

Viewing 500 hours of raw material took two to three weeks alone.

“Asif creates his own timelines of 9, 10, 12 hours long on his Avid and I’d make my own selections on the other side of the room,” King says. “There was a huge amount of stuff to wade through.

“My initial take was that all his life is fascinating, chaotic, clamorous from the get-go. There’s no serenity at all. A single year of Diego’s life was a decade of anyone else’s – in their wildest dreams. We couldn’t possibly show it all.”

It became apparent that all roads led to Naples and Maradona’s seven years playing for SSC Napoli as the defining moments for this story – arguably of his life.

“That the most famous, most talented and most valuable footballer in the world went to an unfancied part of Italy, signed to a low ranked unfashionable club, raised questions which we thought we could try and answer in the rest of the film,” King says.

But to tell the story of Naples, they had to understand what had brought him there.

“The centrepiece was Naples but that would only work if we felt happy that we’d understood what came before,” Kapadia says. “The heart of the story only emerged after two years of trying and failing and trying again.”

The deal the film’s producers had struck with Maradona included securing his image rights and access to his personal video archive. In addition, they had contracted for a series of three three-hour interviews.

“In reality his attention span would wander after about 90 minutes,” says Kapadia who visited Maradona at his home on one of the Palm islands in Dubai. “I said, ‘let’s leave it today and come back tomorrow’. When I came back, I’d be lucky if I got maybe two hours total.”

To interview other witnesses for the film, including Diego’s personal trainer during the Naples period, Kapadia travelled to Buenos Aires and Barcelona.

“Our budget was only so much that we could afford a trip every 8-10 months. Many people have tried to pin down who Diego Maradona is but it’s challenging. He comes with a bit of an entourage and he’s also quite strong character. No one would talk to us unless he gave them the OK.”

“Half-angel, half-devil”The interviews with Maradona themselves were so patchy and unrevealing that the filmmakers considered not using them at all.

“They were a bit vague and just a kind of familiar version of stories he had already told,” King says. “He launched into a lot of anecdotes that we’d already seen in other material. He was definitely not going near the more contentious areas of his life. He didn’t answer some of the questions put to him.”

Probing into his cocaine habit, which began in Barcelona and continued at pace in Italy, was brushed aside by a pat statement “as if he were in a press conference rather than in any detail,” King says.

In addition to which the way Maradona spoke was “quite slurred or tired…maybe he was on medication. It was difficult to understand him. There was quite a low energy to the recording.”

Senna and Amy may have lacked a first person narrative but with Maradona “we felt we’d lost the immediacy of those films.”

It wasn’t a complete disaster. “Of the chaos of arriving in Naples we got the sense that he didn’t really understand what was going on around him. He explained his reasons for going there were financial as much as anything but that he felt compelled to go.”

Naples was the only club prepared to finance his wages and pay the then world record transfer fee of £6.9 million.

“When he did say something illuminating or funny we felt with the right kind of audio trimming and sound mix it could be pepped up. There was a temptation not to use it all, but it did offer a different dimension to the other films.”

King says Maradona’s time at Barcelona was particularly difficult. “He felt racially diminished as a South American in the city. His wife did too.”

His calamitous time in Spain included a shocking leg breaking tackle by Athletic Bilbao defender Andoni Goikoetxea and an incredible on the field brawl with the same player which ended Maradona’s Barca career.

All that, plus his childhood growing up in a Buenos Aires slum, his introduction to Europe’s club scene and experience at the 1978 World Cup, took up the opening 45 minutes of the film but ended up condensed to 5 minutes and intercut with footage of his mad arrival by car to Napoli’s stadium.

“Post Naples we had a really powerful 30 minutes as well but this version was four hours long so we had some really tough decisions to make,” Kapadia says.

With five weeks until picture lock the opening sequence was “elegiac, slow, quiet, melancholic” explains King. “We realised we were making the same mistake we’d made with Senna and Amy. We’re signposting at the beginning that it’s going to have a tragic end and it didn’t reflect the youthful energy that Diego had at that time. The music was working counter to the footage.”

Sound of GodInspired by the Giorgio Moroder-style disco that Maradona would have been partying to in Naples during the eighties, King replaced the score for the title opening with the track ‘Delorean Dynamite’ by Todd Terje.

“It pumped it up much more and underlined that all these events are inexorably leading toward this fateful encounter in Naples.”

He then rescored the first half the film using more electronic music and 80s-90s synth electro to keep the same energic feel. “As the film darkened in the second half we bring in [composer] Anthony Pinto’s score more and more until it dominates.”

As one might expect, much of Maradona’s story is told using footage of him playing but sports production in the mid-to late eighties was in the pre-Sky dark ages.

“It wasn’t like now where you have 50 cameras trained on the pitch with a dedicated super-slow mo picking him out of the crowd,” Kapadia says. “The broadcast footage was mostly from the high gantry position which we used once but didn’t capture any close-up action. Fortunately, his personal crew were able to follow up onto the touchline and behind the goals.”

King talks of trying to emulate the violence with which football was played at the time. “It was a contact sport. We wanted to undo a decade of TV innovation and get it back to showing a bunch of tough men running around, clattering into each other, interspersed with moments of unbelievable skill of Maradona threading his way through physical assault.”

The audio that accompanied the football footage, however, was very weak. Any official broadcast had loud Italian commentary all over it, which is used occasionally, but for the most part all the sound had to be stripped sound out and rebuilt in the edit.

“The audio is equal to the picture in terms of the amount of work,” King says. “Every tackle, every shot, every run and tumble is a bit of foley I’ve added in to create something authentic and of the period. The whole thing is a huge artifice in one way or another.”

The mosaic style which Kapadia and King have pioneered eschews talking heads and came about by accident, says King, when making Senna.

“Asif had recorded interviews with a few people on a handheld mic just for research and the intention of using it as a signpost for building the story but when we listened back it was mesmeric. He felt he didn’t want to break the momentum of the film archive by cutting to a modern interview. It would take you out of the moment. So that meant we had to find a style and the one we used was drama fiction with spare bits of audio to guide us through.”

The documentary convention of the time typically had an explanatory talking head followed by archive b-roll footage to illustrate what had just been said.

“For us the b-roll is the A-roll,” King says. “The film is the archive. Asif is genius at watching what appears to be something relatively banal and understanding that when shown on a big screen with sound it becomes this dramatic moment.”

A clip in the film shows Diego in a bowling alley. Its footage from the collection of a Napoli Ultra (hardcore fan) who had befriended Diego at the start of his time in the city.

“There’s a fear and vulnerability in Diego’s eyes as if he realises what Naples is going to mean to his life,” King says. “Asif saw that and used it and you wouldn’t do that if you were producing a more conventional doc. He is crediting the audience with a lot more intelligence to read the film.”

King and Kapadia’s treatment of the picture is layered. Some official match replays of Maradona’s goals were only available in slow-motion. King sped them up 150 per cent to run at normal speed. Footage is frequently panned and scanned from the original TV standard 4:3 to 1:85 widescreen, retimed and colour corrected.

“I use lots of blur to subtly draw your eyes to Diego while the rest of the picture is out of focus,” he says.

The rough and ready look to the film is deliberate to evoke the period. “We initially grab in all the material [digitise it] for assembly edits and when we go back to get the masters we spent a bit of money on standard conversion or re-telecineing original pieces of film. The technical post is huge.”

At time of release, Maradona still hadn’t seen the film.

“I’m still owed an interview by him,” Kapadia says. “I held one set of our three interviews back until I had the final film to show him assuming he would want to comment on what happened or say that this or that didn’t happen. Every time we tried to show him, he was too busy. He was in Colombia or Mexico or Moscow for the World Cup. We tried really hard to show him the film.”

They toyed with adding a coda about his life and appearance today. “There’s already enough material out there. It was unnecessary,” King says.

Despite the unprecedented access, the real Diego Maradona remains an enigma. “He was born in this rough, rough place on the outskirts of Buenos Aires and he feels that no-one gave a shit about him when he was living in real poverty and only gave a shit once he was rich and famous,” Kapadia says. “He’s saying ‘you can’t judge me’ because you don’t understand me. He may live in a gilded cage, but it’s a cage nonetheless.”

AI can make you appear 30 years younger. Actors everywhere rejoice!

RedShark

https://www.redsharknews.com/production/item/6753-ai-can-make-you-appear-30-years-younger-actors-everywhere-rejoice

De-ageing is all the rage just now. Robert De Niro has had his mug replaced with a facsimile of himself from thirty years ago in Martin Scorsese’s mob epic, The Irishman. Will Smith confronts the fresh prints of his younger self in Gemini Man.

Big budget productions like these, and in case of Irishman, produce a bill which proved too steep for Hollywood studios, with Netflix stepping in to fund the $150 million production, the bulk of which was spent on finessing the de-ageing process in order for the director to tell the story the way he wanted.

Traditionally, de-aging work like this has been done in one of two ways: through filtering (which saves time, but is hard to scale) or CG replacements (better quality at a higher cost) which can take six months to a year.

Now, there’s an algorithm for that, and if it works it could open the technique up to lower budget shows including the notoriously cash-strapped fast turnaround post production schedules of TV.

LA post house Gradient Effects has devised an AI-driven tech called Shapeshifter and applied it to the visage of John Goodman, transporting the actor back thirty years (that’s Rosanne era Goodman), for a storyline on one episode of HBO televangelist comedy The Righteous Gemstones.

In so doing, Gradient claim they’ve sidestepped the Uncanny Valley to shave decades off Goodman while keeping his performance and persona intact.

Gradient explains that their tool allows them to “reshape” an individual frame and the performers in it, and then extend those results across the rest of a shot therefore de-aging people in a fraction of the time of big budgeted shows.

The tool starts by ‘analysing’ the underlying shape of Goodman’s face. It then extracted important anatomical characteristics, like skin details, stretching and muscle movements. With the extracted elements saved out as layers to be reapplied at the end of the process, artists could begin reshaping his face without breaking the original performance or footage. Artists could tweak additional frames in 3D down the line as needed, but they often didn’t need to, making the de-aging process nearly automated.

“One of the first shots of ‘Interlude’ shows stage crew walking in front of John Goodman,” explained Olcun Tan, owner and visual effects supervisor at Gradient Effects. “In the past, a studio would have recommended a full CGI replacement for Goodman’s character because it would be too hard or take too much time to maintain consistency across the shot. With Shapeshifter, we can just reshape one frame and the work is done.”

Based on the trailer their work appears pretty good although you can’t escape the jarring knowledge that you retain of how the actor looks today. He’s so well known, it strikes you as odd to see him, not exactly wrinkle free, but with tighter skin and a tighter stomach, although he is a lot trimmer in real life now than he was 30 years ago. The wigged hair, which is not CG, is actually more alarmingly noticeable than anything in the CG-Goodman’s eyes.

And it’s the eyes where the uncanny valley is most noticeable. The windows to the soul and all that are especially hard to fake. In most previous attempts it is the cold empty stare which gives the game away (think Jeff Bridges in the Tron reboot, reprising his earlier role).

It could be that Gradient left Goodman’s eyes from his original performance well alone rather than overly doctoring them, which was the case in Will Smith’s full-head replacement in Gemini Man, to dubious result.

The other point about de-aging it that has to be editorially appropriate rather than gratuitous. In the HBO episode in question the story time travels back to 1989 when the Gemstone empire was still growing and Goodman’s character’s wife was still alive.

We are likely to see a lot more of de-aged and back-from-the dead actors (Peter Cushing’s resuscitation in recent Star Wars being the prime example) because creatively the technique, clearly advancing in sophistication, opens up new storytelling / money-making opportunities.

There are questions, though, some ethical, some flippant. The story in The Righteous Gemstones could have been told just as easily (actually more easily) with a younger actor playing a version of Goodman’s character. Has that just put another actor out of a job? If the technique becomes so good that it can, as in Gemini Man, carry the whole feature, what would be the point of hiring James McAvoy when you can get Patrick Stewart as Professor Xavier?

How far back do you go? Could you get a believable child performance out of Robert Redford, Clint Eastwood, Arnie? (not sure why you’d want to, but that imagine an elevator pitch that reads ‘Terminator meets Home Alone.')

If it can de-age, surely AI can ‘age’ an actor also, rather than having their youthful face caked in make-up like Dustin Hoffman in Little Big Man.

The elixir of youth is far from new – Michael Douglas, Brad Pitt, Kurt Russell, Samuel Jackson – have all had it applied. As yet few female actors have had the pleasure, although Sean Young did, controversially, for Blade Runner 2049 (when Harrison Ford in the same scene got to keep his warts and all).

It’s all data after all. So who owns the IP on that? The actor, the studio or the digital capture team? 

Behind the scenes: His Dark Materials

IBC

Any adaptation of Philip Pullman’s multi-layered fantasy His Dark Materials must decide how to represent the cast of animalistic ‘daemons’ which feature constantly throughout the story. Adrian Pennington spoke with key members of the VFX team to find out how. 

https://www.ibc.org/create-and-produce/behind-the-scenes-his-dark-materials/5149.article

In the 2003 stage play, the creature companions were operated by black-clad puppeteers. In the 2007 feature film The Golden Compass, they were computer-generated. Their creation is also a visual effect for the new BBC and HBO series, but crucially, one based on extensive on-set puppetry.

“It’s an exciting and nerve-wracking challenge,” says visual effects supervisor, Russell Dodgson.

“Do too much to the animation and we end up ruining the drama. What makes it more complicated is that these creatures have to accent and complement the nuance of the actors because both character and daemon are embodiments of each other. Working with puppetry was priceless.”

The eight-episode series is adapted by scriptwriter Jack Thorne, produced by Bad Wolf with Warner Bros’ New Line Cinema and based on the first of the His Dark Materials books, Northern Lights. Principal actors include Dafne Keen as the heroine Lyra, James McAvoy (Lord Asriel), Ruth Wilson (the ice maiden Mrs Coulter) and Lin-Manuel Miranda (aeronaut Lee Scoresby).

Production designer Joel Collins and VFX art director and previs supervisor, Dan May (both of Painting Practice, which has worked on Black Mirror, London Has Fallen and The Hitchhiker’s Guide to the Galaxy) had conceptualised the show’s worlds and produced concept art before Framestore boarded the project just six weeks ahead of principal photography.

Having won its first VFX Oscar in 2008 for The Golden Compass, the facility was the obvious choice.

“It’s in the company’s blood,” Dodgson says. “I met with [producer] Jane Tranter and Joel and we hit it off really well. They had a deep passion for the material and strong ideas for how it should be treated from a creature perspective.

Designing daemons
He elaborates, “It was clear that this wasn’t a Doctor Dolittle-type show with cartoon animals or one with scary monsters which function as spectacle. These animals are part of the fabric of the world in which the characters exist. I don’t think anyone has attempted to do this level of character creature work in TV before. Arguably few have tried to do it in film. Placing this kind of CG character consistently alongside primary actors is really unique and not possible without keeping it all within one company.”

Of the mammoth 2050 shot order delivered by Framestore, 1000 of them were for creatures, 400 for bears with the remainder for environments and clean up.

“We have Ruth Wilson giving a really delicate performance and we have to put a monkey next to her. That can go horribly wrong.”

The first step was to work with the showrunners, actors and directors to plot the daemons’ story arcs alongside that of their corresponding human. This allowed them to clearly identify the emotional beats of each scene.

“We had lots of conversations about their energy,” Dodgson says. “The daemons are part of your soul, your free will, your curiosity. They are a reflection of emotional state, so they have to work alongside the actors, sometimes betraying the character’s thoughts.”

The VFX team went straight into rehearsals, helping to block scenes and working out the relationship between actors and their daemons.

Among decisions to be made upfront: how much screen time to devote to the daemons? how anthropomorphic should they be? And how should they appear, disappear or transform?

In the novel, a child’s daemon can change into any number of creatures but when they reach puberty it settles into one form.

“The concept works on the page because the narrative always refers to a daemon by its name even if it has changed form but on screen that wouldn’t work. There would be a visual change and the association of a daemon being the same character while in different forms would be lost.”

Lyra’s daemon is an ermine called Pantalaimon which transmutes into over 30 different kinds of creature.

“We realised that it would be very hard for an audience to fall in love with a daemon or build a relationship with it if it changed all the time so we reduced the instances of transformation to a handful and only when it made editorial sense.”

Some animals don’t transform well from one to another. Massive creatures don’t segue into tiny ones with ease.

“Huge transformations are not the point of the show. Diverting you from the story with an ugly transform would be a bigger crime than not showing it.”

They concluded that the transform should happen through movement and be grounded in the reality of the character’s world rather than appear magically.

Pan, for example, transforms into a bird and back again, the similar body type allowing animators to bring the bird’s wings into its body to form a silhouette of the next creature.

“We’d rather it happen off-screen than be a horrible morph on-screen,” Dodgson says. “We show it when it means something and when we can achieve it in animation.”

In order to cast the daemons for the principal actors they went to the books to see what Pullman had chosen and, in most cases, followed his lead. Then, from hundreds of photographic references, they tried to find an animal that best represented the spirit of the corresponding human character.

In the book, for example, Mrs Coulter is described as a “beautiful young lady” who is “a cess-pit of moral filth” a wicked witch in the disarming guise of a fairy godmother with a golden monkey for a daemon.

“Coulter presents herself as beautiful and meticulous in appearance with a softness to her that is alluring,” says Dodgson. “We thought the golden snub-nosed monkey was a better fit than a Tamarin Golden Lion (which was chosen in The Golden Compass). It can look cute but, much like Mrs Coulter, when it bares teeth it is truly vicious.”

Blending anthropomorphic with animalistic
In the books, the daemons are the opposite sex to their human persona, a concept to which Framestore stayed faithful (using a male hawk of the species, a female fox and so on) but finding the exact match for Mrs Coulter proved tricky.

“Neither male nor female snub-nosed monkeys felt exactly right – the male has a different, unsightly lumps of skin in the corners of their mouths while the females don’t, so we made her daemon a bit of a hybrid to find exactly what we wanted.”

Everyone was adamant that the daemons were to be naturalistic but characterful. “They are animals with a human consciousness. They are present. But how much is too much humanity?”

Framestore established an animation for the creature that was as realistic in behaviour as possible then removed some of the creature’s more primal instincts. To put it bluntly, these daemons don’t defecate.

“The trick was to spend time with real animals and study their natural instincts such as when sniffing for food or mating or hiding when scared,” Dodgson explains.

They started with anatomically accurate faces but the more human the animal is, the more they had to break the skeletal face structure.

“From a baseline of them performing very specific animalistic movements we could then add in more human awareness, and if anything felt wrong we could quickly refer back to the original naturalistic animation and correct it.”

Framestore’s team in Canada went off to film polar bears. Dodgson spent a day with a caged pine martin. Wild cats and foxes were brought on set with a trained handler, not only to reference photograph but for the actors to interact with.

Scenes featuring a real snake were shot to give the actor a better idea of how the reptile would feel and move in their hands even if most of these shots were replaced with a CG snake.

Another consideration was the inscrutability of animals and human propensity to apply emotion to their face. For that reason, Dodgson says cats and dogs are among the hardest to animate – we are simply too familiar with how we think they are thinking.

“With other animals, we have more licence to say this is how they look when sad or happy. The lack of familiarity grants us the freedom to create our own rules. Our goal was to find an extra layer of emotion in a scene and to get that across.”

Master of puppets
This VFX groundwork was dovetailed with scenes shot with actors performing with puppets. These were rudimentary rather than sophisticated animatronics but emotive nonetheless with the puppeteers playing a vital role in bringing out the best in the actor’s performance.

“Using puppets is far more than giving our animators an eyeline to work from, as critical as that is,” he says. “This was vital in giving the actors an emotional connection with their daemon through the brilliant work of the puppeteer team. When Daphne looks at Pan when she’s feeling sad, she needs to see Pan as a source of comfort.”

On set, the process would begin with a ‘puppet pass’, typically a master shot establishing where the daemons are and to give the actor’s muscle memory of its position. In progressive takes the puppets are removed, replaced just by eyeline. A further ‘clean’ pass with nothing but the set would be made, then a reference pass for lighting and colour.

Every shot was scanned with Lidar by 3D scanning specialists Clear Angle Studios, the data turned into geometry for Framestore and used for precise camera tracking and accurate lighting and shadow effects.

The puppetry process also gave Framestore “free post viz” in that they could soft matte animations into the scene directly from the rushes.

“It was great for the animators since we had detailed information about how to light for the skin and fur of the creatures and for how the creatures would navigate the space.”

Smart lenses, mostly Arri Master Primes and Ultra Primes, provided additional data about depth of field to drive the post process.

The first season was photographed on the large format sensor Arri Alexa LF with additional camera work on the Alexa Mini. Material was primarily recorded In UHD in Prores 444 converted to EXR files for VFX and colour managed with an ACES workflow through to the grade by Jean-Clément Soret at Technicolor.

Season 2, which has been shot back-to-back with the first series, has shifted to the Sony Venice camera recording native 6K to allow for more re-racking and reframing of the image in post. The ability to detach the camera body from the image sensor block for use in a ‘snorkel’ mode is also advantageous.

The large sets at Cardiff’s Wolf Studios were augmented with blue screen, including scenes of London skylines and views of Oxford from the window of an airship, and green screen of Lyra riding the bear. The more northerly the narrative gets, the more blue-screen was required, including a 360-degree blue screen shoot of the mountainous icebound archipelago Svalbard.

Framestore won the Oscar specifically for its work on Iorek Byrnison but opted for a tonally different take on the character this time.

“He’s much truer to the book,” Dodgson explains. “When we meet Iorek he is a down and out drunk in an alleyway who won’t look you in the eye. He has lost his pride so when he looks at you for the first time you feel that power. This is carried throughout the series in posture which changes from head down to more upright, his back is initially more arched and weaker before gradually getting stronger.”

The puppeteers acted dialogue with the actors on set, with their voices captured via ADR in post. The exception was Joe Tandberg, the puppeteer/actor playing heroic bear king Iorek Byrnison.

“Using a relatively unknown actor rather than star talent allowed our animators to lean into the facial performance more. The team have done such a great job. Iorek feels like a fully rounded character and we use him a lot. There are five-minute long conversation scenes with him.”

Another seven-minute sequence features Iofur (another polar bear) in conversation with Daphne. “These are full 4K single headshots. The VFX is quite unflinching.”

Framestore assigned its international teams to work on the show. London was dedicated to the monkey and the majority of Pan. Montreal took charge of the bears and later scenes of Pan. Its Indian operation was involved in comp work, tracking, roto and prep.

Most shows with this amount of VFX would have split CG characters or sequences between vendors. Dodgson says the efficiency and consistency of the achievement for His Dark Materials could not have been achieved with a division of spoils.

“It’s down to Jane and the fact that she was very open to letting us be the keeper of the daemons and the bears.”

Materials