How 'Endurance' brought Ernest Shackleton’s epic Antarctic adventures to life

RedShark News

article here

Considered the world’s first documentary feature, South was a record of Sir Ernest Shackleton’s 1914 to 1916 Endurance expedition to Antarctica, during which the ship was crushed by ice, stranding the crew. Over a hundred years after the ship was lost beneath the ice and Shackleton had led his crew to safety in an epic feat of survival, sponsored expedition organised by the Falklands Maritime Heritage Trust attempted to find and film the wreck.

They did so, with historian and presenter Dan Snow on board the South African icebreaker Agulhas II, to publicise the event. That 2022 expedition and the original heroic failure are the subject of new film Endurance which not only colourises the original footage but uses AI to bring Shackelton’s voice to back to life.

It’s directed by Elizabeth Chai Vasarhelyi and Jimmy Chin Oscar-winning filmmakers of Free Solo and Emmy-winning documentary The Rescue, about the against-all-odds rescue of 12 boys and their coach from inside a flooded cave in northern Thailand.

Bob Eisenhardt edited both of those docs and is also a producer on Endurance. The production had access to the 4K restoration of South from the BFI that was released in 2019.

“We wanted to make sure we could tell the Shackleton story and give it weight in the film because it's just one of the greatest survival stories there is.”

There was around 40 minutes of remastered footage but in the process of accessing it from the BFI they discovered another unrestored reel of 10 minutes.

 “The Hurley footage is spectacular. It looks like it was shot yesterday, but unfortunately when they abandoned the ship, they abandoned the cameras.

“So we had half the story we could tell through Hurley and the other half of the adventure, the fight for survival, was dramatically reconstructed.”

They filmed original recreations in California and Iceland, on glaciers and recreated boats in actual ice and freezing temperatures.

Treating the archive

“For the longest time I was just looking at the footage as a 1.3:3 image in black and white and it felt like we should keep that as an artifact. But when compared to the other material it also began to look like you were looking through a little porthole.

“The first decision was to blow it up a little to 1.6:6 and as soon as we did that we could see what was happening in the images much better. Then we talked about how we could the whole story more immediate. The big problem with history is how you make it resonate for audiences. That’s when the idea of trying colour came in.”

Hurley had in fact tinted his original film with sequences in blue, green and amber. It looks somewhat crude today but was cutting edge for its time.

However, their agreement with the BFI explicitly forbade any colourisation. “We couldn’t touch the footage,” says Eisenhardt. “So we ran an experiment. We had our partners at BigStar use AI to colour a sample of the image and it looked amazing.

“I’d been living with the black and white footage for six months and suddenly the images jumped off the screen. You can see that they’re eating peas for dinner and there was Shackleton in the middle of the scenes which you never really noticed before.”

They still had to get permission from the BFI. “They were adamant against colouring it. The archivists were very afraid of the Peter Jackson effect - that we would be creating something completely new. We wanted to stay away from that too, but they also insisted that the colours be accurate. That was very complicated but we solved it by devising a colour wash. When the BFI saw the colour wash samples they allowed us to do it. The fact that we were able to use colour gave the story so much more life.”

Like Jackson’s work on the Imperial War Museum’s archive to make They Shall Not Grow Old however, they did use AI to interpolate or create new frames to enable the film to be presented at 24 fps.

Colouring the Endurance

“We did a lot of research around the colour of clothing, the colour of ships and other textiles and materials of the voyage,” explains Josh Norton, Founder, and creative director at New York based creative agency BigStar (styled BGSTR) which previously worked on Free Solo. “We standardised those colours and spent weeks digital crafting each frame with some advanced software and AI to get to our final result.”

They determined that AI was not of a high enough fidelity to do all the work. “You can do an okay job quickly using straight out of the box AI approaches and effects - and there's a lot of different packages out there that do that,” Norton says, “but the amount of control that we needed to stay true to the material and to give a consistent result needed a large amount of manual craft work.

“This includes tracking clothing, and having just the right Burberry green for all the parkas, making sure that there was no fluctuation in the tones on the painted surface of the Endurance. There’s a degree of exactitude that needs to be achieved when it comes to the details of the tactile nature of that world. Every piece of linen and rope, every piece of wood, all the hair on the dogs, and the colour of the clothes needed very specific attention. AI processes as they exist right now cannot afford that attention.”

Norton says, “A wash technique really allows the black and white imagery to create all the value and contrast difference within the frame. We're just simply adding colour rather than adding any other kind of visual information or overriding the black and white. We didn't want to create a highly saturated result. We wanted the material to still feel aged.”

BigStar also did graphic design and title sequence for the show as well as taking in all the survey data and high rez photography to create a 3D model of the wreckage.

“The 3D model serviced several points throughout the film as far as explaining the status of the wreck and also giving us material for the title sequence itself,” Norton says.

Exploration and human endurance

The doc finds parallels with Shackleton's story and the recent expedition to find the sunken ship. “Thematically they were very similar,” says Eisenhardt. “Both expeditions got stuck in the ice, they suffered ups and downs and harsh weather. Both are tales of friendships. To make the story work, you had to find specifics that kind of spoke to each other. The idea of exploration as pushing the boundaries is a universal theme. But finding those moments in both the Shackelton footage and the new expedition that ‘talk’ to each other so that it would enhance both stories takes a lot of time. It took months to figure out the right percentage of each story to include.”

Footage of the 2022 expedition itself amounted to over 500 hours, the bulk of which was from shooting three cameras onboard the remote operated submarine that would spend six hours at a time underwater searching for and filming the wreck. The film’s co-director Natalie Hewit was onboard the Agulhas II to supervise.

“We watched everything, literally,” says co-editor Simona Ferrari. “We had a team that went through the footage shot 3000 meters below sea level so instead of having six hours’ worth we got maybe 2-3 hours per dive.”

AI brings Shackelton’s voice to life

The documentary also narrates portions of the 1914-16 expedition in the words of Shackelton and his crew who all kept diaries. “There were thousands and thousands of pages, some of it unpublished,” Eisenhardt relates. “All that had to be copied and sorted for the best material and from there we began a discussion about AI. It was a real discussion about whether using AI to voice their words was the proper thing to do. Our conclusion, was that AI is the perfect tool for this situation.”

The alternative might be to have some celebrity or actor pretend they're Shackleton but if they could use AI to replicate his voice speaking words he wrote, that felt closely to the truth.

It was easier said than done though. The only recording of Shackleton’s voice lasts just four minutes and was made on an Edison Phonograph wax cylinder.

“The noise on the recording was as loud as his voice, and he was speaking strangely since he was projecting into a giant megaphone. It was unusable.”

They turned to Ukrainian AI voice specialists Respeecher which, among other work, had resynthesized Mark Hamill’s voice for a young Luke Skywalker in an episode of The Mandalorian. They were able to scrub away the noise to leave a clean vocal track. Meanwhile, the crew’s diaries were whittled down to the raw material that they could exact dialogue from. They hired an actor with a neutral accent to record the dialogue, giving the appropriate intonations, from Respeecher made a model. They then applied their AI app using the voices of Shackelton and his crew.

“Everybody knew the story of The Rescue because it had been headline news for over a week. People know the outcome of Shackelton’s heroism. I think what matters to me is finding out who these people are and what makes them tick. That means digging deeper into the characters and wondering why did what they did. We started from asking what their motivation is to do what they do and we build up from that.”

The same filmmaking team are already embarked on another project, this one about climbing Everest. 

History of South

The original footage from South was donated to the BFI in the 1950s and the archive began to restore it back in 1994. There was no one complete original negative source for South.’ Overall, 99 different copies of film relating to Shackleton in the Antarctic, varying in length and age, were examined to piece together a restored version as authentically as possible.

The restoration used original camera neg from the expedition, prints from the sound reissue, nitrate release prints from the EYE Filmmuseum in the Netherlands with colour tinting, as well as 18 photographic glass slides.

The original photochemical techniques for colour tinting and toning were also recreated by the conservation team. This was completed in 1998, then digitally remastered for the film’s centenary, with renewed intertitle cards and a newly commissioned score by Neil Brand, in 2019.

 

Sports piracy: who’s pulling ahead in the AI arms race

IBC

article here

Live sports is the battlefield as AI plays both sword and shield in the ongoing war with piracy.

Piracy of sports streaming is rampant. So much so that an ESPN reporter was accused of watching an illegal streaming site when he posted a comment relating to a recent NFL game on social media.

Meanwhile, having paid €400m a season for rights to cover France’s top football division, Ligue 1, sports streamer DAZN set a target of 1.5 million subscribers only to find that in the first week of its broadcasts around 200,000 people were illicitly streaming its coverage.

Cybercrime is endemic according to anti-piracy solutions vendor Synamedia which suggests the sports industry is missing out on $28bn a year as a result. Even that figure, calculated in league with Ampere Analysis, accepts that a hardcore 26% of viewers will never pay.

“Piracy is exacerbated by the fragmented content market,” says Tim Pearson, Product, Solution and Partner Marketing Leader at Nagra. “There’s anecdotal consumer feedback that says ‘I’ve paid for two or three services, I can’t pay for any more so I’ll access this one through a much cheaper service’. The problem is that many consumers don’t realise that they are buying into a criminal enterprise that is probably harvesting data from them as well.”

Werner Strydom, Head of Advanced Technology and Innovation, Irdeto, says: “There is more live streaming piracy than in the past because live is the most valuable content to try and monetise with a pirate business model.”

AI sword

AI is escalating the problem. For example, Generative AI is accelerating the process by which pirates can create teasers, clip content and publish on social media to drive audiences to their platform.

Pearson says: “Pirates run sophisticated marketing organisations and AI is making their fake content look as good as if it were created by someone like the BBC. When content looks this professional, it dupes the consumer into thinking that the site must be legitimate.”

From a forensics perspective, the use of AI by pirates is a genuine concern. High-value content like a Champions League football match is embedded with a watermark that is so subtle that pirates find it difficult to determine whether it’s present or not, let alone remove it.

“Usually, [pirates] do a lot of content manipulation in the hope that if there’s a watermark present, it damages it to such a degree that you can’t read it again,” says Strydom. “With AI they probably no longer have to guess; they will know for a fact.”

Similar risks are now occurring with upscaling. “Even if pirates have stolen a crappy SD version, they can upscale it to HD. That used to take a lot of processing and video editing skills but AI has made it a lot easier.”

That upscaling process can also scrub away the watermark.

AI shield

The shield side of the picture is that AI is also accelerating defensive capabilities. Andy Haynes, SVP of Engineering at Friend MTS, an anti-piracy provider which counts UEFA among its clients, says: “AI is a really hot topic. There’s a lot of misinformation going around about how vulnerable the industry might be to AI, but it can also be extremely useful. The real value for AI is going to be in a lot of small things that help us work more efficiently, rather than one big system that just fixes everything for you.”

He reports: “We’ve seen cases where people have got a TV on in the background and they’re effectively doing the commentary with the live broadcast. It’s not the same content as was originally broadcast but we can start to use AI to detect that and then investigate further.”

The power of AI mostly comes down to automating what used to be an extremely manual workflow. A lot of metadata gets added manually based on human judgement and with human interpretation of the results. The latest generations of AI are making it a lot easier to automate those processes.

“It’s not completely taking the human out of the loop, and I doubt that will ever be the case,” says Strydom. “But what it certainly does is make it possible for us to extend the scope of our [web] crawling to look for piracy and to process a much larger quantity of potential piracy candidates.”

Dealing with live piracy requires responding to an illegal stream within minutes of an event having started, and according to Pearson, this is a major area where AI can help in triggering and accelerating workflows automatically.

“If an algorithm detects a watermark or a fingerprint of content that’s distributed illegally, the model is also smart enough to be able to react and deal with it,” he says. “You can scan for a lot more patterns and do a lot more pattern matching with AI than you can do conventionally.”

For a popular live event, there may be tens of thousands of potential streams that need to be investigated, but only a fraction are relevant to the actual event requiring protection. “Step one is differentiating between what is in scope and what is out of scope,” says Strydom. “Using AI to make a judgement call about whether something is tennis or football or some other sport is an established capability.”

On top of that, logo recognition can be added to filter out legitimate candidates. The next step might be to issue takedown notices to the few streams that are left. There are, however, risks of scooping up legal streams in the trawl.

“A lot of operators are not entirely comfortable with takedowns being completely automated,” Strydom continues. “Instead, they may try to prioritise known pirates who are causing the most harm from a brand or revenue perspective, and not worry too much about the smaller guys. Many operators are willing to accept a certain degree of piracy because they don’t want to create too much disruption for the legitimate customer base.”

In Italy, anti-piracy platform Piracy Shield – which has been in operation since the beginning of this year and is managed by the nation’s media regulator to protect sports rights on behalf of Prime Video, DAZN, and Sky – has managed to take down legitimate providers on more than one occasion.

“Clearly that’s not a good thing,” Haynes says. “We don’t want that to happen [to us]. We have a certain amount of reticence about using AI in that decision-making process for the very reason that if it gets it wrong, the negative consequences are pretty staggering.”

Haynes says police forces are similarly hesitant about using AI in pre-emptive decision making. “They can use it for pattern analysis and to predict where things might happen, but it’s usually a very robust process when it comes to enforcement because you have to be able to stand by how you’ve reached a conclusion. It’s not good enough to just say ‘That’s what the model spat out’. You’ve got to have the actual evidential chain.”

Minority report

One of the most compelling ways AI could transform the battle against sports piracy is through advanced content recognition and detection systems. Could algorithms predict potential instances of piracy before they occur?

Haynes is sceptical. “I wouldn’t like to say ‘never’, but I don’t think you’d need to detect it before it appears so much as you just need to find it sooner. Especially for boxing matches or events that can be over very quickly, the time taken to respond is hugely important.

“You can assume people will be trying to pirate an event and you could probably use some AI behavioural analysis to judge where to look. You don’t necessarily need to recover the stolen goods, but you need to have enough evidence to suggest that someone is committing a crime. We have techniques in place in certain areas that look for indicators of piracy to support that evidential chain,” he says.

Irdeto’s Strydom says it is possible to predict piracy based on subtle signals that have been accumulated as data sets, as a result of interactions by customers with call centres and conditional access broadcast systems.

“The patterns are so subtle that a human probably can’t see anything wrong with it but if the data set were large enough and you train a model on ‘normal behaviour’ versus abnormal behaviour, it may be able to filter for possible pirates. Whether you’d go a step further and pre-empt piracy before it happens, I’m not sure. It sounds a little sci-fi, even a little scary. We’re not experimenting with anything in that area right now.”

Disrupt to desist

Disruption is sometimes better than cessation, especially when blocking streams becomes a game of whack-a-mole. “Shut down a pirate stream, another one will open up. Whereas if you disrupt the experience, it is less easy for the pirate to monitor,” Pearson says. “That’s where AI can really disrupt the operation. For example, you could put up an overlay on the illegal stream to tell viewers it’s not authentic but they can scan a QR code and watch the rest of the game legitimately.”

Various other counter measures can be deployed using AI that will disrupt the viewing experience. “Ultimately, it’s not necessary to always kill the stream, but to make it such a bad experience that subscribers are going to give up trying cheap versions that don’t work and will then convert back to paying subscribers.”

AI-driven dynamic pricing models

Another possible deterrent is to alter the price of legitimate streams depending on precise consumer analytics based on their preferences and local market.

“It’s definitely possible,” says Pearson. “In fact, one of our products (Insight Negotiation Agent) uses AI in a virtual call centre agent, so if a customer thinks they’re paying too much for sports they could re-negotiate a new price within the guardrail set by the operators, via the virtual customer agent.

“That technology is here. Dynamic pricing on a match day is also possible. However, you’ve got to be careful that your core paid-up subscribers don’t lose out to people who are paying a third less after converting from a pirate service. It’s a balance, and tricky to do on a match day if you’re working on a pay-per-view basis.”

After DAZN had its nose bloodied on the first weekend of its Ligue 1 coverage, it offered a temporary discount on its monthly subscription fees (from €39.99 to €19.99) in an attempt to address pricing concerns. According to a survey from French market research and polling company Odoxa, Nearly two-thirds (65%) of French football fans believe the cost of a subscription to DAZN will encourage more illegal streaming of Ligue 1.

“The price was enough to drive the illicit behaviour,” says Pearson. “Once DAZN reduced the price and deployed a load of counter-piracy measures they saw an improvement.”

No AI silver bullet

Experts emphasise that tackling the criminals requires a belt-and-braces and straitjacket approach.

“There’s always something new coming along that will thwart whatever model you’ve already built,” Haynes says. “Pirates try to keep in business by avoiding detection, so we find that some of the things we’ve done in the past don’t always work the second time around.”

Nagra research quotes an operator of a pirate organisation whose candid response was to call it a game of ‘who can outwit the other the quickest’. “Just because pirates are using AI, that doesn’t mean that the industry isn’t using AI,” says Pearson. “So for every benefit the pirate gets, the industry also gets another one.”

There are no signs that cybercriminals are using AI to break the cryptography associated with the digital rights management (DRM) system, yet. Irdeto’s Strydom calls it “an arms race”, but also that there does seem to be greater awareness among clients that security is no longer something that can be skimped on.

“For a long time, anti-piracy has been seen as a ‘nice to have’, not an imperative; almost as if operators were asking why they should police the net,” he says. “Now that they can quantify losses due to piracy in cold financial terms, we’re seeing a change in anti-piracy attitudes.

“For years everybody talked about multi-DRM, but that is now just a hygiene factor. Operators are learning they need a lot more than that.”

 

Behind the Scenes: Gladiator II

IBC

In a world of green screen and AI, the sets for Gladiator II might be the last great build in movies.  

article here 

In 2000, Gladiator reaped more than U$465 million worldwide, revitalised the historical epic, catapulted Russell Crowe to international stardom, and won five Oscars from 12 nominations, including Best Picture.

“Twenty-five years ago, we made G1 and I know it was special,” director Ridley Scott said at a Bafta preview of the film alongside cast and crew. “It wouldn't go away. I’ve been busy making 17 other movies and along the way I kept being told by different generations, different nationalities ‘I love Gladiator’. They’d seen it online. The great thing about the platforms is they perpetuate all films all the time and they look as good as the day you made it.”

‘Are you not entertained?’ baited Russell Crowe’s Maximus Decimus Meridius to a baying crowd in Gladiator. With the sequel Scott appears to be baiting the cinema audience with action set piece on top of set piece.

Producer Douglas Wick says of ancient Rome, “The audience has seen grand combat many times over and their thirst for more was unquenchable.”

Production designer Arthur Max describes G2 as “Gladiator on steroids.”

In Malta, they assembled the palace, a grand city entry arch adorned with Romulus and Remus motifs and whole blocks of ancient Rome in an area approximately 8 km long. There was even a life size statue of Pedro Pascal, playing a Roman general, on his horse.

“It would be hard to overstate how massive a production Gladiator II was,” says producer Lucy Fisher. “The scope was overwhelming. In Morocco, there were over 80 huge tents dedicated just for the extras’ hair and makeup, and to house countless props and costumes.”

To film the Colosseum the production returned to Fort Ricasoli in Malta, the 17th-century building that had served as the site of the Colosseum set in the original film. The practical build was roughly one third the correct height of the real Colosseum, and somewhere between a quarter to a third of the span.

In a world of green screen and AI, this might be the last great set build in movies. 

Scott disagrees, “I want to build them bigger and bigger! We worked out it was cheaper to build a set than to use blue screen. Each time you add blue, it means money. There would be some element of blue in almost every frame of this film. So, what you see is real and none of it is blue screen.”

He shot the opening scene’s sea battle of Numidia in the middle of the Moroccan desert repurposing the old set from his 2005 film Kingdom of Heaven. “That was very economical,” he says.

“Ridley wanted two 150-foot ships coming toward this wall where a huge battle is taking place,” says Special Effects Supervisor Neil Corbould. “But there was no water there.”

They deployed hydraulic building movers (capable of holding nuclear reactors or tanks) and used them as platforms to steer two full-scale ships over the desert to simulate an invasion by sea.

“I’d seen these [machines] on the internet and had wanted to use them for years,” says Corbould. “This was the perfect job for them.”

ILM added water, sails and the rigging for the boats as well as arrows and fireballs.

“We replaced the clear skies with ominous dark clouds. And then we put in a few birds because the way to Ridley’s heart is always to add some birds to the shot,” says VFX Supervisor Mark Bakowski.

Multicam theatre

Scott, who is known for using up to four cameras at a time, regularly used eight to 12 for this shoot, plus additional drones and crash cams. He proudly claims to have shot the film in 51 days as a result.

“I can capture Paul [Mescal’s] entrance into the Colosseum in two takes as opposed to it taking all day,” explains the director. “You have to know exactly where to place the cameras. I can do that because I’ve storyboarded it all in advance. For even the best camera operator it can be hell. I don’t rehearse with the actors, but I do rehearse with the camera operators, and I dress them in costume on the set because they could end up in a scene.”

Mescal, who plays the hero Lucius, explains what it was like for the actors. “When Lucius arrives into Rome in caged-cart and into the arena it was shot in a single set up. Ridley had mapped it out half a mile of coverage. All of that was shot before lunch. Which is absolutely absurd.”

He added, “It felt like theatre to me because the cameras are always on.”

Scott likens the technique to directing each scene like a play, with simultaneous action taking place all over the set. “It helps the actor because their performance is not interrupted [by stopping for many set-ups]. I'm going to run the scene and the camera never stops. Even if you're not speaking [he told the actors] you're on. I'll be watching you.”

Denzel Washington, who plays scheming former gladiator Macrinus, agrees, “Everywhere you turn is Rome. It’s 360. That made our jobs easier than looking at markers for visual effects.”

Director of photography John Mathieson BSC, nominated for an Oscar for Gladiator, admitted, “I wanted to tear my hair out some days. Ridley works with a great deal of urgency. He has a lot to get done and this makes the process much faster.”

There was little conversation between the DP and the director while shooting, Mathieson says.

“I don’t do anything fancy. I place the lights and the cameras in the right positions. Some people claim we just mumble and grumble at each other on set but we don’t need to talk about the image. We’ve done this before. I know what he likes, I know what’s expected and I know it must look good.”

Calling the film “vivid, gaudy and a little camp” the DP’s visual cues were taken from the way Victorian painters romanticised neoclassical subjects.

“They painted idealised pictures of what Rome might have been,” Mathieson says. “There were goddesses in diaphanous gowns, beautiful marble stonework, opulent furniture, over-the-top feasts, and flowers. Rome was a bit of a mess by the 19th century, so it was primarily from the artists’ imaginations. These are not intellectual paintings but there is magic there.”

Enter the rhino

The Mill famously landed the UK’s first ever Oscar for VFX for its work on the original. This time around it is ILM in charge, including a gladiator-versus-rhino sequence which Scott had wanted to stage back in 2000, but was too expensive at the time to do with CGI.  Though never filmed, the CG test for the sequence was included on the film’s DVD release while Corbould, who worked on the original, dug into his own archive.

“I found some old storyboards of the rhino fight,” Corbould explains. “When I showed them again to Ridley he said, ‘Let’s do it this time.’”

Building the creature was a joint effort between Corbould and prosthetics designer Conor O’Sullivan. A wrinkled skin made of thick plastic was draped over the frame that became the rhino.

“We made a mechanical rhinoceros that could shake its head, flick its nose up in the air, and move its eyes and ears,” says Corbould. “We could literally drive it around the Colosseum like a go-kart.”

Flooding the Colosseum

In another scene inspired by historical fact, the Colosseum is flooded with water and filled with tiger sharks. Gladiators fight for their lives in a staged naval battle.

“There were two obvious ways we could approach it,” says Corbould. “We could build the Colosseum in a tank or use VFX. The best solution was to do both.”

Many of the larger shots were filmed on dry land, with Bakowski and the ILM team adding water in post. That meant Corbould had to find a way to create the sensation of floating with real boats filled with actors.

They brought back the industrial building movers, using them as a base to maneuver and crash a pair of galleons in any way Scott requested.

“Ridley was sometimes shooting with as many as 12 cameras,” Corbould says. “You want to get something in front of each of the cameras, whether it was boats or explosions or smoke or crashing water.”

The colour and depth of the water provoked debate. “We did many iterations, from the canals of Venice to Ridley’s LA swimming pool,” Corbould says. “The sharks, relatively speaking, went to plan but certainly didn’t make things easier.”  

Sound

More than 500 extras were brought in to play the Romans who crowded the Colosseum, with thousands added digitally.

“We wanted the actors to have as realistic an experience of the arena as possible,” says production sound mixer Stèphane Bucher. “We outfitted the set with huge speakers and assembled a wide variety of crowd noises to create the ambiance of the real games.”

Matthew Collinge and Danny Sheehan, founders of London-based sound studio Phaze UK, supervised sound editing and mixing.

To replicate the sound of 10,000 spectators, they recorded background players on the set, built that into layers, then added recordings of cheers and jeers from real-life bullfights, cricket matches, rugby and baseball games.

“We transformed them into a cohesive roar using a Kyma workstation,” says Sheehan. “Another device helped shape the roar of the crowd making it seem even bigger and louder.”

In the battle sequences, actors were outfitted with two mics concealed in their costumes, positioned to record dialogue no matter which way their heads were turned.

Nonetheless, “it was almost impossible to capture the dialogue audibly, but I take my hat off to Stèphane,” says Paul Massey, re-recording sound mixer. “He worked miracles so that we could minimise any ADR and preserve the original performances.”

Baboons go ape

The gladiator’s also fight vicious baboons in the arena. It was an idea that stemmed from Scott’s viewing of a zoo documentary.

“I'd seen a documentary about a wildlife park. There was ice cream, a tea shop and into shot come some baboons and some lady goes up to try and pat it. These are carnivores. It will tear your arms off.”

Envisioning a scene in which the gladiators face a troop of baboons Scott says, “Actors have to have an opponent to get the physicality and movement of the fight. So, I cast the smallest stuntmen and women we could possibly find. They are in black tights with black masks and Nikki (stunt coordinator, Nikki Berwick) made them short crunches that fit under their armpit so they could move on all-fours.”

Like the first film, the hero is fighting to restore democracy and honour in opposition to tyranny. Might the film’s release virtually day and date with the US presidential election have timely political resonance?

“Are you kidding?” Scott responds. “A billionaire wants to be the leader of the universe! Evil is evil. A sword will kill you just like an atomic bomb will kill millions. Death is death and where we are together today [as a society] we've really got to reign it and sort it out.”

 

Where the Wild Things Are

interview and copy written for RED 

article here

Combining blue-chip natural history footage and ‘in the moment’ observational documentary, independent Botswana based Natural History Film Unit (NHFU) is pioneering a fresh approach to wildlife programming. Operating from a private ‘film camp’ in the breath-taking Okavango Delta, the NHFU sends elite nature cinematographers into the field almost every day of the year to record raw, unfiltered and unique animal behavior which it then makes into cinematic award-winning films and series.

“We’ve kind of reverse-engineered the commissioning process,” says Brad Bestelink, the Emmy nominated filmmaker who co-founded NHFU in 2008 and whose extensive credits include The Flood, Living With Leopards, Okavango: A Flood of Life and many others. “Myself and up to five cinematographers are permanently in the field following all the big cats and animals regardless of whether we are commissioned or not.

“We live with the wildlife and wait for the stories to reveal themselves to us. As soon as we recognize the story, we start drilling down into that particular character or that particular circumstance.”

Bestelink will typically have shot 70 to 80 percent of the content needed to make a film before approaching a broadcaster.

“If I pitch an idea of a story, I join the line of everybody else pitching stories. It's also just a paper treatment and with that comes expectations about what the commissioning producer wants to achieve and that can be a straitjacket when you’re out filming.

“Instead, we go in with material already shot. We've already got a strong sense of what the story is. We've captured a lot of the key behavior. That gives the broadcaster a much clearer idea of the look and feel of what the film can be. There's a lot less risk involved for the broadcaster in making a decision to commission because they're more secure in what they what they're getting. The flip side is there's a lot more commitment and risk on our side, initially at least, but that's long been my approach to the majority of the films that we've made.”

NHFU is currently filming a second series of Big Cats 24/7, a six-hour documentary for BBC Studios Natural History Unit co-produced by PBS that follows the dramatic lives of lions, leopards, and cheetahs in the Delta. Bestelink’s team are following individual big cats around the clock, capturing their behaviour day and night.

“All of my camera operators are committed to working in the field. It’s their lifestyle and their passion. They will spend probably 275 days a year behind a camera in the field, following these predators all the time and gaining a deep understanding of their dynamics. Because we’ve invested so much time with them, we’ve already established biographies for many of the cats. The BBC is coming into something that is active and running instead of going into an area and trying to hire guides.

He stresses, “We know the individual characters, we know their territories, we know the terrain. There's a lot of experience and depth to our knowledge of these cats which just makes producing films a lot easier.”

Over the years, NHFU has amassed a 2 Petabyte archive of material that the company can exclusively draw on when producing new films.

“It's an enormous library that no one other than us has access to and because it’s all original material that we’ve been building since 2010 it has huge value,” he says.

A key reason for that is that Bestelink had the foresight to record virtually everything on RED. “It's the codec that is so exciting for me. We’ve invested in every iteration of RED camera but it’s the consistency and excellence of the codec that means everything we capture will have a very long lifespan. Our media doesn’t age.”

In 2010 Bestelink shot his first independent film on a popular professional camera, but when it came to deliver later that year the commissioner had moved onto requesting different formats.

“I thought, if I'm investing my life into making these films, I need to make sure that the format is going to be sustainable over time. At that point RED was not really utilized in Natural History but I had a friend working in commercials with an EPIC. On his invitation I went over to Australia for a month and tested the camera out. After that, I put my order in for one of the first RED cameras and have not looked back.”

Today, NHFU has one of the biggest fleets of RED cameras on the continent. “The compact ergonomics and the ease of the workflow are fantastic but more than anything being able to record at 6K and beyond has future proofed the media.”

His camera team go out solo into the bush and spend three to four days there filming wildlife before returning to film camp.

“A single person in the field is more wide awake, much more aware and much more in tune with the bush,” Bestelink explains. “As soon as you put two people in a jeep they will talk to each other and then that becomes their world, whereas a person on their own means it’s entirely up to them. They are listening and looking outwards all the time.”

Their camera kit consists of a HELIUM, WEAPON or V-RAPTOR and either Fujinon Cabrio 25-300mm or a Canon 50-1000mm which Bestelink calls “the ultimate wildlife lens”.

They also carry portable drives onto which they download the 4-6TB of media they are likely to generate on each field trip.

Back in camp their first job will be hand the drives to a colleague who will ensure it’s all backed up with the masters stored onto LTO tape.

“If the operator come across an incident in the field where there's a lot of action they'll radio in and one of the other cameramen will join them. Often, we'll have two or three photographers on one sequence, all cross-shooting on RED and in the same format. That’s quite an efficient way of working.”

The cinematographers rotate in shifts and are now able to shoot around midnight using military grade thermal imaging cameras. “You don't need any lights whatsoever and you can get great images without disturbing animal behavior,” he says.

The basic kit is complemented by a variety of specialist film equipment, including Phantom 4K FLEX, Shotover F1 Gimbal, DJI drones and even underwater housings and a submersible remotely operated vehicle.

The NHFU’s bespoke ‘film camp’ deep in the Okavango Delta houses a complete postproduction infrastructure with offices, suites and equipment for media management and processing. Edit teams prep proxies, tag and select media.

“I’ve brought several projects to a rough cut in the field right here,” Bestelink says. “It’s a one-stop shop. We've got multiple cameras with accessories and spares to prepare and repair them. We’ve got a full complement of editing software and we run eight customized filming vehicles out of this area.”

“There is safari tourism that is permitted in the area, but NHFU has exclusive rights for filming in this private area. We support that photographic tourism pays to keep the Delta as wild as it is and we work closely with operators to maintain this precedent”. We don't facilitate crews or operate as an agent for third party productions. Any production that we're working on, like Big Cat 24/7, is a partnership between us and the broadcaster so we’re very much entrenched in the production.”

Bestelink, who has lived in the Delta since he was four days old, also operates camera and spends almost as much time in the field as his camera team.

“I balance that with being at film camp producing and with my family. You know, I'm not a young cameraman whose heart is solely in the bush but I do live here with my family out in the middle of the bush.”

His work is increasingly focused on projects that have a conservation and environmental message. To do that, he also shows on-screen the experiences and relationships his cinematographers have with the cats as their stories unfold.

“Natural History filmmaking has experienced a boom but there’s some audience fatigue setting in because of the number of shows with the same glossy, high-end presentation.

“Incorporating people into the stories is a way to make it more accessible. The primary focus remains on the wildlife but the cinematographers are our primary storytellers. It’s through their relationships with the big cats that we learn so much more about them.”

He says, “We have to make people care about animals and the big cats in particular. If people don't emotionally connect with individual characters they're not going to develop in interest and passion for the wellbeing and future in their species in the wild.”

Visible from space, the Okavango is the world’s largest in-land delta. A combination of marshland and seasonal flood plains, it is rich in biodiversity and is often described as one of Africa’s last wildernesses. Yet the combination of population pressure and climate change is putting the whole biome at risk.

“The Okavango lives and dies by its annual flood and the amount of water that flows into the Delta. I just hope that we can protect it for long enough for the wet cycle to return. I am very concerned about its future.”

 

TAMS: fulfilling the promise of IP interoperability

IBC

article here

The transition to IP using SMPTE 2110 has been broadly successful in a studio environment but interoperability in the live and near live domain still has work to go. A recent innovation from the BBC could provide the answer.

The Time-Addressable Media Store (TAMS) API developed by BBC R&D is a new way of working with content in the cloud. It’s an open specification that fuses object storage, segmented media and time-based indexing, expressed via a simple HTTP API. It is intended to lay the foundations for a multi-vendor ecosystem of tools and algorithms operating concurrently on shared content all via a common interface. In effect, blending the best of live and file-based working.

The open-source API specification was launched to the industry at IBC2023 which is where AWS sourced it as the basis for a proof-of-concept Cloud-Native Agile Production (CNAP) workflow, demonstrated at IBC2024.

AWS was particularly interested in the potential of TAMS to streamline the process of fast-turnaround editing in the cloud in an open, modular way.

“The most important outcome of all of this is interoperability,” says Robert Wadge, Lead Research Engineer at BBC R&D. “That’s really what we’re driving at. TAMS enables sharing between systems and sharing between workflows across and between organisations. The aim is to give the media industry a way into near-live fast turnaround cloud production that doesn’t require them to buy into a single vendor’s vertically integrated solution.”

The BBC and AWS approaches are part of a wave of similar software-defined architectures coming to market. TAMs dovetails with the EBU’s Dynamic Media Facility; systems integrator Qvest is proposing to build video streaming platforms using what it calls ‘Composable OTT’.

What is TAMS?

Work leading up to TAMS stems at least as far back as the IP Studio project showcased as a live ‘IP-end-to-end’ outside broadcast at the 2014 Commonwealth Games in Glasgow.

The initial goal with TAMS was to bring the worlds of live and post-production closer together. Wadge explains: “Until very recently those two worlds have been quite disparate because everything was locked into hardware devices and bespoke systems. It’s almost like you record video onto a bunch of files then you bring it into your post-production and half the referencing gets lost on the way. The move to software means that for the first time we had the opportunity to do things differently and make media addressability reliable and consistent.”

The shift to software certainly promises flexibility benefits, but it’s not enough on its own to solve the problems of scalability and interoperability. Simply replacing signal processing with software won’t move the dial beyond the limitations of workflows designed originally for coax cables and tapes.

Wadge continues: “We wanted to move beyond the ‘lift and shift’ of taking a bunch of black box fixed function devices in racks and putting them in a data centre. Instead, we designed TAMS to be cloud native. With that comes a new philosophy about the way you write and deploy software. You can take a more modular microservices approach to media workflows and the infrastructure that supports that. Crucially, it enables us to architect horizontal capabilities that can be shared among a variety of people rather than having a very specific integration for each workflow in order for people to access and move media around.”

Vendors that may have been reluctant to cede a competitive edge by opening their systems up before are apparently changing their tune. It helps that AWS has backed the project and brought in partners CuttingRoom, Drastic Technologies, Adobe, Vizrt, and Techex for the CNAP demo at IBC. Sky also participated. It’s worth noting that TAMS is cloud vendor agnostic.

“With this project we’ve seen a different approach to vendor collaboration,” Wadge says. “A lot of vendors we’ve spoken to are facing a situation where they have to do a lot of bespoke integrations themselves on behalf of their end users.

“For example, there are a whole variety of different media asset management systems (MAMS) which any tool vendor in this space is under pressure to integrate with. The interoperability interface that TAMS offers gives vendors an opportunity to integrate at a [foundational] level which means that they do one integration and everybody wins. In that scenario, people are starting to see that it will save a lot of time and effort on integration that could be spent on adding features and innovating their own products.”

Breaking video down into smaller chunks is not new. It’s pretty much ubiquitous in streaming for distribution. HLS and MPEG-DASH are both based on the concept but these are optimised for linear playout. TAMS effectively takes those short-duration segments, stores them in HTTP-accessible object storage, and applies a time-based index over the top. This creates an immutable database from which any piece of the media can be accessed via the API.

Although its prime application is to smooth inefficiencies for producing near-live sports and news content, there’s no reason why TAMS can’t be used further downstream, Wadge says.

The ‘store once, use many’ approach to repurposing media means simple edits can be expressed as a metadata ‘publish’ rather than a new asset or exported file. This strategy reduces storage duplication, time spent processing storage, and the volume of space required for the same workload. Basic operations like time-shifting, clipping, or simple assembly can be achieved without knowing the media type or format, described purely in terms of timelines.

Nor does TAMS place any constraints on the media format. Indeed, BBC R&D has experimented using uncompressed video. Most users however will want less data-heavy workflows, especially in remote production scenarios which require media to be streamed.

“The idea is to abstract everything to a timeline and that’s the key principle behind interoperability,” Wadge says. “One benefit that flows from that is that TAMS will work with any media type today or media types that might arise tomorrow.”

Next steps

The IBC demo was reportedly a success with interest in the technology from across vendors and end users globally.

“We’ve taken a lot of feedback from AWS and the partners who’ve been involved in CNAP to refine the specification and we expect to continue doing that with a much broader range of vendors. What we really want to happen is for people to pick up the TAMs API and to build products based on that.”

He says the BBC is looking to use TAMS internally, specifically for fast turnaround news workflows and for extraction of VOD assets from live streams.

“Beyond that, TAMS really starts to come into its own when it’s used to share media by reference more widely across the supply chain. For instance, you can store your media once in a serverless repository which is accessible by everyone who needs to access it and then people can just go and get all or a portion of it to work with. They could transform it and then write that transformation back into the store to be shared with others. That sharing function is extremely valuable. It starts to break down the silos between a lot of the different functional blocks on the supply chain.”

The identity and timing model that underpins TAMS aligns well with SMPTE 2110 and NMOS, as well as MXF and IMF file-based delivery protocols for interchanging finished assets between organisations.

“There are common principles that map very nicely between these different areas which we’d like to build on. We think that the real value here is to have that timing and identity flow throughout the supply chain. Then it becomes a foundation which we can use for richer discovery of media and management of media. That’s a big focus.”

TAMS also dovetails with project work undertaken by the BBC with the EBU.

Like BBC R&D’s foundational contributions to SMPTE ST 2110, the JT-NM Reference Architecture and the NMOS family of specifications, this is another project which could only have come out of a body that does not have a vested commercial interest. The BBC will benefit from the work just like any other media organisation if TAMS enables them to integrate best-of-breed solutions from different vendors to build better supply chains.

“We want to build the BBC’s technology estate in a more modern way, one that’s not limited by the interoperability issues that that we would have otherwise,” Wadge says. “We’ve removed the barriers to adoption by making TAMS an open and freely available spec with no license fees. It means that there’s very there’s very little friction there for vendors to come on board. So we’re really excited to see what people build with it and hopefully it can help them innovate rather than having to focus on reinventing the basics.”

Overlap with EBU Dynamic Media Facility

The EBU Dynamic Media Facility (DMF) initiative is focused on design patterns for systems that integrate software-based Media Functions, proposing a layered model and recommending the use of containers for deployment on a common host platform.

In the reference architecture, published just before IBC, Media Functions are interconnected using the Media Exchange Layer, forming chains or pipelines that can be instantiated and torn down dynamically as needed, on a common infrastructure platform. The Media Exchange Layer “provides high-performance transport of uncompressed or compressed media payloads between software Media Functions running on containers on the same compute node, or on different compute nodes in a compute cluster.” Wadge comments that this is a low-latency transfer between running processing functions and a clear point where interoperable approaches will be needed.

TAMS, on the other hand, focuses on how media can be stored in short-duration segments in an object store such as AWS S3 and accessed by ID and time index via an HTTP API. This can be used to share media between tools and systems with a fast turnaround from the live edge of an ingesting stream.

“The two projects are complementary, and there are common threads in the different domains that we’re interested in drawing together,” he says.

BTS: Joker: Folie a Deux

IBC

Editor Jeff Groth asks for the film to be judged on its artistic merits and explains his process with flashbacks, story cards and musical numbers.

article here

Joker: Folie a Deux has been pummelled in Hollywood for not meeting the $1 billion box office take of the first movie whilst crashing its franchise future.  Director and co-writer Todd Phillips, lauded for Joker, has produced a sequel that has barely recouped the $200 million it cost Warner Bros. to make. But is it really just the numbers that are bad?

Unlike, say, Cats, Universal’s 2019 critically savaged musical that took just $80m on a budget of $300m, Joker II may have life as a cult movie. After all, the 1980 financial catastrophe that was Heaven’s Gate has been partly rehabilitated since it broke the back of United Artists. Warren Beatty’s notorious box office dud Ishtar from 1987 has also been reappraised.

Joker II has its fans including Quentin Tarantino.  “I’m just nihilistic enough to kind of enjoy a movie that doesn’t quite work as a movie or that’s like a big, giant mess to some degree,” he told a podcast recently. “The Joker directed the movie. The entire concept, even [Phillips] spending the studio’s money — he’s spending it like the Joker would spend it, all right?” Tarantino said. “He’s saying fuck you to Hollywood. He’s saying fuck you to anybody who owns any stock at DC and Warner Brothers. Todd Phillips is the Joker.”

Even the BBC is calling it “a weirdly postmodern joke at Hollywood's expense.” Noting that in the first film Arthur Fleck attacks Gotham's wealthiest people and railed against the smugness of the entertainment industry, in Folie à Deux “the over-spending and underperforming of the film itself have achieved the same goal,” said the critic, “reminiscent of Heath Ledger's Joker setting fire to a ziggurat of hundred-dollar bills in The Dark Knight.”

The film’s editor Jeff Groth ACE is generous enough to give his own appreciation of the film’s box office failure. “Obviously, I would have hoped to have a better reaction to the movie,” he told IBC365. “I certainly enjoyed working on it and I do still love the movie. I think it's unfortunate that a commercial piece of art gets judged in many ways monetarily and not necessarily on its artistic merits. That can be unfortunate but it is the risks that you take.”

Decent into madness

If Joker was a character study of a decent into madness then the sequel is a romantic character study of two human beings with a shared madness.

“It's a study of the effects of what happened in the first movie,” agrees the editor whose work on Joker was nominated for an Oscar and a BAFTA, “though in many ways Arthur is already [mad]. Arthur has a dual personality which is on the one hand sensitive and on the other it’s devilish.”

Locked away in Arkham Asylum, Arthur (Joaquin Phoenix) and on trial for murder, another inmate Harleen ‘Lee’ Quinzel (Lady Gaga) appears to be obsessed with Joker to the point of mimicking his appearance and actions.

“I think it’s fairly clear that Lee is in love with Joker, which is ultimately the tragedy of the movie,” Groth says. “Arthur can never be what she wants him to be and, in many ways, he can never be what everybody wants. He actually says in his closing arguments to the court that he can't be what people want him to be. He is accepting that he's just Arthur.”

Phillips, DoP Lawrence Sher and production designer Mark Friedberg created sets that could be shot in 360-degrees using multiple cameras in many scenes. These scenes weren’t precisely blocked in advance in order to give the actors free reign to perform in the environment.

“There was a lot of flexibility on set in terms of letting the actors do what they felt was right for the scene,” Groth says. “Joaquin very much likes to work with how he's feeling. Because Larry’s (Sher) visuals are so brilliant and he operates one of the cameras himself that is the one I use as my primary camera. I’ll watch what he has shot first for the performance and then go back to look for different angles and details. Because of that the first cut tends to take me a long time.”

On Joker, the filmmakers employed multi-cameras for the scene in which Arthur appears on the Murray Franklin TV show. This included mocking up ARRI 65 bodies, Alexa LF and four Alexa Minis to look like TV studio cameras from the 1970s

The same technique is used to record Arthur in several scenes of Folie à Deux including in the courtroom. These scenes are both shot by the film’s camera operators and simultaneously recorded on the TV cameras shown in the scene, the footage from which Groth edited into full screen TV shots for the show ‘America’s TV’.

“The scene was all shot live with the original performances then once we had edited the selects we wanted to appear on a TV screen we actually shot those pieces on a TV in a dark room and inserted them back into the film. It's almost like adding a filter to give it a TV look but we actually ran it through a TV screen so that there was no question it would have this authentic feel of the curvature and scan lines and some moire on the image as if watching TV in the early 80s.”

Groth chose not to study any musical cinema as a guide to cutting the numbers sung by Pheonix and Gaga. His approach was to rely on the memory of variety shows from the late 1970s because that would evoke the feeling of hallucination and lucidity in Arthur’s schizophrenic mind.

“We’re setting the story in a certain time and so there's a degree of memory that you have of things from that period which I wanted to lean into. We wanted to be authentic to the memory of that time as opposed to slavishly depicting something that actually happened. I'd rather have you feel like you remember it this way than to question whether something was hundred percent accurate.”

The other reason not to rely on specific references was to better convey the music as coming authentically from the central characters.

“When people break out into song in the middle of a performance the challenge is to make it feel like all the talking has been done and all there is left to do is to sing,” Groth says. “We were trying to be true to that idea by having our characters sing naturalistically. There’s no sudden choreography as they break into a big song and dance routine with all the background characters – the prison guards for instance - joining in. We’re treating the musical numbers almost like dialogue scenes. They are meant to stem from the emotion of the performances and what they were telling us about their characters at that moment.”

The film flashbacks to the first film using both footage that appeared in Joker as well as shots that did not. Groth had access to the original dailies which are archived at Warner Bros. Having edited the first film, he was already familiar with the material.

“The flashbacks are meant to be something that Arthur would see or would think about at that moment. If you look at the piece in the courtroom where the social worker from the first movie is reading testimony from his journal out loud, we see him in flashback writing in his journal. What that does is help you understand that he's writing that journal entry on the same day that he had killed Randall (his work mate). He’s sitting at his kitchen counter with blood on his hands and face and he's writing in his journal. It kind of becomes that much more horrifying when you get to see where this piece of evidence is coming from. That was not a shot that we had used in the first movie.”

To help him structure the story in the edit, Groth made story cards with a scene number, a brief scene description and an image and pinned them to a board. Editor Walter Murch uses a similar technique to help capture the essence of a scene and help suggest different ways of putting a scene together.

“I do that on pretty much every movie,” Groth explains. “It's a technique that perhaps goes back to the days of cutting on film when you’d hand reels up. I print one picture on each card and use it more as a tool so that multiple people can quickly see a simple representation of the movie.

“Even though you've got a timeline sitting on the Avid with all the technical nuts and bolts of how the movie is fitting together, that doesn't give you an overall feeling of what the movie is. We’ll look at the board frequently to gauge where we are in the process. As the thrust of the scene changes, we’ll reprint the cards for a more accurate visual representation.”