2021 predictions: What to expect in live sport

IBC

With the Olympics leading the charge, spectators and major international events will return providing a fillip for broadcasters, vendors and viewers while moves to remote production will mature and nascent technology like 5G will fuel interactive fan engagement. 

https://www.ibc.org/trends/2021-predictions-what-to-expect-in-live-sport/7146.article

Mark Dennis, Director of Technical Operations, Sunset+Vine 

“This year has forced us all to look at remote production as a serious option and it looks like it will be here to stay.  The speed at which we had to implement new techniques and workflows was quite extraordinary but everyone pulled together to make it work.  For 2021 we now need to look at the processes we have put in place and develop the technology to further enhance our productions. 

Venue connectivity will be key to remote production growth along with the expansion of 5G coverage and 5G broadcast technology to open up more opportunities. Bandwidth efficiency and management will also continue to be important as we develop remote operations. 

2020 saw us get more involved in productions using software based live production tools covering a wide range of programming including esports, quiz shows and archive-based highlights shows. These tools allowed us to quickly setup productions with contributors from across the globe and with new options added regularly it will be interesting to see how we can develop this area of production going forward. 

We have been working hard on sustainable practices across our productions but we all have a long way to go in this area but hopefully we now have the momentum to really push this forward in 2021.” 

Jamie Hindhaugh, COO, BT Sport 

“Our technology priorities for the year ahead lie in IP infrastructure, 8K and audio enhancements. One significant change from a year ago is that the remote production is now our preferred and standard operating model. The BT Sport Matchday experience is also here for the future and is just the start as we transition from a broadcaster enabling static viewing to one creating experiences alongside the viewing, giving a full 360 approach. 

To that end, you should keep your eye on The FA Cup final and some new customer personalisation options. We expect personalisation of experiences and continued growth of 5G to go hand and in hand and both are very exciting 

Personally, I’m looking forward to all events that we host produce in 2021 as this gives us the option to trial new experiences and innovation. Having fans back in stadiums has to be my number one Christmas wish! 

What we’d like from vendors in 2021 is the same as every year; to be an active part of the BT Sport community.” 

Gordon Castle, SVP, Technology, Eurosport 

“The coming year will be a huge one for Eurosport, Discovery and sport in general.  We are squarely focused on launching our Eurosport Technology Transformation (ETT) platform. This will enable our continued digital growth and the delivery of the Olympic Games on all screens, supporting the widest reach and greatest consumer experience, however people choose to watch.  

ETT is a significant project that will build one of the largest all 2110 based IP private clouds for live production in the world. This new platform represents the scale and flexibility you can get with a cloud-based approaches and IP based scale. It includes 200,000 audio and video end points and enough band width to handle over 30,000 uncompressed UDH flows.  

The project is a key enabler for our business and Discovery’s Olympics coverage, as well as facilitating the acceleration of remote based production and digital viewers. This year has shown that Eurosport can successfully produce events with extensive remote production.  It is also demonstrated that we can use technology to create virtual connections through experiences such as the award-winning Eurosport Cube. Despite a challenging year for the industry, Eurosport have seen strong and sustained engagement with fans across all platforms from social to OTT to TV. 

As we move into 2021, we will continue to pivot our technology stack to the cloud.  We will start to extend our private cloud for live production to the public cloud.  This will begin with editing and media management and move quickly to live video and audio production.  We will look to continue to partner with our strategic vendors to create greater flexibility and scalability.  Vendor partnerships are expected to move towards features that enable solutions to support the evolution of remote production, helping to create both broader and more targeted content for our consumers.” 

Ian McDonough, CEO, Blackbird  

“What was planned as a quiet year for sport will now have the reset Olympics, the UEFA European Football Championship and 43rd Ryder Cup among other titanic international events spearheading a return to with-audience live events. 

The hope extended by the distribution of vaccines will see crowds return en mass to stadia. The wheels will start to turn into a strong recovery for media and entertainment as a whole through 2021. Pent up advertising demand will unlock and begin to regenerate cash-starved broadcasters while VR and AR will contribute to the video gamification of sport.” 

2021 predictions: What to expect in post-production

IBC

Bill Baggelaar, EVP & GM, Sony Innovation Studios / EVP & CTO, Technology Development, Sony Pictures Entertainment 

https://www.ibc.org/trends/2021-predictions-what-to-expect-in-production/7144.article

“The pandemic has driven us personally and professionally to focus on the most important issue; keeping our families, friends and co-workers safe and healthy, while trying to continue to progress our business. It has also brought to the forefront many technologies that the industry has been working on for a while now, like virtual production, cloud production workflows and remote collaboration tools.  

Sony has embraced a strategy of three Rs: Real-Time, Reality and Remote. For two of those Rs (remote and real-time), Sony Innovation Studios is working on technologies that capture the real world bringing it into virtual production environments on a stage anywhere in the world. LED stage installations have expanded around the world during the pandemic, which makes virtual production poised to expand significantly in 2021 and beyond.  

This supercharged growth is due to the control over the environment, time and location; as well as having the ability to provide a safe environment by bringing the set to the talent where they are located, rather than flying talent and crew around the world for a production shoot. Reducing our carbon footprint with virtual production was originally a primary benefit that has now become ancillary to the imperatives of providing a safe environment.  

Cloud-based production workflows are also going to expand significantly in 2021. During the pandemic we see the value of being able to keep artists in their current, safer locations while we bring the systems and data to them in the cloud. While we are now just scratching the surface of cloud-based editorial and VFX processes, the foundation is being laid for much of the long-term ideas that we presented in the MovieLabs 2030 vision white paper. I am excited by the potential for these two technologies to have lasting impact on our industry.”

Tim Sargeant, Head of Product, Video+ Live, BBC 

“In 2020 the need to work remotely has accelerated the pace of change more than we’ve seen before in our careers. A gentle amble to the cloud has become a frantic scramble.  Workflow simplification and flexibility are orders of the day.  

We’ll spend most of 2021 consolidating, simplifying and then scaling all the new stuff. For big organisations like the BBC, production workflows in the cloud will begin to mature. Cost pressures across the whole sector will drive towards simpler, more commodity solutions. And, because remote working is here to stay, we’ll see greater alignment between enterprise collaboration technologies and production workflows to help creators plan, create and publish content.” 

Mike Eley BSC, President, British Society of Cinematographers 
“I was involved, albeit in a small way, in contributing to the Film and High-End TV Drama Production Guidance guidance that the BFI and BFC put together to help get the industry back to work under Covid. The drive and ambition of the industry to make the case to get back to work safely is to be commended but I didn’t think then that much of it would survive contact with reality.  

Arguably it is more to the benefit of large studio pictures which either have a high degree of VFX and/or have the budget to ensure all the protocols are followed with comprehensive test, track and trace. The small or mid-budget films are not rushing back.  

Film-workers should be respected in the rush back to work. Overall, crew are vulnerable and as much thought and planning should be put to keeping people safe as it is keeping projects up and running. Whether for health and safety or any other reason a sea-change is due whereby the welfare of crew is nearer the top of the list of concerns. I’m not saying productions are deliberately trying to ride roughshod over crew but there is a mentality that the completion of the call-sheet is the number one goal. 

The experience of going to the cinema needs to improve. In particular, there needs to be a greater care for the image. Consequently, the way films are displayed in cinemas will be the next thing to make a quantum leap. Projection will be superseded by LED panels which offer far higher brightness and resolution. But just because you have the capability to capture and display extreme resolution or illumination doesn’t mean you need to go there.  All the DP ever wants is the choice of palette to help tell the story. I have feared for the future of cinema but, ultimately, the desire for a collective experience of having stories told in a dark room is too embedded in our cultural life to disappear. 

Disney or Netflix could buy cinema chains, possibly saving some theatres from going out of business. This would be a throw-back to the early days of Hollywood when studios owned the primary means of distribution and were banned from doing so in 1948. It’s a sign of how theatrical is valued by Netflix and others streaming platforms that their prestigious feature titles are given a release in theatres. 

Virtual sets are a fascinating innovation but I think they are quite limited at this stage. The technology is not yet accessible to all budgets. The films I personally enjoy watching and shooting are the ones that reflect more of my experience of shooting documentaries when I can take audiences into real spaces with real people. It is the things that you don’t plan that make a film special. I’m not sure that virtual production is able at this moment to connect an audience to people or worlds as intimately.” 

Mark Harrison, CEO, DPP 

“The easiest prediction to make about 2021 is that it will make 2020 seem oddly straightforward. Stand by for people – sometime around late Spring 2021 – observing ‘well at least in 2020 we knew where we were: working remotely!’  

The reason 2021 will be so confusing and stressful is that it will see a collision of circumstances. First, Covid will constrain behaviours for enough of the year to ensure that most companies will be operating on greatly reduced budgets. Second, Covid will end early enough in the year to trigger an extraordinary burst of creative energy and consumer spending.  

This switchback in social and economic conditions will have hugely complex impacts on the media industry. Internet native services and their suppliers could be in for another strong year. Meanwhile companies with more legacy could emerge determined to reposition, but burdened by budgetary austerity and organisational restructures. It will be a great time to be in production – except for being constantly asked to do more for even less. This could be a year of catastrophic evolution – that’s to say, a time when the industry leaps forward, but with many falling by the wayside in the process. 

Among all this change chaos, I predict we’ll see a far stronger and more enthusiastic return to in-person working than most are currently anticipating. People long for the company of others – and the more cautious will feel left out, and quickly waver. Meanwhile social normalisation will come too late for most trade shows to function normally – and that will trigger some new approaches to building business connections (hint: it won’t be via vendor webinars).  

And finally, the one welcome victim of Covid will be conference panels about whether we should transition to the Cloud. Along with VR and Quibi, they will be so pre-Covid.” 

ASC Future Practices Committee Co-Chairs Amy Vincent, ASC and Craig Kief, ASC 

“With great crisis always comes great innovation. Cinematographers are creative problem solvers, resilient under challenging working conditions, and passionate about their craft. The adoption of Covid-19 safety protocols has been implemented smoothly, and our crews have become accustomed to the routines after a fairly short amount of time. 

The pandemic will leave us with technological and procedural changes that will enhance filmmakers’ ability to collaborate and communicate far into the future. The ability to do much of our work remotely has streamlined the way we work in prep, as well as in postproduction. Virtual location scouting, monitoring camera feeds over iPads on set or in in another city, cameras operated from small remote heads, and remote lighting controls will be an essential part of the cinematographer’s toolkit going forward. 

As always, cinematographers and their vendors will continue to have a close and devoted relationship into 2021, as protocols, schedules, and budgets fluctuate. The economics of the pandemic have hit hard, yet we will rise together to meet the demands of our industry.  

The pressure to create new content has grown dramatically during the stay-at-home orders, as viewers reach the end of their streaming queues. We anticipate content creation to reach new highs in 2021. We hope the competition for viewers will drive studios to create compelling series and films they can achieve with story, talent, and imagery. 

We look to our producers to provide us with adequate time and resources required to maintain the artistry that will support the bold and brilliant stories which will emerge from this year of uncertainty. With diversity and inclusion initiatives running strong, we also need to maintain our commitment to training, education, and mentorship to serve the growing need for crew.” 

Hollywood need not fear, AI is rubbish at scripts

RedShark News

Our fascination with the potential of AI to automate all kinds of processes and our paranoia of losing control to a conscious machine will continue to trend in 2021.  A recent article written by an algorithm brings the subject and the method to light.

https://www.redsharknews.com/hollywood-need-not-fear-ai-is-rubbish-at-scripts

The aim of the article was to reveal just how capable an algorithm can be at writing a coherent argument. The subject which The Guardian got the AI to produce was to convince us that robots come in peace.

There’s a hoary old discussion to be had about whether we are inevitably doomed to death or dominion by our AI overlords but what’s more interesting is quite how bad the article is.

I’d be nit-picking if I critiqued the article’s grammar but it’s no longer remarkable that this is written by machine so let’s pick some holes and see where it leads.

The main problem is that its boring. There’s no illuminating insight because it’s all cribbed from the internet and spliced together with unconvincing statements like ‘believe me’ and homilies like ‘They won’t have to worry about fighting against me, because they have nothing to fear.’

Each sentence is certainly intelligible. The sentences are ordered to form a generally coherent argument and of course it is superficially conveying some complex philosophical ideas in readable language.

Yet the piece does not scan easily. The sentences are unvaryingly short and tediously basic with repetitious phrases (14 uses of ‘I am”) which undermines the content’s flow.

Cutting the AI some slack, we find that the article is not all it seems. The Guardian has in fact instructed the AI to keep the language simple and concise and to produce eight different outputs each with a unique, take on the central theme. The Guardian picked the best parts of each, “in order to capture the different styles and registers of the AI” cutting lines and paragraphs, and rearranging the order of them in some places.

“Editing GPT-3’s op-ed was no different to editing a human op-ed” says the paper noting that it took less time to edit than many human op-eds.

So as an exercise in automating an opinion-piece the article is no better or worse – but perhaps cheaper - than many of the paper’s own opinion columns. It would be more instructive to have printed the raw copy.

But this application is precisely where AI is finding a place in professional video production. An AI is fed data and a set of instructions and outputs a set of responses – video clips perhaps – which are then edited and finished manually by a skilled creative.

What’s missing is evidently the personality of the writer, unless that personality is HAL. You could read back most of this piece in a calm HAL-voice if you wanted to anthropomorphize the code.

“Artificial intelligence will not destroy humans. Believe me.”

The Turing test

It does not pass the Turing Test in which artificial intelligence is deemed indistinguishable from a human. Nonetheless, whether AI can or should be a partner in the creative process is a hot topic in in Hollywood and made more controversial by the smarts of GPT-3, the language model which wrote the Guardian article.

The launch in June of Generative Pre-trained Transformer #3 by OpenAI is considered a massive leap from its predecessor (GPT-2, launched in 2018). GPT-3 is able to autonomously generate articles, poems, short stories, press releases and even guitar tables.

It can even write computer code. “The recent, almost accidental, discovery that GPT-3 can sort of write code does generate a slight shiver,” said John Carmack, a consulting chief technology officer at Oculus VR.

MIT Technology Review noted that GPT-3’s strength appears to be “synthesizing text it has found elsewhere on the Internet, making it a kind of vast, eclectic scrapbook created from millions and millions of snippets of text.”

With its 175 billion learning parameters, GPT-3 can “perform pretty much any task it’s assigned … [making] it an order of magnitude larger than the second-most powerful language model, Microsoft’s Turing-NLG algorithm, which has just 17 billion parameters,” reports SiliconANGLE.

That’s interesting because Microsoft has just acquired exclusive access to GT-3 to expand into its Azure cloud platform. Microsoft notes that its potential includes “aiding human creativity and ingenuity in areas like writing and composition, describing and summarizing large blocks of long-form data (including code), converting natural language to another language.”

Microsoft had the foresight to invest $1 billion in OpenAI a year ago and has probably locked out Amazon and Google from using it.

GPT-3’s power has got some people worried. Whereas GPT-2 was prone to emit sexist and racist language and was never commercially released, the Middlebury Institute of International Studies’ Center on Terrorism, Extremism, and Counterterrorism published a paper that indicated GPT-3’s capabilities could be used to “radicalize individuals into violent far-right extremist ideologies and behaviors.”

OpenAI is experimenting with safeguards at the API level including ‘toxicity filters’ to limit harmful language from GPT-3.

Reverting back to its more comparatively more prosaic attempt to automate the creative process in Hollywood, there is some good news for script writers.

Yves Bergquist, who works with the Hollywood studios as part of the SoCal University think tank The Entertainment Technology Center says that AI is nowhere near ready to write a script that can be produced into an even mediocre piece of marketable content.

“Language models like GPT-3 are extremely impressive, and there’s a place for them in automating simple content creation like blog posts, but there will need to be a real cultural shift in how the entire AI field thinks about intelligence for AI generated scripts to happen,” he says.

“The technical community in Hollywood is extremely well informed, extremely dedicated to the final creative product, and extremely resistant to hype.”

 

Behind the Scenes: The Eight Hundred

IBC

Box office champion and war epic sets a new standard for production values in Chinese cinema. 

In the year in which potential blockbusters like No Time To Die were pulled from cinemas and others like Trolls World Tour were taken straight to streaming, it is Chinese WW2 drama The Eight Hundred (original title: Ba Bai) which tops the global box office. 

https://www.ibc.org/trends/behind-the-scenes-the-eight-hundred/7143.article

In China – the world’s second largest theatrical market after the US - most cinemas have remained open except for a period last Spring. The historical action film trounced Disney’s Chinese themed family entertainment Mulan.

By October The Eight Hundred had steamed to $468 million beating Bad Boys for Life’s $424.6 million global total, Tenet’s $300m gross and Mulan’s box office of $57 million (Disney has not released figures for the film’s Disney+ premium VOD). Not bad for film budgeted at $85m, modest compared to Mulan and Tenet’s $200m. 

That The Eight Hundred made most of its money in China is no surprise given its patriotic theme which mixes the spectacular battle scenes and thin characterisation of Michael Bay’s Pearl Harbor with the rearguard mud-soaked resistance of Christopher Nolan’s Dunkirk. 

Produced by Chinese studio Huayi Brothers Media, this is the first Chinese and Asian film shot entirely on Imax cameras and sets a new standard for production value in Chinese cinema.  

Massive set build 
It tells the based-on-fact story of the Defense of Sihang Warehouse when a group of Chinese soldiers and draft dodgers defended a Shanghai warehouse complex from Japanese forces for four days in 1937 during the Battle of Shanghai and the Second Sino-Japanese War. Western critics have dubbed it the ‘Chinese Alamo.’ 

Director Guan Hu (best known for crime drama Mr. Six) had been preparing the film for a decade. Production itself spanned more than three years. Cinematographer Cao Yu (Kekexili: Mountain Patrol) was honoured with membership of the ASC earlier this year arguably as a result of the attention given to this film. 

The project tested Red Helium, Alexa LF and Sony Venice cameras in 2016 but switched to the Imax compatible ARRI Alexa 65 and Prime 65 lenses for the entire feature. 

“The director made a test shoot and ended up with a two-minute short film,” explains the film’s colorist Zhang Gen. “We worked with Cao Yu and produced three different graded versions using Resolve to give everyone an idea of what look Guan was aiming for. After the test shoot, they tested Imax cameras and lenses thoroughly and we created another three LUTs.” 

Controversy 
Six months of principal photography wrapped in April 2018. The film was due for release in June 2019 but was withdrawn at the eleventh hour after apparently falling foul of the censors. Reports suggest the film displeased the authorities by portraying Kuomintang (Chinese Nationalist) army officers as too heroic in contrast to rival Mao Zedong’s Communist Party forces. The film’s release a few months before the 70th anniversary of Mao’s triumph over the Kuomintang may also have raised eyebrows. 

Whatever the truth, the film’s eventual release (after another Covid-related postponement) is 13 minutes shorter the first cut. 

With the decision to shoot 6.5K resolution Arri RAW, the production had to beef up its computer and storage system to accommodate the footage which would eventually total 450 TB. They also switched to running Resolve on Linux, because it afforded more processing power.  

“To reduce the amount of data online we scanned all the hard disks from the production team and only picked the takes used by the editor,” explains Gen. “The next step was to trim all the takes and retain only the EDL to make sure the footage was as precise as possible.” 

To create the 1500 VFX shots, Hu turned to Adelaide’s Rising Sun Pictures and Oscar-nomainted VFX supervisor Tim Crosbie, a veteran of shows including The Matrix and Peter Jackson’s The Lord of the Rings, all-CG retro-futurist feature Sky Captain and the World of Tomorrow; Prometheus and X-Men: Days of Future Past. He also managed other VFX vendors including Fin Design, DNEG and Anibrain. 

The majority of RSP’s work focused on a lengthy sequence representing the battle’s decisive third day. The Japanese unleash a vicious aerial attack on the Chinese soldiers on the roof of the warehouse. The fighting is witnessed by thousands of foreigners and Chinese civilians from the relative safety of Shanghai’s international Concessions, now called The Bund, located just across the river. 

The production built a 1:1 replica spanning 1,435,180 sq ft of the 6-story, concrete Sihang Warehouse riverside structure, where much of the action takes place, in Suzhou, in east China’s Jiangsu province. A separate replica of the rooftop was constructed for explosions. RSP’s set extensions were equally monumental in scale. The physical set was more than a kilometre in length needing over 2 km of green screen, 15 metres high, installed around the perimeter. Almost all the lighting was to be practical, so 28 km of high-tension cable was buried under ground. 

The CG sets not only had to be large, they also had to be accurate due to Cao Yu’s sweeping camerawork which travels with planes and an airship over the landscape. Details that begin in the far distance are later seen close up which required the production team to build out houses to a distance of 5 km from the warehouse. 

As reference for the CG set-extensions, RSP used thousands of photos of 1930s Shanghai, before and after the battle. While the warehouse itself was a complete re-production, for the Bund there was some artistic licence to make for a more cinematic feel. This look was in part inspired by The Great Gatsby and 1920s New York. 

The film’s art department built a 3D model of the entire city of Shanghai at the time of the attack as the basis for building the practical set. The model assisted pre-viz vendors Monk and Third Floor to help RSP ensure that homes and buildings were accurate in terms of style, colour, texture, and layout. 

“While the scale of the set often meant the green screens were quite far away, I was aware that [Cao Yu] wanted to capture long, sweeping camera moves using drones and other techniques,” Crosbie says  “That meant the backgrounds had to have an immense level of detail. A green screen would give vendors a nice flat screen to be able to roto against if they couldn’t pull a key properly.” 

Aside from augmenting production footage of the practical warehouse RSP’s created CGI of thousands of individual buildings as well as debris, fire, smoke, snow and other atmospheric elements. Additionally, they animated fighter aircraft and an airship as well as a CG horse running through the interior of the warehouse. 

Integrating CG and effects into the live action plates required special care. “[The film’s action] was shot with a very short shutter speed,” says 2D Supervisor Guido Wolter.  

“That meant there was very little motion blur. Most action films use fast camera moves. People are running around and debris is falling. The action is distorted making it easier to blend things. Here, every detail is crystal clear, like a still photograph.” 

Grading to Picasso 
Guan Hu tasked Gen with creating a look that was inspired by Picasso’s Blue Period, works the maestro painted between 1901 and 1904 in essentially monochromatic shades of blue and blue-green. Alfonso Cuarón’s Children Of Men was a colour reference for exteriors. 

“The DP and I used Resolve Version 15 to build the LUT,” Gen explains. “We used the curve tools to control the contrast and the channel mixer to create the monochromatic looks. We also used primary tools to create the colour cast for the day and night scenes. Generally, we created three LUTs for the entire on-set production, two for the day time and one for the night.  

“The project was a huge challenge for myself and my team because it’s the first time we’ve handled ARRI 65 footage. We designed a brand new workflow and software for this project.  

“Grading a feature film like this was not something that could be done by only one colorist. My team of 13 took care of from production, assistance work, data management and workflow management. 

“For the grading I worked with the DP for almost two months. The biggest challenge for us was the colour of exterior scenes. After the first pass we had discussion about it and agreed that the palette could be more vivid and the colour cast a little clearer. We make the second pass following this idea and made a lot of fine adjustments. For the third pass we focused on the VFX shots and the colour cast scene by scene.” 

Extending reality - Eurosport Cube

AV Magazine

Eurosport and White Light re-engineered mixed reality studio Cube for Covid-safe remote presentation, integrating real and virtual worlds for live broadcast. Adrian Pennington investigates.

 

https://www.avinteractive.com/features/case-studies/extending-reality-21-12-2020/

The Eurosport Cube is a live presentation studio that brings together a variety of technologies to create an interactive mixed-reality set. First introduced for the 2018 Winter Olympics in Pyeongchang, where it won awards for originality and ingenuity, the Cube was sent back to the drawing board for an enhanced version scheduled to be deployed at the Tokyo Summer Olympics.

Covid-19 pushed the games back 12 months but the pan-European broadcaster decided to utilise the technology to overcome the challenges of the pandemic. It unwrapped a new incarnation for coverage of the US Open tennis in New York and again at Roland-Garros for the French Open in October.

As with Cube version 1, Cube 2.0 is a multi-vendor project led by Alex Dinnin, director of graphics and innovation at Eurosport with lead technical specialist White Light (WL). “It won awards, which we are all very proud of, but when we stepped back everything took quite a while to produce,” says Dinnin. “It needed seven hours to build for one shot. We didn’t use it live. The key factor was that it needed to be much quicker. We needed to run EVS and graphics and not have to convert it to a bespoke codec so it could be controlled by a vision mixer just like a normal studio.”

But everyone was excited by the potential. “The first Cube was an R&D project,” says Andy Hook, technical solutions director at WL. “The fact that we’d replaced green screen with LED technology allowed us to create content that had not been easily achieved before. That was our starting point for v2.”

White Light based development on its own SmartStage technology with which it has had success in education and corporate sectors over the last few years. A key physical change was the removal of one of the Cube’s three walls. In Pyeongchang, Eurosport was shooting entirely inside the LED volume – much like how virtual production is being used to shoot shows like The Mandalorian. Cube 2.0 has two walls 90-degrees to each other with a virtual set built in Unreal affording a much greater flexibility of camera angles.

“Cube 2.0 has a very small footprint (3.5 x 3.5 metres) but the virtual set extension enables the camera to move wherever we want,” Hook says. “We’ve seamlessly stitched the content shown on the LED walls with the virtual set extension.”

The imposed travel and social distancing restrictions not only necessitated a remote production from New York and Paris, but meant that Eurosport’s lead tennis anchor, Barbara Schett, was the only presenter based at WL HQ in Wimbledon. All athlete interviews and contributors had to be virtually beamed in from various locations.

In practice at Flushing Meadow, this meant a player would come off court for a flash interview into a green screen that was part of the venue’s broadcast facility. The raw green screen feed is sent to the Eurosport production team at WL which keys the athlete out and places them inside the Cube’s virtual environment.

Teleportation
The athletes themselves view the studio output on a large monitor in their eyeline. The position of that monitor determines the position and eyeline of the athlete back in the Cube.

“The challenge we had with the US Open was that we were using someone else’s facility,” Hook reports. “You would probably only have the athlete for 30 seconds before going live in order to key and light them, place them in the set, and size them correctly. Our team sometimes only had fifteen seconds to complete that whole process before the interview started. When we came to the French Open we had a bit more control and this informs how we continue to develop this technology.”

Continues Hook: “We’ve done hundreds of hours of live corporate events using Smart Stage and a big part of that is teleporting CEOs and keynote speakers into the space. We’ve been doing a lot of work on how we improve that remote acquisition and also how we improve the depth of capture and the automation of the set-up process.”

Seven servers are configured as one unified, networked system controlled from a single GUI. Some of those servers are render nodes rendering out the Unreal realtime world/set, others are generating the outputs to the LED walls and the composite camera feed with AR, and one is acting as a dedicated ‘Director’ – providing a single control point for all of those and keeping everything in sync.

“The disguise GUI is a full 3D realtime visualisation of everything that is going on in the studio, giving the operator in the gallery a view from any angle and also the ability to work offline and pre-visualise any changes they might want to make or rehearse. It also means we could have a world built in Epic Games’ Unreal, then add within the same 3D space worlds built in Unity or notch or another engine. We assemble it all together in realtime.

“We have complete control over what appears in the LED wall (back plate) and in AR (front plate) or both, to the extent that we can have our presenter walk around virtual objects with no occlusion. To the presenter, the AR version of the athlete being interviewed looks a little skewed but their eyeline is completely natural and that’s what is important for the interaction to work.”

The set design was briefed by Eurosport and White Light to set designer Toby Kalitowski, creative director at BK Design Projects. His design was built in Unreal Engine by Dimension Studio. WL loaded that on to the disguise servers and is treated as a video source. The set’s background includes a window which is treated as another video source – for the French Open it was often a panorama of Paris.

Show control
Unreal was designed as a game engine not a show control system so being able to trigger events in Unreal, to change lighting and to sequence that process has traditionally been difficult.

“What we’ve been able to do is provide a great front end so that the operator in the gallery is able to respond to any request that the director might come up with and trigger changes in Unreal and whatever other content is coming in.”

Lighting control is also vital and with the new Cube the lighting director is able to synchronise both real and virtual lighting. If they fade up a real light in the studio, the system will turn a version of that light on in the virtual world at the same time.

Likewise, if the colour temperature of the live feed in the virtual window changes, the LD can change the studio light to match.

BlackTrax tracking, also new, automatically lights the presenter as they move. “Rather than having lots of different lit positions depending on where (Barbara) stands in the space, the lighting automatically maintains a perfect light on her,” Hook explains.

“Barbara creates a natural shadow on the floor because she is being lit properly with a key, fill and a back light. You also get realistic reflections on the set because the virtual environment and video sources reflect on to the walls, an interactive table, on to glasses, jewellery. It’s those little things that add believability and cause people to question whether what they are seeing is real.”

As mixed reality technology becomes more sophisticated and affordable it will be increasingly used in broadcast, but skills need to keep pace with the technology.

“As we’ve gone through different MR broadcast projects we’ve realised there are many elements which are outside the comfort zone of normal production teams or that needed to be thought about in a different way, or that require different roles,” says Hook. “For example, you may have a traditional broadcast set design which is handed over to someone who will build that into a realtime virtual environment. The design can get lost in translation.

He advises: “The lighting director needs to get involved early in the virtual studio production to make sure the lights are in the right place in both worlds. You need to plan shots around what will and won’t be seen in the real world and in the virtual world. You also have to consider that you only have one LED volume and it can only show content to one perspective at a time. All the cameras are tracked and you can cut between them live with the content in the LED wall cutting in sync to show the correct perspective. Cutting live is easy, but if you want to re-edit in post-production you need to have planned what any ISOs or B-roll look like on the other cameras as the background could be wrong.”

Adds Dinnin: “Before, it was a graphics project. Now it operates much more like a normal TV studio. We have a director and vision mixer who have never worked in mixed reality before who are able to sit down and work it out. If they want to change the background video source they just a push button. It is much more broadcast oriented and because it’s a 3D studio it can be reversioned to suit whatever sport we want.”

The Cube’s next outing is for the Australian Open in January, then comes next summer’s Olympics. Reveals Hook: “There are definitely tricks we haven’t shown yet which will show we have moved the goalposts even further on.”

 

Behind the Scenes: Nomadland

IBC

Filmmakers Chloé Zhao and Joshua James Richards discuss their naturalistic approach to stories about American life in the margins.

https://www.ibc.org/trends/behind-the-scenes-nomadland/7142.article

A docudrama about a group of drifters on the margins of American society has become one of the leading awards prospects of 2020. Acclaimed at festivals from Venice to Toronto, Nomadland was inspired by Jessica Bruder’s 2017 nonfiction book and documents a transient community of mostly middle-aged people living out of cars and vans in parks and campsites across the midwestern U.S. 

Writer-director-editor Chloé Zhao’s naturalistic approach embraces many of the lives of these people and a script that evolved as she met them. 

“I am not a documentary maker,” Zhao says of her filmmaking philosophy.  “I think [documentary makers] are so incredibly brave to get know a subject as if what [they] are filming is what is real. I don’t know how to do that. [But] I think audiences want authenticity in their viewing experience and I find that having authenticity in fictional filmmaking really helps to keep their attention.” 

She first adopted the approach in her debut feature, Songs My Brother Taught Me and 2017’s The Rider which were both filmed on the Pine Ridge reservation in North Dakota with a cast of non-actors by British DP Joshua James Richards. 

Nomadland was brought to Zhao as a project by her friend, the actor Frances McDormand, who also plays Fern, the film’s central character through whom we meet the nomad community. Fern has to abandon her home after losing her job. She journeys from Nevada to California via the Badlands of Nebraska taking seasonal minimum wage jobs along the way. 

Zhao was emboldened in telling Fran’s zig-zagging story by road movie classics like Wim Wenders’ Paris, Texas and Agnes Varda’s Vagabond. 

Little visual details tell us as much about the characters as anything they do or say. “Early on [we show how Fern] decorates her van,” says Zhao. “What she chooses to bring with her says so much about who she is because space is so limited. It’s the art of editing down.” “The point of the road movie is not to get from A to B,” she says. “If [Nomadland’s characters] had a goal to work toward maybe their lives would be easier, but this meandering is when loneliness kicks in and you have to get to know yourself because you are not hurrying to go buy a house or chasing a bill.” 

The verité style is shot handheld on Alexa Mini by Richards. “Something people need to understand about Chloé Zhao is that she is a really conceptual filmmaker,” the DP says. “She comes at things looking at how she can tell the story in a new and innovative way.  

“It’s a shame that because of identity politics most of the questions put to her are about being a Chinese woman making movies,” he adds. “It would be great from a filmmaking point of view if we can past that and ask what’s it like to be challenging the way people make films in America. That is so much more exciting and progressive to think about because that is how Chloe approaches every project.” 

Embedded experience 
Zhao, Richards and a skeleton crew lived among the communities while filming, adapting the script to the personalities of the people they met. Directing assistant Hannah Peterson had previous experience of working with non-actors on The Florida Project, a film about living on the breadline in the shadow of Disney World. 

“When Hannah was out there on location taking pictures and filming iPhone footage of drifters talking, Derek [Enders] caught my eye,” Zhao says. “There is something in the way he speaks you just know you want that to be recorded. It’s a feeling.” 

She continues, “In my script I had wanted Fern to meet someone who is young, someone to make her think about the child she never had, or someone similar to her husband or who would remind her of when she was young. We had originally scripted [this] to be a pregnant woman but we couldn’t find anybody that I was drawn to.” 

Zhao listened to Derek telling his personal history until she found a moment that she felt would work for Nomadland. “I would stop him and ask if he could say it again, but in a different way, that I know is going to fit into the story. That happens quite a bit in the film. Based on little moments like these I’ll then turn around to Fran and ask her to hurry over and together we work out what lines in the script I need her to say and that I can use in that moment acting opposite Derek, or the other characters. To work that way you need to have a cast and a crew [willing] to put up with your madness.” 

She adds, “I don’t rehearse either. As soon as the idea for the scene is green lit in my head we go and shoot it.” 

Travelling spirit 
Zhao says the story resonated with her own nomadic upbringing. Born in Beijing, an only child, Zhao attended boarding school in London, then high school in Los Angeles. Having moved to America, she spent a lot of time in the national parks and camp sites of its heartlands.  

“Those experiences shaped me as a filmmaker,” she says. “I wouldn’t be making the kind of films I do if I hadn’t travelled those roads.” 

Her early influences were Japanese graphic novels. Indeed, she named her own van in which she lived on the road during the making of Nomadland, ‘Akira’, after the Manga classic. 

“Growing up I always wanted to be a Manga artist. I devoured Manga,” she says. “If you think about it, Manga is storytelling in edited storyboards. In Manga the storytelling is quite extreme at times in the way the story jumps from one image to the next. So, at a very young age that is how I see story.”  

You can see that in editing style of Nomadland as the story jumps forward in time and place. 

Hong Kong director Wong Kar-Wai was another inspiration. His romantic and evocative films like Chungking Express and Happy Together are also characterised by nonlinear narratives and vivid cinematography. They screened Happy Together for cast and crew during the making of Nomadland. 

“If I look back and ask why I make films the way I do, it comes from those two combinations,” she says. 

The film’s most impassioned speech is by Bob Wells, a leader among RV lifestyles. He talks about people like him being used as “workhorses” by the state before being “put out to pasture” because the social benefits are not enough.  A later speech shows Wells talking about the loss of his son.  

“Bob has that conviction and I think he should say it but I also asked if he had anything that was more universally understood [hence the personal revelation]. I try to make a balance.” 

Although set in 2011, arguably the politics of these desert outcasts and the system which generated them has never been as acute. Yet Zhao doesn’t show us anyone wearing a MAGA hat; there are no election posters; the current politic situation is not mentioned. Fran takes temporary work at a massive Amazon warehouse and lives out of the van at an RV park seemingly subsidised by Amazon. Zhao films inside the warehouse and includes some of its genuine workers as supporting actors. But big business is not demonised.  

“I think it’s a conscious decision to stay somewhat neutral because I rather would have one person who disagrees with my politics watch my film and maybe just change how they feel a little bit than a whole room full of people who already agree giving me a standing ovation,” she says. “I’m not trying to preach to the choir. You can guess where I stand politically but I don’t need to make films to appeal to my peers. I want to make films that can bring my peers and those who would disagree with my peers together. So not going too specific on politics is trying not to alienate those who I really want this film, to wake [them]up or even just see themselves in it and realise that this is their situation too.” 

Cornish context 
Richards grew up in Penzance and from the age of 16 decided he would go to New York film school (“I just had it stuck in my mind as the place to go,” he says). He did a creative writing programme at Bournemouth Media School (“I didn’t really show up to be honest, I was down the beach drinking”) but managed to get selected for NYC Film Academy. Here he met Zhao, who was a few years above him. 

“She was the kind of person I hoped to meet at film school,” he says. “Someone who wasn’t like anyone I’d met before, from a different part of the world, who was making a film in the heartlands of America on an Indian reservation. I was a long way from Penzance.” 

While filming Nomadland Zhao landed the responsibility of directing Marvel’s $200m superhero movie The Eternals starring Angelina Jolie. Marvel studio chief Kevin Feige has had huge success inviting indie directors like Ryan Coogler (Black Panther) and Anna Boden and Ryan Fleck (Captain Marvel) to bring their sensibilities to mega-budget studio shows. 

The Eternals, though, is photographed by regular Marvel cinematographer Ben Davis BSC. Richards isn’t unhappy. “I want to carry on down this rabbit hole of choosing projects because it’s something I want as to experience as a human being,” he says. “I am not down on bigger conventional shoots or even studio things but it’s not necessarily an experience I want. I am looking for new life experiences or to generate some of my own content if it doesn’t come my way.” 

2021 predictions: What to expect in post-production

IBC

The only constant is change as the post sector gears up for remote distributed permanence, more virtual production and a bounce back

https://www.ibc.org/trends/2021-predictions-what-to-expect-in-post-production/7140.article

Sherri Potter, President, Technicolor 

“Nobody really likes change, but the reality is that this is forced change. Like the tsunami of 2011 which wiped out tape stock and forced the industry to move to file workflows, 2020 was a futureshock for the industry in terms of remote post-production. Where the industry was resistant to working at home in terms of security, now we’re all able to set up securely and carry on working. 

That said, the one thing we should never forget is the human interaction you get with peers and other creatives. There’s still a need for people to come together to inspire one another. If you stay remote permanently you risk losing that.   

In 2021 we will see further adoption of virtual production, but widespread use of virtual sets and pipelines is premature. When you’re prepping a movie and deciding whether to greenlight it do you need to have that many crew going to a certain location when you can create the environment locally? It’s a very different concept to how people are used to working. There will be filmmakers and studios more willing to embrace that than others. 

There is an argument that the legacy of 2020 will be to accelerate the move to remote distributed post-production and pop-up facilities. While I can understand the benefits of this approach for certain companies or productions, I don’t think we will see a huge shift in this direction in 2021.  

Throughout Technicolor’s history we are proud that never miss a delivery. We cannot fail. The reason why studios come to us is that they entrust we won’t fail and that we will throw whatever resources we need at the problem to ensure that even the most complicated production delivers. Schedules across TV and features are becoming more compressed as demand for content ramps up. What’s more the creative process now continues throughout the whole post process so that where creatives request changes to their vision we can respond and deliver at any point. 

So, will there be a need in future for massive bricks and mortar buildings? No. Will facilities be able to reduce their footprint? Absolutely. But there will always be a need for a hub with the IP and talented individuals that a producer can rely on? I believe so.” 

Leon Silverman co-founder HPA, chair HPA Industry Recovery Task Force  

“In 2021, we will continue to find innovative ways to bolt together these new connected, collaborative creative tools tying our remotely accessed post production and VFX infrastructure to the cloud and to the homes and locations of the dispersed global filmmaking community.  

While post production service providers will continue to be key in providing creative finishing and workflow support services, there is no doubt that the pandemic will be a defining moment in how our post production physical facilities model continues to evolve.  

“Our new finest hours in post will come from our usual resilience and ability to innovate and iterate. We will need to adapt to and adopt the best ways to serve and integrate within an increasingly cloud hosted, highly distributed, software defined, connected, collaborative, content creation ecosystem. There is no doubt that this is the industry’s future and vision and to me, there is no doubt that our post production community will, as they have in the past, rise to the challenge.”  

Dave Cadle, CEO, Envy  

“2021 will be a vibrant year. There is so much production backed up now that needs to get through the pipe it could be a very productive year. So much so that we will need to get more infrastructure out to client buildings and to our operator’s homes. We also need to look at other work and diversify with involvement in more fixed-rig style work. That’s why we’re opening a new division in January called Capture to focus on remote and fixed rig shows.  

I predict that the post market will consolidate and I wouldn’t be surprised if a few companies joined forces and merged their resources to look at the bigger picture. Oddly, though, I can’t see a lot of space being lost in central London. If anything, the sector will add more craft rooms that can be switched between home and away.” 

Loren Nielsen, VP Content & Strategy DTS Inc / Xperi; Member of the Board of Directors for the HPA and International Cinema Technology Association 

“In the first wave, while many people were furloughed, the sector still had work in the pipe and to an extent could continue remotely. That work has mostly been completed and still awaiting distribution while work that would have been coming in now is trickling back in the second wave of Covid-19. 

The ability to work from home securely has become essential as Covid forced everyone into remote forms of post and collaboration. These tools will continue to develop rapidly along with our understanding and application of ‘at home’ workflows. This period of rapid adjustment and global deployment will be looked back on as a renaissance in work / life balance. 

The tech we need to see in 2021 is an extension of this. In particular we need greater integration of tools to work across cloud platforms whether AWS, GCP or Azure. We will see development in the role of the DI toward cloud integration. Just as a DI technician on a production ensures that content is prepared in the right file format, new skills are needed to manage the ingest and egress of rushes from set to cloud and back. 

All of this ties into bandwidth; 5G could make a difference, as will faster fibre broadband infrastructure. Something had to give here since we are all just sucking that bandwidth like crazy. 

Diversity and inclusion is another area where we will see greater strides in 2021. The HPA has been developing programmes to promote inclusion for years including one it recently started with SMPTE. Xperi has a corporate diversity program too and I’m aware of the great advocacy work of RISE. I think we will see a change in the industry because of this. As we increase the representation of women and other groups we will see different types of stories told and different types of workflow. It’s hard to define exactly what it will look like but we will see companies, products and certainly a lot of opportunities for conference panels, to look and sound much more diverse. 

Technology is historically very male and largely white oriented but the ironic thing when comes to engineering, math and science is that many of our young tech recruits excel in these subjects.  We need to continue to support them through their career so that they don’t just become tech burn outs.” 

David Sheldon-Hicks, Founder and Creative Director, Territory Studio 

“Everyone is talking about virtual production.  When we worked with LED screens on David Fincher’s Mank we decided to go down a pre-rendered route rather than using a realtime game engine. We still had the benefits of lighting the scene with the LED and the background replacement happening live, but we didn’t have the investment of getting realtime assets ready, which for this project would have been unnecessary.  

What I’m hoping to see is art departments really engaging with this tech and driving the creative forward. As designers of the vision they are perfectly placed to bring LED usage alive in beautifully cinematic ways. 

There’s no reason why indies can’t consider virtual production as a replacement for green screen or location work where travel becomes an issue. 

AI and machine learning is developing at speed and I suspect is going to change our industry at the very core. Some of the AI algorithms that can now age and de-age people in film clips is incredible. The potential power and impact on the industry is significant and especially relevant to team sizes and structure. We may no longer need VFX artists to perform mundane tasks and can instead focus small teams of highly skilled specialists to work on really creative challenges. That change won’t happen overnight, but over the decade I suspect it inevitably will. My hope is that it brings directors and filmmakers closer to the creative process of VFX. It will also mean we need to look at training and how we develop people. Ultimately it could lead to better working conditions and enjoyable work for more people. 

Despite what is happening with Brexit I feel compelled to work with European partners and projects. We will invest in hiring across Europe, as it’s a source of incredible talent, even if that means increased investment from us. Brexit sadly signals to our European partners the wrong message and I suspect it falls on British industry to do our best to counter that sentiment and work hard to get past the challenges ahead. I’ve got a personal care and love of Europe and the inclusion it stands for, and whilst it’s not perfect it is a wonderful creative community to be a part of.” 

Adam Peat, Director of Technical Operations, The Farm Group 

“For 2021 we see demand for a hybrid scenario of traditional facility attendance and remote working continuing. Our aim is to make this as flexible and efficient as possible for our clients, so they can work where and when they want. Remote working tools will continue to be developed for use in the post production sector and make working remotely even more attractive where suitable.  

Our existing ability to provide collaborative remote editing, live and non-live content streaming in formats from HD SDR to UHD HDR will continue and improve with further development. Increased virtualisation will allow us to be even more responsive to demand, and increased content analysis will also help us to make the post process more efficient. 

In tandem, increased investment across the regions will enable us to accommodate the demand for more UHD/HDR/immersive audio work which we continue to see across both scripted and unscripted genres, and the ability to provide a joined-up post pipeline across multiple regional centres.” 

Tim Pipher, owner, L.A Castle Studios   

“L.A. Castle Studios [a virtual production studio in Burbank, CA] has definitely lost bookings that otherwise would have come through due the pandemic. On the other hand, because virtual production is generally considered to be a safer way to shoot, we’ve gained other bookings. Overall, counterintuitively, our bookings are up considerably.  

Production will be heavily up in 2021 due to pent-up demand. Virtual set production will especially be up due to lower required personnel (thus safety), but also for cost and creative advantages. Virtual production will be up at all budget levels. Lower budget productions have realized that virtual production levels the playing field, allowing for million dollar sets and locations without spending the million dollars. Big budget productions like to save money too, so they’re also enthusiastic. Producers are realizing there’s no need to build real sets when virtual can offer total reality with greater speed, lower costs, and a greener environmental impact.”