LEDs: the new digital palette

written for ROE Visual in British Cinematographer

Not all LEDs are created equal. As virtual production increases, DPs need to get to grips with the nuances and limitations of working with LED volume.

https://britishcinematographer.co.uk/leds-the-new-digital-palette/

The pandemic has prevented many creative directors from making content in the traditional way, forcing them to explore alternatives. This has coincided with a coming of age for the next generation of virtual set technology, with increasingly powerful photorealistic real-time rendering changing the way content is made.

“The use of image output from real-time engines to a live LED wall in combination with camera tracking to produce final-pixel imagery, completely in-camera, represents the state-of-the-art for virtual production,” says Victor Kortekaas, technical director, ROE Visual Europe. “When COVID hit and the studios were forced to close down almost overnight, attention was focused on how virtual production stages and remote workflows could combine to get content creation back up and running. At the same time, there’s been a confluence of technical advances in processing power, real-time graphics, camera-tracking systems and LED screens all combining into a perfect storm of possibility.”

Allowing the heads of production and acting talent to see what’s happening around them instead of a blank green screen, creates a more natural flow to a production. Virtual production removes the need for chroma keying in post and minimises travel (time and cost) for crew. Since LED screens enable realistic reflections and refractions, DPs and camera operators can easily see and explore the scene in-camera, again allowing for an easier workflow.

“Demand for virtual sets has never been higher across all verticals, from feature film and episodic drama to sports broadcasts, live events and advertising,” says Kortekaas. “By its nature, it is heavily reliant on technology. If you want to build a virtual stage, you don’t want to find out that it won’t work in the way you imagined when you’re in the heat of production. It is more important to have an open and honest conversation about what it will take to prevent problems arising.”

LED panel specifications

ROE Visual has considerable pedigree in this field. As a designer and manufacturer of LED screens for many years, ROE Visual’s technology has been sold to rental companies for use in film and TV production as well as for permanent installs at the world’s most prestigious studios including ILM Studios at Pinewood. Its screens have been used by (among others) NantStudios in LA, NEP Group xR stages, LMG’s Collosal Cube studio and Pixomondo and for production of several movies and episodic TV shows including Nightflyers, Westworld and most notably on both seasons of Disney+ ground-breaking virtual production The Mandalorian.

“Building an LED panel is not that difficult but to build one with the quality and reliability demanded by media and entertainment producers is an order of magnitude more challenging,” says Kortekaas. “We don’t give you a box and wish you good luck. We want to help you with any question and because we partner with all the leading players in the field including ARRI, disguise, Lux Machina, and Epic Games (Unreal Engine) we believe we can. Only by combining our knowledge, endless testing, and syncing the equipment used will optimal results be achieved.”

There is no off-the-shelf virtual production system. Building one requires in-depth knowledge and experience with all the facets and elements that come into play. When it comes to LED screens there is more than meets the eye.

On-camera performance

“Consistency and quality are a hallmark of our LED technology. If we replace a panel it has to have the same quality ingrained and it has to fit with the existing screen set-up. We support install and set-up with 1-1 training sessions and with technicians onsite,” says Kortekaas.

Synchronising input sources to the camera and playback on screen is critical to the success of the virtual stages production. “Syncing the camera settings with the LED is a process that requires time and precision. If your LED panels and camera are not well synced, this may result in all types of visual disturbances or artifacts, like ghosting, jitter, or image distortion. We carry genlock all the way to the end and have a lot of features to make sure we adjust the genlock up to the nano second.”

Issues to be worked through upfront include Pixel Pitch, a factor defined by the use or the type of shots you want to make. The more close-ups you need, the smaller the pixel pitch is advised, but beware that finer pixel pitch means less light thrown from the LEDs for your scene.

The screen’s refresh rate has a baring on visual quality. The larger the difference between the refresh rate of the LED screen and the camera, the more difficult it is for the camera to pick it up. A high frame rate may be desirable especially for fast pace live-action, however, today there are still practical limitations on the content rendering. Even when the LED screen can perform at 120 frames per second, it can be difficult for the rendering to keep up.

Fine tuning colorimetry

Perhaps the biggest questions concern colour and colour accuracy. “LEDs have a certain colour range with certain colours that can’t yet be attained at the outer reaches of the palette,” explains Kortekaas. “This is most noticeable in shows calibrated with a Rec.2020 colour space and HDR. We can explain what can and cannot be reached within the colour gamut of the LED. It’s essential to maintain a stable colour display under different viewing angles, making the virtual scene more realistic. Because we can measure the raw data of every single LED and because we work closely with digital cine camera makers like ARRI we can calibrate the colour science of the LED volume to the sensor and the specific needs of the DP on any project.”

On Mank, for example, Erik Messerschmidt ASC and colourist Eric Weidt, to director David Fincher’s vision, opted for a digital black-and-white film emulation. That project, which used ROE Visual screens for much of the film’s Oscar-nominated VFX work, required a completely different LED screen set-up than for another Best VFX Oscar-nominated science-fiction film which also used ROE Visual technology.

“We try to optimise every aspect of the technology for the creative behind each project,” says Kortekaas. “The production should not have to worry about the quality and reliability of the LED, but just concentrate on creating their vision on set.”