Practical advice for lighting the volume

IBC

article here

If virtual production is to sell the illusion of what’s being filmed the LED lighting and background environment must be merged with physical sets and practical lighting as seamlessly as possible.

A LED volume not only provides an extension of the scene environment it essentially acts as a massive light box. Light emitted by the walls can be used to create dynamic reflections that interact with the set and actors in realtime. This lighting can be adjusted and fine-tuned by using light cards as well as colour and brightness controls.

“While the volume is a great base source of lighting we highly recommend pairing it with traditional practical lighting for the best result,” says Jamie Sims, VP Projects Manager at MARS Volume. “This is where a skilled Unreal Operator can make a huge difference. Our Unreal Operators and VP Supervisors work hand in glove with DOPs and gaffers to achieve the creative vision.”

Dan Hall, VP Supervisor at Slough’s Virtual Production Studios by 80six says, “Candles, lamps, even fish tanks are fantastic examples of practical lights because they’re subtle and give you an accurate representation of how light will work in a room. Additionally, it takes the eye away from the background, which should not be the focal point.”

Soft and hard lighting

LED panels are ideal at creating soft lighting which generates soft edged shadows but they can’t produce hard light such as hard edged, crisp shadows, spot lights or ‘beauty lighting’. This is where working in creative collaboration with the Gaffer and DoP on a production is crucial to creating the required look.

“While LED screens are an excellent source of interactive lighting and reflection they are behind on colour rendition when compared to today’s practical LED fixtures,” says Sam Kemp, Virtual Production, Technical Lead, Garden Studios.

Hard light is produced by a point source light, such as a tungsten Fresnel or an LED point-source fixture. Consequently, a volume without any additional fixtures can't produce hard light and therefore scenes in daylight require the addition of practical fixtures to 'sell' the idea of direct sunlight.

Kemp notes, “Practical fixtures can replicate hard sources such as sunlight and also help to fill the spectral deficiencies of RGB LED panels. Standard lighting communications control like DMX can be used from the engine for synced effects.”

Image Based Lighting

Image Based Lighting (IBL) is a form of pixel mapping that uses calibrated photographic (video) colour (RGB) information to generate subject and environment lighting. The technique – which some practicioners describe as a philosophy - uses images and lighting displayed on LED sets to produce realistic reflections and ambient lighting in a scene.

“The three main benefits are accuracy, time saving and control,” says Tim Kang, Principal Engineer, Imaging Applications at lighting vendor Aputure. “The biggest one for me is control. We’ve been chasing naturalism in lighting for 100 years but have only been approximating the real world. With IBL you can get the naturalism you want and you can control the variables and much more directly.”

Garden Studios has been using IBL since 2021 primarily for driving and VFX heavy scenes. It has recently developed a workflow for tracking hard sources, allowing for a sun source to automatically move around a car driving down winding lanes.

“The key is finding a good balance between IBL and traditional lighting controls; between the VP team and the Desk Op,” says Kemp. “Image based lighting doesn't really apply to specific sources when talking about practical fixtures (such as a normal light on a stand) and more to the conceptual control of those sources, such as mapping the colour and intensity of a video to a light fixture’s output colour.”

An accurate colour pipeline is key to matching colours, and this includes the pipeline for IBL. Allowing adequate time to complete camera calibration leads to a smoother shooting experience.

“Garden Studios calibrates its screens’ colour pipeline so virtual fixtures lighting virtual content will correctly match their physical equivalents,” explains Kemp. “A colour meter helps match lighting from LED panels (e.g from a ceiling panel) to physical fixtures, as does using DMX modes such as CIE-XY (which denotes universal colour space representing the colour spectrum visible to the 'average human'). Newer fixtures can define a source colour space when using RGB modes for pixel mapping.”

It's not always as straightforward as it sounds since identical LED panels might have been produced in different batches and therefore emit light differently.

“Assuming that the colour pipeline has been set correctly for the Volume, we can pixel map lighting fixtures from the environment to ensure accurate colour replication,” says Hall. “But trying to match an LED panel and a lighting fixture, that are in no way identical, is extremely hard as they display different colour gamut. You must ensure your colour pipeline is set correctly and then dial it by eye. You have to trust your trained eye to see what looks right or not.”

Virtual and real camera team collaboration

The clear advice to production is to pair the DOP, Gaffer and Production Designer with the Virtual Production Supervisor at the earliest stage possible.

“We always recommend a pre-light before a shoot so that the gaffer and DOP can run through all of the shots and lock off any variables before the shoot day,” says Sims. “Working in a Volume gives you so many possibilities, but with that we find that leaving the experimentation to shoot day is an unwise strategy - as it can lead to the time on a shoot day running away. A pre-light day is highly recommended to find what works, confirm approaches and lock everything off so that when it comes to shoot, everything can be achieved quickly and smoothly.”

It is also important for the Production Designer to be “synced” with the Virtual Production Supervisor from an early stage in production. Sims explains, “This is to ensure that the virtual set can be married up to the physical set that is being built. This becomes especially important when trying to make the line between virtual and physical set seamless. Once the set is built and in situ the VP team can then colour match the virtual environment to the physical set.”

Matching practical set and fixtures with virtual assets

Some of the biggest challenges on a virtual production set make themselves abundantly apparent when trying to extend the physical elements of an environment seamlessly into the virtual world. The complexity of this challenge completely depends on what it is you are trying to bring together and the illusion you are trying to masterfully create. 

Sims cites the example of attempting to convincingly marry physical and virtual sets for the outside of a building. “You need to match up straight solid lines and subtle block colours so anything that isn’t bang on perfect or colour matched will be glaringly obvious. This also means your camera tracking needs to be inch perfect to avoid jumping or unwanted shaking.”

Less challenging environments are ones where the line between physical and virtual aren’t as strict, for example, a sandy desert. Colour matching is vital here to sell the illusion.

“To overcome these challenges, we have to underscore the importance of the pre-light day, and getting up close and personal with your VP team at your volume stage. Construction collaboration is key here. The more time the VP Supervisor has to colour match with the set in position the better. Set build days and pre-light days allow for this care and consideration to be taken.”

Fighting on a freight train

Garden recently shot a fight scene on a moving freight train with its custom lighting controller using a combination of IBL mapping, DMX cues and OSC variables (Open Sound Control/OSC is a protocol for networking sound synthesisers and other devices for musical performance or show control).

“As the train moves around corners and through a tunnel, a hard-source light array kept the sun in the correct relative position, flickering behind trees, and pixel-mapped LED tubes gave full-spectrum soft fill on the talent, automatically changing intensity in the tunnel,” Kemp explains. “Closeup fill lights were manually set; everything else could be fully automated.”

80Six worked on a recent car shoot where the windscreen was taken out and therefore there was no LED ceiling for the shoot.

“Traditionally, when you shoot through a windscreen while someone is driving, there will be reflections of the sky on the windscreen,” Hall notes. “Because the shoot we were doing was as if the camera were inside the car and we only shot out of the lateral windows, we didn’t require an LED ceiling because there was no reflective surface.

“We put an old school light on a revolving wheel that spun in time with the plate playback to simulate the illusion of orange streetlights passing overhead. The colour of the orange sent to the fixture was selected from the footage of the driving plate.”