Will Motion Grading become a new cinematographic tool?

RedShark News

When Tom Cruise urged us all to switch off motion smoothing he knew it was mission impossible. Even if people do make the effort to retune the default setting on their TVs, the outcome is often worse.

https://www.redsharknews.com/will-motion-grading-become-a-new-cinematographic-tool

Motion smoothing certainly helps remove picture imperfections like strobing, blur and judder but it leaves the image looking less filmic. The disparaging term for this is that it leaves cinematic looking shows looking like a soap opera. That’s because TVs use video interpolation to artificially boost the frame rate and smooth the viewing experience of fast-action sports and gaming but leaving it on can “make most movies look like they were shot on high-speed video rather than film,” according to Cruise.

The situation appears to be getting worse. Line up a dozen different TVs and you’ll see a dozen different looks, none of which match what the DP saw in the grading room. HDR only amplifies this issue.

“Judder increases by factor of three or more and shutter speed seem faster and more choppy and there’s nothing a TV can do about it,” says Richard Miller, EVP Technology at Pixelworks. “Motion smoothing can take off the worst of it but it brings in these other artefacts and Tom Cruise will come looking for you.”

But there is a tech solution riding to the rescue.

Motion control

The appearance of motion is affected by factors like camera and subject motion within a shot, contrast range and frame rate.

“When you watch streaming TV the image you see does not match the approved reference in the grading room,” explains Miller. “The issue is exacerbated by brighter screens, larger screens and higher dynamic range and increasing use of more aggressive camera work.”

Some of the issues include:

The Queen's Gambit - Episode 1, approx 3 mins and 5 secs in showing a pan with a bridge.

Stranger Things - S3, Ep2 - about 25 mins showing a pan inside a mall. 

Unbelievable - Ep1, the opening sequence of a pan across a street.

The Stranger Things example is most noticeable where the bright lights and colours seem to blur and the lines of the mall jag. You want to tear your eyes away.

“The dilemma is that most users have motion smoothing turned on,” Miller says. “This is the way the TV makers determine is most comfortable for users. Filmmakers hate it. It causes the fake soap opera effect- artefacts like halos behind heads. Sometimes it janks back and forth. You wouldn’t know that unless you were an expert but over the course of a show it does change the feel of storytelling. Spielberg was looking for that harsh hard look in Saving Private Ryan to make you think you are on that beach on D-Day, not something so smooth it destroys any suspension of disbelief.”

Miller explains that judder becomes much more objectionable when the overall brightness is a lot higher than usual. This is a human perceptual effect that’s becoming a problem as more and more filmmakers look to make use of high dynamic range and more devices are capable of displaying it.

“If too much contrast is put into the grade, judder and strobing can become a real problem,” Miller says. “While HDR is supposed to go up to 10,000 nits, most of what we are seeing is actually around the same as SDR. It does have wider dynamic range so you get a bit more in the dark areas but colorists are having to compromise their HDR grades in order to avoid judder artifacts.”

He says, “If you want to use HDR you have to deal with motion. HDR without motion grading is compromised.”

Pixelworks, a Silicon Valley-based imaging specialist, claims to have developed the only solution to the problem. The motion grading capability is layered onto the company’s TrueCut content creation tool and delivery platform. It has been out for a while and now the company is making a new push, hinting that streaming services are getting ready to back the technology.

“TrueCut allows the filmmaker to control the judder, shutter speed or frame rate for an entire show or shot by shot,” Miller explains.

A key element is a Motion Appearance Model. Just as algorithms (such as TrueCut’s Color Appearance Model) are used in the grade to maintain the perception of a colour hue in any lighting environment, the MAM does the same for movement. When combined with a Color Appearance Model, a new Motion Picture Appearance Model is created. This is the first time such a model has been developed.

How it works

The software works during capture, into post and through to distribution.

On-set, much like a Colour Decision List is created today, cinematographers can use the TrueCut tools to preview and create an initial motion look which is captured as a Motion Decision List. TrueCut Motion can be used to deliver this motion look for dailies as well.

It provides a ‘virtual frame rate’, enabling higher frame rates to be used where appropriate in production, while retaining a 24fps filmic look if desired.

“Even though HFR scares some people, we are focused on keeping a 24 fps look to the final show if that is the director’s intent,” says Miller.

Members of the American Society of Cinematographers were also consulted for their input and Pixelworks says feedback is positive.

In post-production, TrueCut Motion is generally used after colour grading. It’s available as a plug-in to DaVinci Resolve. For example, the colorist/DP could adjust Judder on a scale from 0 to 360, where 0 is the judder typically seen in 24 fps footage and 360 is no perceptible judder. Motion Blur is calibrated on a scale from 0 to 720, where 0 is no motion blur added to the original, and 720 is 720 degrees of effective shutter added.

For home streaming, Pixelworks has developed TrueCut as a high frame-rate streaming format to play back across TVs and mobile devices. It is compatible with HDR formats like Dolby Vision and HDR10.

Pixelworks says the format is supported by streaming services and suggests that announcements will made later this year. You’d expect that this at least one of Netflix, Amazon and Disney+.

It explains that, when selecting a title, the TV or STB streaming app will play the TrueCut motion graded version if such a version of the title exists and the device is certified compatible. This is similar to the approach used already to determine whether to play the SDR, HDR or other formatted version of the title.

TrueCut is backwards compatible with “tens of millions” of TVs in homes today including many if not most of LG displays and Samsung TVs sold since 2019.

“We expect most new models [of any TV maker] coming to market from 2022 will support the format,” says Miller.

In China, six features have been mastered using TrueCut Motion including Pegasus and The Bravest. Sony has also remastered Men In Black: International as a showcase screening for its theatrical LED screens.

With a few of the top streamers on board the workflow is likely to take off. Time will tell.