The Path to Holographic TV

NAB

The Tokyo Olympics was always intended to showcase Japan’s technological prowess and, as a curtain raiser, the R&D lab of state broadcaster NHK is throwing open its doors.

https://amplify.nabshow.com/articles/the-path-to-holographic-tv/

For NHK this is an annual affair, but this year its month-long open house will be a virtual affair, although that does mean potentially wider access given the restrictions on travel to the country.

As expected, 8K is front of center of the NHK Science & Technology Research Lab’s (STRL) achievements. The Tokyo Games will see the most number of hours from the largest event yet broadcast in the 8K Super Hi-Vision format, domestically of course.

It has remastered classic movies like My Fair Lady, 2001: A Space Odyssey and West Side Story into 8K and produced several 8K drama including Taiyo No Ko (Gift of Fire), coproduced by NHK and Los Angeles-based distributor Eleven Arts.

While the rest of the world remains in SD, STRL has moved beyond 8K. Nor does it seem to be developing any further resolution-based TV formats. Instead, immersive content production and display technologies are to the fore.

“Around 2030-2040, we expect that in addition to smartphones and tablets, AR/VR devices, haptic devices, and other devices will be widespread among viewers and that the ways in which they engage with programs and content will be even more diverse than they are now,” is one its mission statements.

STRL’s research aims to realize a “diverse vision,” which it calls a “new concept in broadcasting” that will allow anyone to view and experience a variety of content using their favorite devices regardless of time or place.

To enable “diverse forms of expression for program producers and different ways to enjoy content for viewers”, a variety of information will be captured. To that end, “meta-studio” technologies will be used, in which video/audio is not merely recorded, but in which detailed information on the subject’s shape in three dimensions, its texture, touch, and resonance of voice and sound are acquired.

New video processing technologies using VFX and super-resolution technology will be exploited. In addition, AI will be “proactively introduced” to simplify the production of high-quality content.

“Important scenes will be automatically extracted and edited via video/audio analysis. Al will also be utilized to generate metadata such as closed captioning and production data and to produce programs using accumulated know-how and previously gathered material.”

The concept seems similar to that of object-based broadcasting being pursued by BBC R&D among others. This is the idea that programs can be broken down into constituent parts (as data) to be reformulated on the viewer’s personal device in accordance to both individual preference and device characteristics (like bandwidth).

“Since the same content can be adapted to be played on devices with different specifications and in different viewing environment, the viewer does not need to be aware of their viewing styles to enjoy the content,” says the Lab. “For example, a viewer in their living room enjoying a full-featured 8K with a large sheet-type display, can at the same time watch a scene of special interest from various angles on a table-type or portable 3D television.”

 If the viewer wears a head-mounted display or a device that reproduces haptic senses, the viewer can watch the same content from either a performer’s or the audience’s perspective, “or feel the shape and hardness of an object.”

We should be taking NHK seriously. In its 90-year history it has played leading roles in bringing the world high-definition and UHD television, plasma displays and direct to home satellite broadcasts.

Among its other projects is research into haptic devices that vibrate according to the scene of the program, and a system that automatically produces sign language CG for weather information across Japan. Both are an aid to media accessibility.

It is also exploring ways to create spatial imagery. That’s holography by any other name. Steps on the road to this include shooting techniques using multiple cameras and display technologies “using high-density light field reproduction”; and high-speed, high-density display devices to create high-quality displays of 3D motion images.

“We are working on 3D TV spatial imaging reproduction in order to enjoy natural 3D images without the use of special glasses,” it explains.

“We are aiming to realize a high-speed optical phased array to extend the depth range and viewing zone of the integral 3D image, and a spin spatial light modulator (spin SLM) to display holographic images with a wide viewing zone.”