Streaming Media
After years of being consigned to niche corners of the industry, codecs are now all the rage. In part that’s because the need to cost efficiently deliver increasingly high bit rate content is outpacing standards-led codec development and in part because of the gap left in the market by the collective failure of HEVC patent holders to offer predictable licence costs.
At IBC, the debate busts out into the open.
V-Nova, for example, will be promoting the merits of LCEVC – or MPEG-5 Part 2 - whilst carefully avoiding stating that the Low Complexity Enhancement Video Codec is based on its own Perseus Pro.
MPEG’s endorsement of Perseus Pro underlines the claims made by V-Nova that the addition of the technology to codecs including H.264, HEVC and AV1 already in the field will improve video quality while reducing processing power.
While LCEVC has yet to be ratified by MPEG as a standard it is no surprise that the first implementation of it is in the Perseus SDK, which V-Nova has now renamed P+.
Among a number of contenders seeking airtime for their compression tech in Amsterdam is start-up iSize. Like V-Nova it is applying machine learning to finesse compression and again like V-Nova it claims to offer a codec-agnostic performance boosting lightweight implementation.
iSize is coming to market late in the day but does so with the benefit of patents based on deep neural networks.
“Instead of internally customising encoder options in a static manner or using boilerplate streaming recipes that are not tailored to the input content and encoder, [we] offer the new concept of deep video precoding (DVP),” Co-Founder and CTO Dr Yiannis Andreopoulos will explain in an IBC technical paper.
The main meat of the codec debate will take place on the conference floor. Real-time decoding and AR playback of MPEG’s video-based point cloud compression standard is being presented by Nokia’s Sebastian Schwarz; while Jonatan Samuelsson of Divideon argues for the emerging MPEG-5 EVC standard, which is effectively a royalty free version of HEVC.
Notwithstanding AV1 being under a cloud of its own now that patent pool holders including JVC, Toshiba and Orange have tasked Sisvel International to protect their interests, expect the AV1 ecosystem to be fleshed out with new product at the show.
In a whole day of sessions sponsored by Intel (to market its Visual Cloud processing firepower), speakers from Netflix, Google, Facebook and Tencent will talk up AV1 for VOD and live streaming.
They can be expected to focus attention on Scalable Video Technology for AV1, a version already announced by Intel and Netflix and designed for encoders to scale their performance levels based on the quality and latency requirements of target applications. SVT-AV1 is capable of running a 4K video stream at 60 frames per second at 10 Mbit/s (using the Intel Xeon processors of course).
It’s also open source for AV1 developers like Harmonic, Bitmovin and Beamr to lend support.
Automation boosts live production
A rack of new product, or rather evolutions of existing ones, promising to automate production with machine learning will dominate the live production space. AI is marketed as the silver bullet required to create, manage, distribute more content to more people with less budget.
The narrative is that AI-infused ingest to archive systems can master the complexity of live production so that broadcasters can maximise their creativity by focusing on story rather than technology.
This will make more sense once entire workflows are ported to the cloud and video is automatically enriched with metadata tagging, facial and object recognition and speech to text.
Among vendors with new product in this area are MAM developer Tedial which its sports centric platform Smartlive; LiveU which is marketing a live sports data-based production platform with ‘smart race car’ manufacturer Griip; Dalet which is arguably the MAM leader in enterprise applications for global news organisations; and IBM which is sponsoring IBC’s behind closed doors leadership forum and will present the automated sports clipping prowess of its AI engine Watson.
Perhaps the most eye-catching product, not least because it is already being used, is TVU Networks’ cloud-based solution for multi-camera production that enables remote audience participation with video.
“The ability to combine metadata generation with timecode information further speeds and simplifies the search process, enabling producers and the AI to locate the exact content down to the video frame,” says David Jorba, EVP, TVU Networks. “From there producers can customise content production tailored to individual audience preferences and platforms.”
Sky Sports live chat show, The Football Social, uses TVUTalkshow, as has new LA-based streaming channel Fox Noire.
Data drives personalised experiences
“The biggest challenge the industry faces in the drive to optimise the user experience is understanding available data, predicting viewers’ motivations, and ultimately using this data to create viewer-tailored experiences,” says Peter Szabo, Department Lead UX/UI at solutions provider 3SS.
Machine Learning has become one of the most crucial research topics in UX. If users can be categorised based on their behaviours, interfaces can be tailored to their needs, ultimately creating the ideal, custom journey for each viewer.
“This categorisation problem is now solvable, thanks to the research related to deep neural networks,” says Szabo. 3SS come to IBC with a mathematical formula to describe this television experience.
Research into a viewer’s emotion and mood as indicators of content preference is also now making headway. Ruwido has developed a system that detects a user’s mood while interacting with ambient voice assistants.
“The main challenge is to understand if people are willing to accept such a system,” says Regina Bernhaupt, ruwido’s head of scientific research. “We also need to come up with ways to visualise mood-based recommendations in the UI.”
Live and VOD content can now be stitched together and presented as a traditional TV channel using technologies like Edgeware Virtual Channel Creation, while at the same time being personalised.
“These channels could consist of content that’s been selected based on viewers’ interests, demographic or location, enabling the development of innovative campaigns and offerings to attract new users and create revenue streams,” says Johan Bolin, CPTO Edgeware.
IP interop and remote production
Some form of IP production tool can be found on most booths but there appear two competing systems: the ST 2110 family of standards and NewTek’s NDI.
“The reality is that we’re unlikely to have a single IP solution,” says Ian Wadgin, senior technology transfer manager, BBC R&D. “What we need is a way for the two systems to interoperate and pass content between them.”
A broadcaster may use ST 2110 in their studio environment but have NDI in their live news production workflows where more compressed workflows are important to deal with less than optimal connectivity.
The answer might lie in NMOS (Network Media Open Specifications) enabling an NDI source to appear on a ST 2110 matrix and vice versa.
Since NewTek was acquired by VizRT in the industry’s most dramatic M&A this year, eyes will be on this integration.
NDI is being positioned under the umbrella brand of Vizrt Group. According to VizRT, “This will afford the NDI brand an increased degree of focus and autonomy, enabling it to deliver more value to NewTek and Vizrt customer solutions, as well as those of third-party partners.
One interesting fly in the ointment is the move to remote production. This concept is rapidly taking hold off in the live events space as routes to transfer AV from a venue to a centralised facility for switching are proving their worth in saving costs.
However, SMPTE 2110 may not be fit for purpose. It was designed to move video uncompressed around a studio hub not over several or even hundreds of kilometres let alone across continents which is what broadcasters need to do for mega-events like an Olympics.
Earlier this year, Swedish broadcaster SVT hosted the largest remote production to date when it routed 80 feeds of the FIS Alpine World Ski Championships from Åre, Sweden back to HQ in Stockholm 600km away. It did this over IP networks from local telco Telia using a mix of uncompressed HD and compressed feeds traffic shaped using Net Insight gear.
“The market needs to understand that remote production and IP are different,” explains Adde Granberg, director of technology, CTO, at SVT. “ST 2110 was made for the studio not for remote. The industry needs to find a new standard for remote video production.”
Embrace 8K
AI won’t be the only fashionable accessory sported by exhibitors. Equally prevalent will be those flaunting an 8K badge.
For some this is an unwelcome distraction from the practicalities of 4K transition, with possible risk of consumer confusion, others view it as the natural progression of an industry which has technological advance written in its DNA.
“There will always be technology Luddites,” says Ben Schwarz, founder CTOi Consulting and communications chair of the Ultra HD Forum. “A devil's advocate would say that 4K is a distraction from HD deployment.”
Back in 2012 when the ITU-R enshrined UHD in two phases, 4K was already seen as a stepping stone to 8K. But UHD-2 was considered so far away that little other than resolution was considered in the specification. That’s why there are strong arguments for deploying 4K with HDR and NGA (Next Gen Audio) or High Frame Rates for sports ahead of an upping of spatial resolution.
BT Sport, arguably Europe’s most technologically progressive broadcaster, is among those currently championing HD/4K with HDR and Dolby Atmos and will show a live broadcast of the Liverpool v Newcastle EPL match in the format.
As it stands, DTH is probably the only viable way to deliver 8K at scale but 8K over IP is following hard on its heels with SVODs like Netflix and Disney+ likely to launch a premium service in the format from 2021 (that’s a guess by the way).
PayTV sportscasters too might be encouraged to launch an 8K tier in the battle for subscribers as more 4K TV’s add 8K upscaling.
You don’t need to wait for MPEG’s VVC codec to be standardised next year to address 8K OTT either. Spain’s Spin Digital will present a new version of its HEVC solutions for 8K live streaming, HDR conversion, and 8K playback at IBC.
The streaming solution runs on Advantech’s encoder platform and delivers 8K HLS and RTP streams with optional cloud support based on AWS to target wide-scale deployments of 8K live events.
It’s a joint development with NHK Technologies, a subsidiary of the Japanese broadcaster that Spin Digital says is planning to offer 8K live services based on this solution.
Meanwhile, China’s state owned AVS working group will present the latest on its
AVS3 codec which is designed for 8K “super high definition videos” and VR applications. AVS patent pool holders include Chinese brands Huawei, TCL and Skyworth.
It’s almost moot whether the eye can resolve the difference between 4K and 8K resolution. More premium content is being recorded in the format in order to deliver a better quality if lower resolution deliverable.
So far as I see it, the interest in 8K is inevitable and exciting. Who wants technology to stand still? Not least because with higher quality source material new forms of content can take shape from decent quality VR to immersive live event viewing as home TVs morph into giant modular wallpaper sized viewing platforms.
No comments:
Post a Comment