AV1: Codec wars erupt

IBC

Just as AV1 gains significant momentum, its royalty-free agenda comes under threat.

https://www.ibc.org/content-management/av1-codec-wars-erupt/3737.article

Reporting on developments in compression is like building with sand. No sooner do you think you’ve grasped it than the silica slips through your hand. Imagine how exasperating that must be for device manufacturers and online content providers who need to get a fix on tomorrow’s roadmap.

The good news for supporters of AV1, a rival or successor to HEVC, is that there is growing momentum behind its implementation - but there’s a fly in the ointment.

On the plus side, Samsung announced its intent to join AV1 development board Alliance for Open Media and seems likely to put the codec into its consumer devices from TVs to Galaxy smartphones.

With Apple, Amazon, Netflix, NVIDIA, ARM, Facebook, Microsoft and Google already AOMedia members the move would seem to lock down AV1 as the codec of choice for streaming media distribution.

It’s only been a year since AV1 specifications were launched but according to analyst Jeff Baumgartner of Light Reading, the codec has made significant progress in terms of adoption and commercial-readiness with “ample evidence that a decoding and encoding product ecosystem is building around AV1.”

Though early implementations of AV1 are software, notably running in Google’s Chrome browser and Mozilla Firefox, Socionext’s FPGA-based encoder (claimed to accelerate AV1 10 times that of software) is among the few AV1 hardware products. More of these can be expected at IBC2019.

New technologies from virtual reality to online gaming and interactive entertainment all have the potential to offer users higher calibre experiences with richer images and a wider range of colours at faster speeds.

“AV1 tackles the challenging technical hurdles to enable this growth – while using compression to reduce data demands,” endorsed Samsung. “By joining AOMedia at the highest level, Samsung will directly participate in expanding adoption of AV1 to help bring better media experiences to customers and their devices around the world.”

Even more significant is the announcement by Intel and Netflix at NAB of a new offshoot of AV1 which is believed will accelerate the codec’s adoption.

The pair are promoting Scalable Video Technology for AV1 (SVT-AV1) capable of running a 4K video stream at 60 frames per second at 10 Mbit/s using the latest Intel Xeon processors.

It’s believed to be the first software-only AV1 implementation capable of such performance which according to expert Jan Ozer “represents an order of magnitude acceleration of AV1 encoding.”

The SVT architecture is not new and is one that has already delivered codecs, including SVT-HEVC and SVT-VP9 but the SVT-AV1 implementation is claimed “unique” by Intel and Netflix in that it allows encoders to scale their performance levels based on the quality and latency requirements of the target applications” - ranging initially from video on demand and in future to live streaming.

While AV1 is marketed as royalty free, there are those arguing that the complexity of implementing AV1 in practice mean that its compute cost is higher than HEVC and many times higher than VP9.

With the processing muscle of Intel behind it, SVT-AV1 could bring that cost down.

“The SVT-AV1 collaboration with Intel brings an alternative AV1 solution to the open-source community, enabling more rapid AV1 algorithm development and spurring innovation for next-generation video-compression technology,” explained David Ronca, Netflix director of encoding technologies.

Importantly, SVT-AV1 is being offered to the open source community and so will be available to AV1 codec developers like Harmonic, Bitmovin and Beamr.

“Compared to today’s most popular codec (H.264 AVC), SVT-AV1 can help service providers save up to half their bandwidth, delivering leading-edge user experiences that can be quickly and cost-effectively delivered at a global scale,” said Intel’s Lynn Comp, vp of the Network Platforms Group.

She pointed to nonlinear interactive programming like Netflix Bear Grylls’ You Vs. Wild as an example of the type of content that requires a higher level of processing.

AV1 faces patent protection challenge
While AV1 has been promoted as royalty free in deliberate contrast to HEVC, there have been those warning that the proof is in the pudding. HEVC has been mired in patent issues but there’s a similar fly in the ointment emerging for AV1.

This is the launch, strategically made prior to NAB, of a new video coding licensing platform targeting AV1 and VP9.

It’s being fronted by Luxembourg-based Sisvel International, a global IP protector and patent pool administrator whose online video pointedly questions whether AV1 and VP9 should be FREE?

The campaign on its website is run with the tagline: ‘Do the right thing. Reward innovation: play the right future’ and essentially argues that successive generations of compression standards have built on the intellectual property of those that have gone before. It suggests that there’s a direct line between some of the companies which held patents in VP8 and VP9 to AV1 and that therefore payment for this development should be expected by the industry.

It has marked out a patent pool for AV1 licenses applied to “consumer display devices” (smartphones, PCs, TVs) and “consumer non-display devices,” (e.g set-top-boxes).

The rate for these, states Sisvel, is €0.32 for display devices and €0.11 for non-display.

While it makes clear the licenses do not cover content encoded in the format, it will cover playback of that content on devices with embedded AV1 encoders.

As streaming video development specialist  Mux points out that could mean a bill for Apple of $29 million a year based on current sales of iOS gear.

Sisvel argues: “If those funding R&D activities are not fairly rewarded, there will be less incentive to innovate further. Securing funding to support R&D activities is fundamental to foster the innovation ecosystem.”

“The video codec technologies used today are the result of decades of investments in innovation by many parties,” it maintains. “These continued investments resulted in different generations of technologies, allowing for continued and drastic improvements in the video codecs domain.

“Revenues from patent licensing can be and regularly are re-invested in R&D activities. This creates a self-sustaining cycle in which the fruits of previous innovation can fund new research, generating an inventive loop in which the intangible assets acquire real economic value. Innovation is a long and expensive process, it requires the possibility of substantial returns to be worth pursuing.”

None of this cuts any ice with the Alliance which promptly rebuffed the approach.

In a terse statement released at NAB said, “AOMedia was founded to leave behind the very environment that the [Sisvel] announcement endorses – one whose high patent royalty requirements and licensing uncertainty limit the potential of free and open online video technology. By settling patent licensing terms up front with the royalty-free AOMedia Patent License 1.0, AOMedia is confident that AV1 overcomes these challenges to help usher in the next generation of video-oriented experiences.”

Members of the pools behind the Sisvel intervention reportedly include JVC, Philips and Toshiba along with operators Orange and NTT all of whom have also licensed patents to MPEG LA for either the AVC, DASH, or HEVC patent pools.

“The idea that these [multinational corporations with multi-billion dollar revenues] need this component of revenue feels unrealistic,” contends Mux streaming specialist Phil Cluff. “While we agree it’s important for intellectual property to be respected, this feels like potential profiteering on the behalf of the patent holders and Sisvel, especially considering the admirable project goals of AOM and the AV1 project in particular.

“With no transparency of the patents offered by this group, it’s hard to gauge the impact or legitimacy of Sisvel’s patent pool.”

Additionally, MPEG was at NAB explaining its decision to fast track Essential Video Coding (EVC), a new codec aimed to compete with HEVC on bit-rate efficiency and to be “licensing-friendly”.

As Harmonic’s vp of Video Strategy, Thierry Fautier has pointed out, EVC is not royalty free. “Its baseline profile might be but its compression performance is less than for HEVC.”

He suggests that if EVC achieves 24% bit-rate efficiency savings over HEVC this will be with royalty; “The devil is in the details.”

The Media Coding Industry Forum, which launched at IBC2018 and includes companies like Canon, MediaKind, Sony, Nokia, and Apple, will have its work cut out policing all of this to avoid another HEVC licence debacle.