BroadcastBridge
Now that AV1 has entered its final stage of development and is getting close to finalizing its features, it’s a perfect time to take a closer look at what’s in store for the future of video streaming. With Apple announcing their decision to join the Alliance for Open Media (AOM) in January, practically all major tech leaders are on board and AV1 looks to be in good shape for becoming a widespread standard in the near future.
But let’s not sign the death warrant for HEVC so hastily. HEVC is delivering on its promise of halving bitrates and quality pictures while AV1 is barely out of the lab. The gloves may be off at NAB so this article weighs the benefits and the negatives of HEVC and its ‘royalty free’ alternative before the bell rings for round 1.
In The Red Corner: HEVC
Since standardization in 2013 the codec has entered widespread tests and deployments across a whole spectrum of applications from satellite DTH, contribution, OTT and VOD to 4K UHD broadcasts where bandwidth efficiency is required alongside high picture quality.
HEVC/H.265 offers up to 50% bandwidth efficiency over H.264 for supporting twice the number of SD and HD channels over the same bandwidth without decreasing the quality of experience delivered to viewers. This frees up bandwidth for new services such as 4K UHD as well as expanded delivery options, such as OTT and LTE-enhanced mobile video and, in certain countries and regions, DVB-T2.
In addition to increased bandwidth efficiency, HEVC also supports picture quality improvement such as enhanced color gamut (BT.2020), higher bit depths (10-bit and beyond), high frame rates (50/60/100/120), and HDR.
Mike Callahan, senior director of solutions marketing for AWS Elemental points to another key benefit of HEVC: “its finesse in handling areas with less detail, resulting in a softer look that is similar to the smeared effects in painting when smudging pastels.”
AWS Elemental customers which have deployed HEVC include the BBC and BT in the UK; Al Kass and Alarab TV in the Middle East; NTV Plus in Russia; STN in Slovenia; Tata Sky in India; Kansai-TV, NBN, KTV, and K-Opticom in Japan; KT SK Broadband in Korea; and, Globo.com and Globosat in South America.
In contrast, AVC produces macroblocks which are much more noticeable to viewers. HEVC is further supported by the emerging MPEG-DASH adaptive-bit rate content packaging format that is designed to serve as a single, open streaming format for all devices and players.
Now let’s take a look at its drawbacks.
Higher efficiency generally is accompanied by increased complexity. As a result, the HEVC codec can require up to 10 times the compute power compared with AVC.
The fragmentation of the codec’s licensing system is described by AWS Elemental as “at best chaotic” with three licensing pools (MPEG LA, HEVC Advance, and Velos Media) and multiple undeclared licensing entities.
While MPEG LA only charges per decoding units capped at $25m a year, HEVC Advance caps its annual encoding unit charge at $40m but demands royalties for streaming, capped at $5m a year. This charge on distribution hits streamers like YouTube and Netflix in particular. Velos Media has not yet declared its royalty rate. Other HEVC IP owners (like Nokia and Technicolor) have not yet joined a pool or announced a royalty policy.
AWS Elemental points out that Apple and nearly every television manufacturer now support the codec and is optimistic that this licencing complexity “should get sorted out eventually.”
HEVC license holders could cut AV1 off at the pass by reducing the cost of their fees although they would be highly unlikely to eliminate costs altogether.
“The licensing for HEVC will need to change, as HEVC is on par on the performance side with AV1 today and AV1 is free," says Thierry Fautier, VP Solutions & Strategy at Harmonic. "However, if you add up the cost of encoding, decoding and device availability post-2020, it might not be enough to force HEVC licensors to switch to AV1."
In fact, the recent intense scrutiny on HEVC licencing has had a direct impact. HEVC Advance has decided to eliminate streaming content distribution royalty fees, on top of reducing certain royalty rates and caps.
It said it would remove "subscription" and "title-by-title" content distribution from the HEVC Advance Patent License, adding that HEVC Advance will no longer license nor seek royalty fees for non-physical HEVC content distribution including Internet streaming, cable, over-the-air broadcast, and satellite.
In The Blue Corner: AV1
Initial demonstrations of AV1 have shown improved quality and bitrates. Companies such as Netflix have expressed the need for AV1 to show a 20% efficiency improvement over HEVC “when measured across a diverse set of content” and would consider a 3X to 5X increase in computational complexity reasonable.
Bitmovin points to the codec’s capability in a number of areas. These include AV1’s film grain synthesis where the goal is to de-noise the initial content before encoding it and then add back the noise or grain effect before output during the decoding process. This way, the unnecessary information would not have to be transmitted at all and the overall load of data could be reduced substantially.
“The potential in bandwidth savings for content providers from using this technology is enormous,” says Christian Feldmann, Codec Engineer at encoder developer Bitmovin.
“More so for very “noisy” content, which can commonly occur in old video footage that has been digitized or in videos, which use film grain for artistic reasons. Either way, this tool can be used to great effect and forms a key benefit in AV1’s list of features.”
Filtering is an essential process in every video codec, as it drastically increases the perceived quality of the encoded video. It mostly occurs along the outlines of each of the blocks, which are used to divide each picture into smaller sub-units during the compression process. AV1 contains various sets of filters, most of which are derived from existing codecs.
AV1 is also said to mark the first time that non-planar motion compensation has been implemented into a video codec. Motion compensation algorithms have been used and theorized upon for a while, but only on a two-dimensional level. AV1 changes that.
“Due to the constant increase in processing power of consumer devices, this technique is now ready to see use in mass-market applications,” says Feldmann. “These techniques work extremely well for predicting large area movements, like background motion or camera movements. Additionally, they can handle consistent backgrounds and color schemes very effectively, which is one of the reasons why animated videos tend to deliver great encoding results, even with very high levels of compression.”
AV1 is also claimed to be better at handling 8K UHD. Because the increase in the size of individual coding units (block size) is considered a more effective way to scale the compression process along with high resolution content than using smaller blocks.
Perhaps AV1’s most important feature is not a technological one: It was designed from the very start to be completely royalty-free, in an effort to provide a truly open video codec capable of providing high quality video streaming at lower bitrates.
“With the availability of high resolution content constantly increasing and technologies like VR and 360° video on the rise, the need for a suitable, technologically advanced and open codec has become apparent among large-scale content providers. This desire is probably best documented by the fact that virtually all leading industry players and tech companies are contributing members of the AOM,” says Feldmann.
These include Apple, ARM, Cisco, Facebook, Google, IBM, Intel, Microsoft, Mozilla, Netflix and NVIDIA.
Amazon - also a founder AOM member – thinks it too early to speculate on AV1’s viability as a competitor to other codec technologies. A good analogy is the evolution of VP9. It is used by new internet companies but not by traditional broadcasters.
“Assessing customers’ disposition about a new codec and factoring in well-established support from the TV industry for HEVC, we can expect a similarly lengthy period of time to elapse before ratification,” says Callahan.
Then there’s the fact that AVC decoders are embedded nearly everywhere. Creating a product with an AV1 decoder will take about 18 months and it will take longer for support to be ubiquitous. It will happen eventually, but not on the day ratification of the standard is complete.
Other variables that could impact AV1 maturation: software development can take unexpected twists and turns; “open-source is an open invitation for the development of multiple codecs, each with different configurations and approaches to experimentation” says Callahan.
“Different codecs offer feature sets well suited for different use cases. For example, we could see legacy OTT broadcasters and pay TV operators continue to invest in HEVC while newer OTT entrants, such as video gaming and security and surveillance lean toward AV1. There will also certainly be patent challenges from third parties, much like there were for VP8 and VP9.”
AV1 maybe royalty free, but it is not “indemnification free” against patent claim violations.
While not impervious to next-generation codec advances, HEVC does have a well-established base at this point with an estimated 1 billion HEVC-enabled end points in the market. A recent survey conducted by Unisphere Research indicated that while there is interest in other codecs (e.g., AV1 and VP9), 58.3 per cent of respondents said they do not plan on adding any new codecs. In that same report, 25 per cent of online content distributors surveyed indicated that they are currently distributing HEVC encoded video. HEVC also has broad support from major OTT streaming services like Amazon, Apple, and Netflix and across the encoding vendor ecosystem.
For its part, AWS Elemental has always taken a codec-agnostic approach, “with the ability through software to quickly and flexibly onboard new compression techniques as customers indicate the need to do so,” says Callahan. “It is conceivable that many media companies and video-capable enterprises will take a multi-codec approach to support video infrastructures. For example, Amazon Prime Video and Netflix take a multi-codec support approach: both are AOM members; both are HEVC users; both distribute content in additional codecs like AVC depending on the device making the request; and, both are active in AV1.”
On The Under Card: Content-Aware Encoding
H.264, despite its limitations, continues to constitute the vast majority of video in broadcast and streaming. And it keeps improving through techniques of Content Aware Encoding (CAE). This is an encoding technique which uses machine learning to compare content against known parameters for a given device and/or media player type, can boost the picture quality and reduce the distribution cost and are already proving to prolong the life of MPEG4/H.264 AVC. However CAE applies largely to file-based transcoding for on-demand content.
Says Fautier, “CAE is used on existing codecs that rely upon VBR methods in ABR delivery. Harmonic’s EyeQ CAE solution is already commercially deployed for live applications. Netflix introduced this technology two years ago. CAE has the potential in AVC to save up to 50 percent bandwidth, but is content dependent and not as powerful as HEVC. Nevertheless, it is a very good replacement.”
AWS Elemental creates its own adaptive bitrate-ready encoder and, as such, content and file complexity awareness are built into the encoder itself for what it describes as “optimal viewing experiences”.
This, says Callahan, contrasts with the use of bitrate ladders or similar optimization approaches that are not built into encoders and that can falter when applied to larger numbers of files with issues such as video quality and noise fidelity limitations.
The Perseus dimension
According to Fabio Murra, SVP Product Marketing at V-Nova, the winner will be the codec that best solves a problem in a given application. This is going to be measured along the dimensions of: compression performance, computational performance and compatibility.
“Is AV1 going to be useful if it compresses 30% better than HEVC but takes 100x more?” asks Murra. “If you’re a large on-demand OTT provider, concerned with delivery costs more than datacentre occupancy, then it may make sense. If you’re a broadcaster worried about live delivery of 4K/UHD programmes, things are going to be different and HEVC or Perseus may be the codecs you evaluate.”
Perseus is of course V-Nova’s own codec.
“Given enough time and computational power all codecs produce good results. But, does everyone have infinite time and small nuclear power-stations feeding their datacenters?” says Murra.
Compatibility is even more important.
“AV1 is supported in software, in beta, only on Firefox Nightly and reliably only for SD resolutions,” he says. “HEVC is supported by the majority of high-end new STB chipsets, making it appealing for broadcast. In browsers things are not quite as rosy, with only Apple Safari supporting it in software from iOS 11 and with hardware where available. This gives it access to ~8% of browsers for streaming.
“Perseus, on the other hand, uses standard hardware blocks already available in modern chipsets making it compatible (in ‘software’) with all devices in the field. For example, in browsers it uses a combination of Javascript/WebGL decoding and is therefore compatible with almost 100% of deployed browsers.”
The future heavyweight
The industry must also develop an entirely new codec for massive data to handle applications from VR and multiple UHD streams to lightfield.
Streaming the entire 360 panorama of virtual reality and augmented reality content smoothly in high quality on a wide range of devices requires a huge amount of bandwidth. High-quality VR video requires 15Mbps and upwards of 25Mbps depending on a number of variables, such as Internet speed and content resolution (HD versus UHD).
“More often than not, the results are mediocre quality and viewing experiences. Just as important – if not more so - than new codec technologies in this regard are new methods of content production and distribution, such as viewport-adaptive encoding and streaming of 360 video for VR and VR tiling technology for transmitting complete UHD fields of view,” says Callahan.
Nonetheless, MPEG, ISO and ITU is exploring the need to include ‘omni-directional’ video coding technologies in a future video coding standard. It has based its work on HEVC. Expect news of its progress at NAB
.
.
No comments:
Post a Comment