All the Ways AR Is Impacting Content Creation

NAB

AR has moved out of the hype cycle and has now reached the mainstream. Here are some of the ways it happened in 2021 and ways it will evolve in 2022.

https://amplify.nabshow.com/articles/all-the-ways-ar-is-impacting-content-creation/

It is expected that 110 million people in the U.S. alone will be using some form of AR at least once per month by 2023, according to eMarketer. Brands and marketers are already making AR an essential tool. This is validated in AR Insider’s mobile AR ad placement projections which expect it to grow from $1.36 billion in 2020 to $6.68 billion in 2025. 

To get a sense as to how active AR for marketing was in 2021, head on over to analyst Tom Emrich’s newsletter Tom Emrich's AR Roundup & More | LinkedIn. Or check out 8th Wall’s Discovery Hub, a growing catalogue of commercial WebAR experiences for brands across industry verticals which already has over 200 featured projects since its launch in November. For example, Prada used gesture-recognition capabilities to let shoppers use their hands to swipe through virtual purse options they are shown wearing.

“Fueled by the pandemic, QR codes have become a necessity in our everyday lives whether this is replacing paper menus to access them in a touchless manner or to show proof of our vaccination status to enter a venue,” Emrich notes in an extensive AR fact check.

In turn, the mainstream has become acclimated to scanning these codes with their smartphones so much so that 45% of US shoppers have said that they have used a marketing-related QR code and 59% believe that QR codes would be a permanent part of using their mobile phone in the future according to a June 2021 survey from Statista. 

AR has also become a core component in entertainment launches especially for the film and television industry. Some notable examples in 2021 listed by Emrich included AR experiences used to market the release of Dune, Candyman, Space Jam, Eternals, Shang-Chi and the Legend of the Ten Rings, and more.

Snap announced major partnerships with both Sony Music and Universal Music to add to its song library and create AR experiences.

The tech is being used deeper than marketing. Fox's new singing competition, Alter Ego, gave avatars a primetime spot as real life human singers put on motion-capture suits to perform on stage as a virtual human. Seeing this show on TV, in addition to the other examples noted below, boosted Emrich’s confidence in a bold prediction that Gen Alpha could be the last generation to watch TV without augmentation.

While live sports broadcast is no stranger to the use of broadcast AR, the use of it went next-level in 2021. Examples include a giant virtual panther leaping around the Bank of America Stadium at a Carolina Panthers game. The panther was only visible to fans on screen, both in the stadium on the Jumbotrons and video captured and shared on air and on social media. The spectacle was produced by The Famous Group who were also responsible for a flying giant raven at a Baltimore Ravens game and who had also produced a mixed-reality segment for a previous Super Bowl.   

Emrich points to the companion AR experiences that enhanced the on-screen show by giving users a different perspective on what they are watching and allowing them to go deeper into the story.

For instance, Apple launched a mobile AR experience for the Apple TV+ show "For All Mankind" and Red Bull TV teamed up with startup eyecandylab and Google TV to invite viewers to join the expedition of "The Last Ascent” with a Emmy nominated AR rendition of the documentary.

In terms of hardware, the number of new smart glasses is set to accelerate with the latest models more wearable and fashionable than ever.

Meta’s latest glasses ‘Ray-Ban Stories’ are built with EssilorLuxottica and being marketed to consumers as a hands-free way to create and share memories.

“While there is no display nor are there the necessary sensors to facilitate AR, this device is playing a significant role in our eventual wearable AR future in that it is acclimating consumers to wearing tech on their face, including a camera which is creating a necessary starting point for next generation AR-enabled options to evolve from,” says Emrich.

Xioami, Huawei, TCL and Oppo all unveiled smartglasses in 2021. All were heads-up displays aimed at bringing information up to eye level. 

According to Emrich, the closest thing we got in 2021 to real consumer AR glasses came from Nreal and Snap. The Nreal Light is a mixed reality headset now sold in Japan, Germany, Korea and the US and a scaled-down and cheaper version, Nreal Air, are essentially glasses that extend the smartphone screen optimal for watching video or playing smartphone games.

Snap debuted its next-gen Spectacles offering AR capabilities with front-facing cameras and a dual waveguide display “which is a huge change from previous generations which were more like Ray-Ban Stories in that they were only able to capture photos and videos.”

Unlike Nreal who has its glasses for sale, Snap released its next generation Spectacles to select Creators who are using Snap's Lens Studio to create Lenses for the device.

Creators themselves are making greater use of AR, often powered by AI/ML. Snap Creators Brielle Garcia and Don Allen Stevenson III for example demonstrated how to use Len’s Studio 3D body tracking to transform yourself into real-time virtual characters that can move about your real world space as if you were one of the cartoon characters in Who Killed Roger Rabbit.

“Snap released two AI-enabled Lens in 2021 which instantly went viral. Snap turned us all into Disney-esque characters with its Cartoon 3D Lens and followed up with another AI-powered Lens just before the end of the year which turned us into our avatar twins.” 

Another example was the Candyman WebAR experience which integrated Google's Speech-to-Text software with 8th Wall to use voice to trigger the experience if users dared to say Candyman’s name five times. 

What’s coming down the track?

To take AR experiences to the next level Emrich points to the integration of spatial sensors onboard smart glasses to deliver greater depth information to AR experiences.

“There is still a need for more devices to support these sensors and continued efforts to provide more resources for developers, including greater access to APIs, in order for immersive experiences on the smartphone to move beyond the camera.”  

Set to launch this year, Magic Leap 2 features improved optics, a larger field of view and dimming – a feature that enables the headset to be used in brightly lit settings, in addition to a significantly smaller and lighter form factor. This is being aimed at the enterprise.

Meanwhile rumors abound that Apple is to roll out its AR headgear this year.

 AR is also reliant on the rollout of 5G. 2021 saw all three major networks in the US gain nationwide 5G coverage, although not evenly spread out across the country. 

Emrich says, “While 5G was definitely on the road to deliver on its promise of a faster, low-latency network in 2021, the nationwide coverage was still not all equal, and we are still waiting for the newer spectrums, such as the C-Band, to provide more consistent and performant connectivity. The good news is that there are major plans to tackle these challenges in 2022.”

Developers looking to create native AR apps now have a few new platforms to do so. Niantic officially released its Lightship platform for developer access. Qualcomm launched its Snapdragon Spaces XR Developer Platform. Apple also made some updates to its AR framework, RealityKit, including improvements to animation and a new Character Controller.

Developers looking to create WebAR got a huge upgrade with the launch of 8th Wall's all-new Reality Engine. While Zappar introduced its Universal AR for PlayCanvas and Babylon.js.

Developers looking to create social AR saw Snap and Spark AR platforms got powerful updates while TikTok launched its AR development platform, TikTok Effect Studio, in private beta.

Meanwhile, with more devices with LiDAR and new solutions that use the built-in smartphone camera, prosumers were scanning people, places and things to create 3D assets. Apple’s new Object Capture API which launched in 2021, “was a game changer”, according to Emrich, in that it enabled apps like Polycam and PhotoCatch to launch which enables users to create high quality 3D scans with their smartphone camera.

The rise of prosumer volumetric capture was given an additional signal boost in 2021 from various acquisitions including Sketchfab by Epic Games and Scaniverse by Niantic. We also saw Volograms launch Volu, an app that creates 3D volumetric video of people using the smartphone and deep learning, beginning to make the creation of holograms more accessible to everyone.