The Metaverse Is Already Blurring Cultural and Physical Lines

NAB

From virtual teleportation and 3D audio to advanced avatars and genuninfluencers, “a new digital era is on the horizon as the metaverse shifts from a sci-fi concept into reality,” says Emma Chiu, global director of Wunderman Thompson Intelligence.

article here

In “Future 100: Trends and Change to Watch in 2022,” the marketing agency charts a set of 100 emerging and future global trends, consumer change, and innovation patterns. Primarily intended to guide and pique the interest of advertisers, the report nonetheless synthesizes some of the core trends we’ve been reporting here at NAB Amplify.

These include the metaverse disrupting the physical world and augmented reality becoming the chosen medium for advertisers.

Let’s take a look at eight of them, drawing on the report compiled by WTI’s futurism, research and innovation unit.

Meet the Genuinfluencer

Virtual genuinfluencers — a term first coined by consumer trend forecaster WGSN — describes a CG-created influencer designed to be relatable rather than aspirational.

An example is Angie, a virtual influencer on Chinese site Douyin who has gained 300,000 followers by celebrating her “imperfections.”

Unlike other virtual influencers, whose skin has been smoothed to perfection and whose faces are perfectly symmetrical, Angie’s skin is sometimes dry or flushed, she gets acne and acne scars, her makeup creases and her teeth aren’t perfectly aligned,” says Chiu. “Instead of posing in designer clothes, she wears simple white T-shirts and athletic shorts.

 This chimes with data from Wunderman Thompson that 73% of American Gen Z’ers want a brand that understands them, and 76% want a brand that is accepting of a range of identities and experiences. “Being too aspirational is repellent now,” The Guardian reported in August 2021.

This is also filtering into the virtual realm, where brands have an opportunity to craft their avatar influencers around core values and relatability.

Virtual Venues Meet Physical Spaces

Lost Origin is a London-based test project designed to push the boundaries of what’s possible in the nascent world of immersive and interactive tech-led performance storytelling.

“Virtual reality experiences can be solipsistic. You can experience VR alone but you cannot share the experience with others,” explains Maciek Sznabel, project lead at Factory42. “The aim was to design an interactive experience that people can enjoy together and, to achieve that, we decided to mix theatrical performance with mixed reality technology.”

In another example, The Royal Shakespeare Company used Unreal Engine to present Dream, an interactive performance of live actors that brings the audience into the production for an almost game-like experience.

“Liminal spaces, which blend virtual and physical experiences, present brands with an opportunity to reinvent how physical experiences and bricks-and mortar spaces can look,” says Chiu.

As related example of this trend, in Manhattan, High Line Art and westside cultural institution The Shed collaborated to create The Looking Glass, an augmented reality installation of virtual sculptures located in the High Line park last summer.

Chiu adds, “Alongside the rise of fully virtual venues and spaces, extended reality is informing physical spaces, elevating events to limitless interactions and experiences.

Social Media-tainment

Plots that develop entirely on profiles, dramas that unfold in feeds, and fictional characters who chat with their followers. Is this the next big thing in entertainment?

FourFront is “revolutionizing TV through TikTok,” Fast Company reported. FourFront, which secured $1.5m in funding last October, is a new kind of content studio. It produces scripted narratives on social media that have the look and feel of regular posts, featuring fictional characters who have their own storylines, social media pages, and interact with followers organically. As of October 2021, FourFront had 1.9 million followers and 281 million views across its characters’ TikTok accounts.

“We’re blurring the line between reality and storytelling,” FourFront cofounder Ilan Benjamin told Fast Company.

This could point to a new kind of social media-based interactive entertainment format. “Not only have we been creating this universe of characters on TikTok, we’ve also been iterating with a new interactive format,” Benjamin said. “There’s an evolution in entertainment happening from motion pictures, where audiences can engage in what we call living pictures with characters who feel alive, who live in our world, who are on social media, and react in real time to audience engagements. That fourth wall is completely broken.”

Invisible Universe is another startup scripting entertainment specifically for social media. CEO Tricia Biggio, a former SVP of unscripted television at MGM, calls it “the Pixar of the internet.”

With the mission of creating what the company calls “the next 100-year animated franchise,” Invisible Universe develops original animated characters on social media in partnership with celebrities, influencers and brands. It has released characters in partnership with Jennifer Aniston and Serena Williams.

Chiu concludes, “Storytelling is evolving, with emerging formats that are tailored for social media feeds. As movie theaters struggle and TV viewership declines, the entertainment industry is rethinking how it reaches and serves audiences.”

Spatial Audio

Immersive listening ushers in the next generation of audio experiences.

Apple’s new third-generation AirPods and MacBook Pro laptops will be equipped for spatial audio. Startup company Spatial is creating immersive, interactive soundscapes for public spaces, including lobbies, retail stores, offices and even hospitals. Sony introduced two new home speaker systems in 2021 with 360-degree spatial sound mapping technology for an immersive listening experience.

Companies are “fundamentally rethinking the future of work in this hybrid environment,” Spatial CEO Calin Pacurariu told Fast Company. “And they see sound as a competitive advantage.”

Chiu writes, “Social media platforms such as Instagram have driven a hyper-focus on visual elements over the past decade. Now, especially as digital platforms mature and engagement evolves, focus is shifting to multisensory elements — audio, in particular — for a truly immersive experience.”

Next-Gen Avatars

Nvidia is preparing for a future where 3D avatars with conversational AI will operate in both the virtual and physical world. In a November 2021 demo, the company announced the Omniverse Avatar platform. “The dawn of intelligent virtual assistants has arrived,” said Jensen Huang, founder and CEO. “Omniverse Avatar combines Nvidia’s foundational graphics, simulation and AI technologies to make some of the most complex real-time applications ever created.”

Epic Games believes digital humans are the future. A year ago, Unreal Engine launched the MetaHuman Creator, which allows real-time creation of photorealistic avatars in minutes. The cloud-based app can replicate intricate details of a person’s features, from complexion and wrinkles to broken capillaries and scars.

In September 2021, California-based avatar-generating startup DNABlock raised $1.2 million in seed funding to make the metaverse more diverse and inclusive, Silicon Icarus reports. The company’s CEO, Anthony Kelani, said, “The metaverse needs to represent everyone. This needs to represent the world. And with avatars, specifically, you should be able to generate an avatar that looks like you or like someone of color.”

Next December we will see what advances James Cameron will make to his virtual playground in Avatar 2.

“In 2022, new-age avatars will not only seem hyper-realistic, but also reflect the world’s diversity,” says Chiu.

Virtual Teleportation

Transforming communication and collaboration is at the heart of Varjo Teleport VR. The headset, launched in October 2021 by Helsinki-based Varjo, uses a cloud platform to allow for “photorealistic virtual teleportation.”

Chiu thinks technologists are “opening up digital portals — making virtual teleportation a plausible reality.”

One of them is Mark Zuckerberg. “By 2030, the new generations of Oculus will allow users to teleport from one place to another without moving from their couch,” he told The Information’s 411 podcast.

Microsoft Mesh uses mixed reality (MR) to create interconnected worlds where the physical and digital come together. The idea is that people in different physical locations are able to collaborate and work in real time on the same project via holographic experiences across different devices.

“You can actually feel like you’re in the same place with someone sharing content or you can teleport from different mixed reality devices and be present with people even when you’re not physically together,” says Microsoft’s Alex Kipman.

London-based design practice Space Popular proposes an even more ambitious concept — a civic infrastructure that allows for virtual teleportation. The cofounders envisage a “threaded network of virtual textiles that our virtual selves pull aside to move between virtual environments.”

 

Pulling this together, Chiu says, “Distance is becoming less of an obstacle. As people spend more time working, socializing and collaborating online, technology is paving the way for teleportation into a new virtual dimension that offers more intimate, close-to-reality in-person interactions.”

Co-Creative Platforms

Creativity is increasingly informed and powered by technology, setting the stage for the next era of digital platforms and creative influence, believes WTI.

Digital tools have “activated an entirely new world” of creativity — one where “creations can transcend physical limitations,” creative technologist and digital designer Helena Dong tells the consultancy.

Seventy-two percent of gen Z and millennials in the US, UK and China believe that creativity today is dependent on technology, and 92% believe that technology opens up a whole new world of creation, according to WTI’s research conducted in July 2021.

“For generation alpha and generation Z, customization and creation are intricate parts of their gaming experience,” Keith Stuart, games editor at The Guardian, tells WTI. “For them, customization and the play element are part of the same thing — self-expression and exploration.”

IMVU describes itself as a next generation social network. It aims to facilitate emotional connections by providing shared experiences based on 3D avatars “that go beyond the traditional social media loop of posting a picture, liking, commenting, and sharing.”

The company’s CEO, Daren Tsui, tells WTI that “creativity is the new status symbol” — dethroning influence and income. When users come to its platform, “making money is not the most important thing for them. It’s about being recognized for their creations,” Tsui explains.

Chiu extrapolates this to mean that in the digital world of the near future “people are not passive consumers, but creative agents crafting their self-expression and curating their virtual identity.”

Branded Virtual Worlds

Last September on Roblox, Hyundai launched its Hyundai Mobility Adventure. The space contains five “parks” where users can race, play games, learn about Hyundai’s technologies, and take part in festivals. Other brands are creating branded virtual worlds on their own platforms.

Procter & Gamble, for example, invited visitors into a virtual world where they could learn about the company’s full portfolio of products and play games; BMW launched its virtual world Joytopia to present its vision for the future of mobility alongside “festivalesque” elements including an exclusive Coldplay concert. Japanese beauty brand SK-II launched a virtual city where visitors can navigate a virtual rendering of SK-II City to shop, learn about products, and even catch a movie in the SK-II cinema.

“There’s something unique about the growth of gaming as a marketing platform,” said Grant Paterson, head of gaming and esports at Wunderman Thompson. “We talk about gaming as being the nexus of a new consumer paradigm.”

For younger generations especially, gaming is replacing advertising channels such as print and television. “A lot of the traditional ways of marketing to young people are gone forever,” says Stuart. “Gaming is where they are.”

With the in-game advertising market set to grow by $3.54 billion between 2021 and 2025, per Technavio, brands are diving headfirst into gaming with branded virtual worlds. Expect to see more branded virtual worlds as companies and marketers tap into this growing space.

 

The Future of Work: Teleportation, Holograms and Roaming Avatars

NAB

Virtual offices are taking over as the shift to remote and hybrid work is likely here to stay. Micropreneurs 2.0, crypto-artisans and nomadic employees are redefining what it means to work in the 21st Century.

article here

In a new report, “Future 100: Trends and Change to Watch in 2022,” marketing agency Wunderman Thompson, via its research unit WTI, charts the future of work.

Let’s take a look.

Metawork

Microsoft will launch Mesh for Microsoft Teams this year, enabling mixed reality for users, with the option to attend meetings as customized avatars and collaborate on projects through shared holographic experiences. Mesh also gives companies the option to build immersive virtual environments. Microsoft describes the feature as “a gateway to the metaverse.”

Meta has its version called Horizon Workrooms. By connecting through VR co-workers can collaborate and create together in the same virtual workspace. “Meet teammates across the table, even if you’re across the world — and transform your home office into your favorite remote meeting room,” Horizon Workrooms promises.

Building branded virtual workspaces are gaining popularity. Kumospace and gen-Z startup Branch focus on heightening “organic interactions” using virtual rooms that mimic physical environments, where users’ avatars can move around and interact.

“The way we connect, live and work is transforming thanks to the metaverse,” says Emma Chiu, global director of WTI. “The future of work will foster a hybrid setup for many, leading with virtual workspaces that generate new forms of creativity, collaboration and immersion.”

Crypto-Artisans

The Web 3.0 economy is providing an alternative to corporate employment in the form of decentralized autonomous organizations (DAOs). This is a digitally native community or organization that could potentially represent the future of work.

“Your next employer could be a DAO,” imagines Chiu. “Strictly speaking, a DAO is a community-led digital organization that runs on blockchain technology. It is managed not by a CEO or board of directors but by a smart contract (lines of code that define its operations). In practice, many DAOs are not yet fully autonomous and so the term is also used colloquially to refer to digital organizations in general. Crucially, DAOs also have built-in treasuries linked to cryptocurrency, which means members can earn tokens in return for their contributions.”

A new breed of crypto worker is already being drawn to DAOs as an alternative to the corporate nine-to-five. WTI says participation in the DAO economy growing fast. According to DeepDAO, a platform that tracks the industry, there were 1.6 million members and token owners in December 2021.

From an employee’s point of view, DAOs have a lot to offer, WTI observes. DAOs are digital, so there are no physical HQs or geographic boundaries—both already outdated notions for many Gen Z’ers. DAOs have little to no hierarchy, so there are no bosses either. Instead, they offer members the ability to collectively influence decisions and share in profits. Above all they offer a readymade community of likeminded, passionate, and highly invested people all pursuing the same goal.

 “While not every company will want to copy the DAO model, nor will it be universally appropriate, there are interesting lessons to learn from this new model in terms of community, culture building and worker empowerment,” WTI notes.

In an era of employee activism, many want to shape the place they work to fit their values, as Julia Rosenberg, cofounder and CEO at Orca Protocol, recently explained at the Mainnet conference (quoted by WTI): “If you disagree, you have the opportunity to effect change, which is not something that exists in centralized organizations,” Chiu says.

Micropreneurs 2.0

In the initial months of the pandemic in 2020, many workers quit their desk jobs to pursue passion projects full-time. Now the trend has snowballed into a nationwide reassessment of work that is precipitating the next era of employment.

British workers are going freelance in droves. April 2021 data from freelancer platform PeoplePerHour found that almost one in five freelancers had become self-employed as a side hustle alongside an employee position, and nearly two-fifths of those began freelancing in the past 12 months.

According to Microsoft’s global Work Trend Index, published in March 2021, 41% of people around the world were likely to consider leaving their jobs within the next year, rising to 54% of Gen Z. The Washington Post dubbed the shift the “great reassessment of work.” So where are workers going? Many are leaving for higher-paying jobs, whether that’s retail and service workers taking entry-level positions or mid-career professionals switching jobs.

As of December 2020, resignations among managers were 12% higher than the previous year, according to workforce analytics company Visier. Others are pursuing passions or side hustles full-time. Microsoft’s research revealed that 46% of people were planning to make a major career pivot or transition.

“Employees are scrutinizing what they want from a career and a workplace, potentially bringing about the end of the workplace as we know it,” suggests WTI.

The Great Reskill

A related trend happening across business as the nature of work changes, so does the need for new employee skillsets.

“As the digital revolution races on, brands are realizing that the fastest and most effective way to keep pace is to upskill their existing workforce,” says Chiu.

According to one educator, employees will be expected to have to learn some sort of new skill every four years.

It’s already happening. Levi Strauss & Co announced a new company-wide Machine Learning Bootcamp. This was an eight-week, full-time, paid program that offers employees training in digital skills such as coding and machine learning. Graduates of the bootcamp will either return to their current job with new skills or will join the company’s strategy and AI team.

Verizon invested more than $200 million in employee learning and development programs in 2020, offering training in topics like data science, 5G and AI. By the end of 2021, Verizon said it had equipped 100,000 employees with digital skills “to ensure team members are ready to keep pace with ever-changing demands of building the future.”

 

The Nomad Economy

What’s more, the new influx of workers won’t be expected to be in the office. At least not all of the time or not in our received notions of what an office constitutes.

Pandemic-induced lockdowns have broken traditional working habits. “With this came a surge of remote workers, some of whom became digital nomads for the first time, choosing to work in new cities, states, or even new countries,” says Chiu. “The future of work is borderless. As ingenious companies step up to iron out the complications that accompany a transient lifestyle, digital nomads will continue to grow in numbers and financial clout. A new economy is emerging that caters to digital nomads’ administrative, financial and logistical needs.”

Metaverse Recruits

“Is it time to hire a chief metaverse officer?” Vogue Business mused in an October 2021 article that noted the rise in augmented retail, virtual venues and digital possessions.

From virtual material designers to creatives across the board, companies are hiring for a metaverse workforce. The metaverse virtually recreates pretty much all aspects of life, and recruitment is no different.

“In the coming years, companies of all kinds are increasingly going to be re-orienting towards a hybrid model of virtual and physical work, production, commerce and communications,” Grant Paterson, head of gaming and esports at Wunderman Thompson, explains. “The dual forces of converging technologies and emerging consumer behaviors in virtual places makes the cultivation of a ‘virtual pillar’ critical for many, if not all, businesses.”

The report notes that Nike filed seven trademark applications in October 2021; they include use of “downloadable virtual goods” and “retail store services featuring virtual goods.” In the same month, the company started recruiting virtual material designers to sit within its digital product creation team. The job spec describes the role as helping to “build the future of Nike Footwear materials” and ignite “the digital and virtual revolution at Nike.”

Last November, British broadcaster ITV posted a position for a metaverse creative within its new Metavision initiative, which “looks to combine the worlds of gaming, entertainment and advertising.” The role involves creating metaverse brand activations from concept stage to execution.

According to Chiu, “Partnering with external tech and games companies has been the trend of the past year; now brands are seeking out talent of their own to accelerate the technological offerings of the metaverse. Just as when social media became mainstream and companies scrambled to hire social media leads, so recruiting talent to help build offerings around the metaverse will be reminiscent of that hurry to hire, and maybe even more urgent.”

 

Accessing the metaverse

IEC

At CES2022 companies were juggling to present solutions, from smart glasses to contact lenses, which enable people to step into this new parallel world called the metaverse.

article here

The metaverse was the buzzword on everybody’s lips at CES in Las Vegas this year even if the term itself is open to interpretation. Most people understand that it points to the evolution of the internet as a digital world that closely mirrors our own. An experience which is enhanced by being three dimensional and persistent across applications and devices.

"We're starting to talk about the metaverse now in the same way we talked about the internet in the early 1990’s,” said Steve Koenig, VP for Research at the Consumer Technology Association (CTA), the organization that runs CES, during a press briefing.  “When we were all using 56k modems and dial-up internet, it was impossible to imagine the things we'd be doing online today. I think the same is true with the metaverse." Koenig pointed to the building blocks which will be used to assemble “a metaversian immersive digital experience”. These include cloud computing, haptics, volumetric video and 5G networks.

In most of these areas, IEC is laying the groundwork with the appropriate standards. IEC Technical Committee 100 has recently published a new document pertaining to haptic technology. (https://etech.iec.ch/issue/2022-01/how-haptic-technology-is-changing-the-multimedia-experience). The joint technical committee between IEC and ISO, ISO/IEC JTC 1, which prepares standards for IT applications, leads the way on standards for cloud technology and volumetric video.

New comfortable glasses

The perspective of accessing the metaverse has given the makers of virtual, augmented and mixed reality (VR/AR/MR) displays fresh impetus. A new range of smart glasses is about to hit the market, promising a more comfortable experience for wearers than earlier devices such as Oculus Rift, Google Glass and Magic Leap One - all discontinued but with new versions in the works.

“Wearing bulky headsets on your face is not going to be the future of the metaverse,” says Michael Hayes, CEO of a leading Californian company in the field of bio-microelectronic mechanical systems (MEMS) and ophthalmic technologies. “Glasses with headsets are uncomfortable, and you can’t deploy them,” agrees Paul Travers, founder and CEO of an optical technology company which specializes in making AR solutions for the workplace and industry. Jaehyeok Kim, CEO of a Korean startup believes that smart glasses are the only device that can combine the metaverse and the real world and will eventually “change the fundamentals of gaming, of business and of our daily lives.”

Most smart glasses work using optical waveguides. These are structures imprinted on a piece of glass (either on one or both lenses) that bends light to the eye from a source (usually a mini projector) embedded in the corner of the frame. The source overlays virtual objects on the wearer’s vision with data typically derived from a smartphone. Recent developments have tended towards enabling wireless connectivity with the smartphone and reducing the size and weight of the headgear.

Only a few manufacturers can produce plastic AR optics, one of which is T-Glasses, which were shown as a prototype at CES. “Plastic fabrication of smart glasses using injection-moulds is very cost-effective and scalable to mass production,” explains Kim. The company has also developed a patented concept called PinMR that enables manufacturers to pair waveguides with OLED micro displays. Kim claims this technique consumes less power and has a smaller form factor than other light sources.

IEC is developing standards for AR and VR as well as smart glasses displays. ISO/IEC JTC 1 /Subcommittee 24 works on interfaces for information technology-based applications relating to computer graphics and virtual reality, image processing, environmental data representation, support for mixed and augmented reality, and interaction with, and visual presentation of, information.

IEC TC 110 prepares standards for electronic displays, including OLED, 3D, holographic as well as flexible screens. It publishes IEC 62341-2-1 on OLED displays for instance, which specifies the essential ratings and characteristics of OLED display modules. It has also issued IEC 62629-41-1, a technical report on 3D and holographic display devices.

Smart glasses for manufacturing and industry

Analysts such as ABI Research expect the consumer market for smart glasses to grow 100% year on year between now and 2026, (when 28 million smart glasses will ship), but it is applications in the corporate and industrial sphere which have led growth to date.

Travers’ corporate solutions stem from the Tac-Eye monocular weapons sighting system his company developed for the US special forces in 1997. Its M400C glasses are being used today to train surgeons in medical procedures and by shift workers for more efficient picking of parts in warehouses. Shield, the firm’s latest development launched at CES, features some breakthrough technology. Travers explains: “Most activities for which you’d wear AR glasses are conducted at arms’ length, but normal waveguides are set at infinity. It means that when you put a 3D image in front of someone’s eyes to appear 2ft (0,6 meters) away there is a focus problem that causes stress in a lot of people. We’ve designed our waveguides to change the focal distance to converge about 1,5 meters away, making the experience much more comfortable.”

The company has coupled this with another proprietary technology capable of emitting 2 million nits of light from MicroLEDs the size of a grain of rice. Nits are units that measure brightness in terms of area coverage. “MicroLEDs, shrunk to the micron level and packaged into a projector smaller than a pencil eraser, means you can put the technology in a pair of glasses and it practically disappears from view,” Travers describes.

Contact lenses are the next step

With technology included in smart glasses already at the nano-level, the next logical evolution is to eliminate glasses all together. Connected contact lenses point one way forward.

Hayes’ company holds patents for embedding component circuitry into soft hydrogel (as opposed to hard gas permeable) contact lenses and has tested them with a leading lens maker. He was at CES on the lookout for developers to licence the “smart biology.” “Our technology has the dual capability to tune a wearer’s eyesight by enabling bifocal short and long-distance focus as well as to augment vision with graphical information just like a heads-up display,” he says. “Contact lenses will be much more discrete and comfortable than having to wear smart glasses.”

The lenses are powered by a micro battery, which connects to the user’s phone via Bluetooth and has a patented method of harvesting energy from the blinking of the wearer’s eyes. Information displayed could be health related (such as monitoring blood sugar levels) or functions like those available on a smart watch, for example a gyroscope. Hayes’ also points to real world alerts such as speed limits and restaurant discounts and, eventually, full immersion. 

Brain-machine interface

The augmented reality and virtual reality software and the hardware with which we will access it remain in the early stages of development. “Down the road you won’t need a phone just a pocket device that is connected,” says Travers. “The tech industry will take all that high compute needed in AR glasses right now and put that in a device which is in your pocket or on your wrist.”

Longer term we may not need a conventional wearable at all. A microchip embedded into our nervous system could enable humans to communicate with the internet by thought alone. An Elon Musk-owned company is taking the brain-machine interface out of science fiction. It is designing a neural implant that will let individuals control a computer or mobile device “unmediated and in high fidelity” according to the company website.  The idea is to insert micron-scale electrodes into areas of the brain that control movement. It is about to launch clinical trials with humans.

 

The Developer’s Role in Building the Creator Economy is More Important Than You Think

NAB

article here

It’s hard to earn a living solely from platforms like Spotify, Twitch and YouTube. An artist on Spotify would need to get about 300,000 streams per month to earn the federal minimum wage. According to SignalFire, 97.5% of YouTubers’ income falls below the US poverty line.

But this problem isn’t the fault of these platforms, or at least one they are not going to fix.

It’s a creator economy problem.

So says Bremner Morris, CEO of Rally, which, lo and behold, is a platform enticing content creators to launch their own digital currency and build a sustainable business with their fan communities.

Outlining his argument in Rolling Stone, Morris even takes aim at direct-to-fan monetization models run by Patreon and Substack. These platforms may enable creators to retain around 90% of revenues generated from subscriptions, “But even these platforms have only produced a handful of millionaires, while most creators fail to meet the minimum wage.”

Instead, he says “Creators should work directly with their communities to drive independent forms of monetization that insulate them from the king-making platforms.”

Naturally, Morris is promoting Rally as the solution, but let’s hear him out. He suggests one way of helping reset the balance toward the creator is to connect artists directly with developers.

“Big tech platforms typically only build features when it benefits them and their shareholders. Unfortunately, that means most of the technical talent that could be building tools for creators is focused on platforms that reap most of the financial gain from creators’ content,” he explains.

“However, there’s a significant movement happening quietly where artists and creators are starting to tap into the tech-savviness of their communities to build new kinds of tools that allow them to create unique fan experiences and monetize in new ways.”

NFTs are one example. Morris calls the tech “an exciting new way for creators and artists to monetize directly instead of relying on ad-based platforms.” However, he also notes that many creators are frustrated that the technology’s success has mostly been driven by speculation and not from genuine fan experiences.

“This is where I believe developers can come in. Twitch was actually one of the places where creators first used developers to build new ways to interact with their fans. Streamers like Swiftor and CohhCarnage have worked with devs to build sophisticated monetization and interaction systems, going beyond the base feature set of Twitch. Today, this trend is becoming more prevalent across different verticals.”

He agrees with Li Jin, founder of Atelier Ventures, and perhaps the leading strategist of the creator economy, that empowering creators to engage with and capitalize on their superfans is a key to unlocking mobility to the middle class of creators.

“If creators get more access to developers, they will be able to build the internet that they and their fans want, instead of the one that mega-platforms give them,” Morris writes.

 

Behind the Scenes of Guillermo del Toro’s Nightmare Alley

IBC

Director of photography Dan Laustsen explains the thinking behind the seductive visuals of the neo-noir feature.

article here 

Twenty-five years ago, Guillermo del Toro and Danish cinematographer Dan Laustsen ASC DFF teamed up for the director’s first American feature, Mimic. Since then, the two have collaborated on Crimson Peak, Oscar winning best picture The Shape of Water (2017) and now the noir crime drama Nightmare Alley.

Set in the 1940s, the film moves between a seedy traveling carnival and the city of Buffalo on the US border with Canada, where Bradley Cooper’s grifter Stanton Carlisle finds himself locked into the orbit of Cate Blanchett’s psychiatrist Lilith Ritter.

Del Toro was less attracted to the noir trappings of the 1947 film of the same name than the original 1946 novel by William Gresham which contains aspects of psychosexual, mystery and magic. His screenplay (with Kim Morgan) also draws on Gresham himself whom the director has described as a truth seeker “like The Fool, the Tarot card.” The author committed suicide in 1962.

Despite having the mechanics of the noir genre including an icy femme fatale and characters doomed to their fate, Nightmare Alley is conceived as having a “dreamlike quality accomplished through the frame of a film noir,” Del Toro told Simon Mayo on Radio 5’s film programme.

Laustsen tells IBC365 that discussions with Del Toro started from mood boards for a story that is awash with moral decay.

“We never talked about noir explicitly, but it is staged in a very classical way like a 1940s movie,” he continues. “The first time you see Cate she has to be this very powerful and mysterious and very beautiful figure which to me meant using direct single source lighting. You can say it’s noir but it’s more a classic Hollywood style. I wanted to light Cate like a film diva.”

This lighting design required the camera and the actors to meet their marks very precisely, down to just 2cm, according to Laustsen.

“We have this single source keylight that was moving together with the camera. If Cate is a little too close to the light we won’t have the shadow on her face but too far away and she’ll be too dark. The light needs to hit her face precisely and not hit the background.

“The camera is moving a little bit and the actors are moving a lot so it was like a ballet dance between lights, camera and actors. I’ve never done this before. It’s a very technical but powerful way to work.”

There is a deliberate “artificiality” to the lighting in the sense “you don’t know exactly where it is coming from” to support the unsettling sense of mystery. Similarly, the filmmakers altered the dimensions of Lilith’s office and designed sets with corridors and ceilings to hem the characters in.

“Her office was much smaller in the beginning but we decided to build it much larger and to have the camera shooting wide from a low angle so we can see the ceiling. We lowered ceilings too to enhance this feeling of our characters being trapped in a box, hiding in the shadows, from which they couldn’t escape.”

Early discussion about framing in the boxy 4:3 ‘Academy’ ratio was dropped in favour of widescreen, a scale emphasised by shooting large format on the Arri Alexa 65. This came into its own in the more expansive opening landscapes of the carnival for which the production built massive sets and Lausten populated with 300ft cranes for high wide angles, keylights and smoke machines.

“We use smoke and rain and steam a lot in this movie in the background because as much as set design or colour these atmospheres create a third dimension to the image,” says Laustsen, who used similar effects on John Wick 3. “Here, we use a lot of steam, particularly when we first see the carnival. Steam disappears quickly and it tells you that this story is not quite real, that everything is a little insecure. It’s not quite horror but it contributes to suspense.”

In contrast to the scenes in Buffalo, the lighting here is a little warmer and softer and less direct on the actor’s faces. The colour palette was built from a background of “steel blue” which Lausten says he and Del Toro first employed on Mimic.

“We like that colour but the thing about steel blue is that you have to be pretty even with exposure otherwise the colour jumps around.”

Laustsen introduced production designer Tamara Deverell to Danish portrait painter Vilhelm Hammershoi as pointers for the composition of interiors while Deverell drew from colours inspired by Matisse for the carnival.

Like 2021 Oscar-winning film Parasite (and the ‘Justice Is Gray’ version of Zack Snyder’s Justice League), Nightmare Alley has also been released in black and white. Arguably this version is more in keeping with the noir look.

“It’s difficult to shoot for both colour and black and white but we shot as a colour movie and thought about how we could make the contrast work better for the black and white,” Lausten explains.

“I shot with a diffusion filter inside the camera which helped a lot with the black and white version because you can have the highlights burning a little a bit more.”

He went back to the raw files and “art-directed” the black levels, grey midtones, contrast and highlights. Laustsen believes audiences will experience this version as a more “brutal” telling of the story.

Had Del Toro managed to make the film two decades ago as he had wanted to (but couldn’t land the rights) it would have looked a lot different.

“This movie required a certain sobriety in the use of directing tools that only comes with age,” he told Radio 5. “I’ve [since] shed some of the whimsy and pageantry to balance the darkness of the tale with the exquisite beauty of the world creation.”

Laustsen, who has been with Del Toro on that journey, picks out one scene as an example of the filmmaker’s more formal (if not quite yet austere) storytelling style.

“We have a scene with Bradley Cooper in a hotel room and use low angle wide shots framing him against the ceiling. He is trapped in this room, alone in the blackness. When we came to shoot this we could have played with light on his face as we do elsewhere. His character is very dark, and we do use deep black shadows with one side of his face in the light but always tending towards the darkness. But in this scene we chose to light him totally in silhouette. You don’t need to see everything. You can feel what is going on. To me, that’s a great example of how you can use camera and lighting to tell the story.”

What’s Really Real? Where Our Current “Timeline” Is Taking Us

 NAB

What we perceive as the nature of truth might need updating.

Article here 

“Is what you see with your [smart] glasses off more real than what you see with them on? If your focus clears, is the image before you fake?” KC Cole, senior correspondent at Wired, asks.

“So much of what’s real is indiscernible,” she writes. “Billions of neutrinos rain right through me as I sip my tea. Are they less real than the tea? Is an NFT less real than a dollar? Is your virtual meeting less real than the vivid memory you have of that meeting that never happened?”

This is classic philosophy. If no one heard the tree fall, did it in fact make a sound? Is Schrödinger’s cat only alive when we see it? Am I seeing the same color as you?

Cole says this queasiness took hold for her when watching The Matrix movies: “Reality is whatever we make it. It’s whatever we collectively agree upon, whether it’s bitcoin, atoms, or the devil…. If you’re not sure the bullet is real, would you stand in its path to find out?”

We can chalk Cole up as a member of the dystopia clan, those who warn about the dangers of ceding all of our agency over to The Man — or in this case, The Matrix.

Jaron Lanier, that sage of the Internet age, wrote in his 2011 treatise You Are Not a Gadget that increasingly, we are. To be specific, we are becoming “computer peripherals attached to the great computing clouds.”

Cole also quotes Microsoft principal researcher Kate Crawford describing the perils of cramming our complex and fluid personal and social realities into “representations of the world made solely for machines.”

AI forces the “systemization of the unsystematizeable,” wrote Crawford. AI “reduces depth, kills grace notes, flattens experience and us along with it,” echoes Cole.

The Matrix is full of analogies for our current existential reality crisis. One of them is that The Matrix itself gets its fuel from human batteries.

“The giant networks of machines we ironically refer to as ‘clouds’ also feed on humans: those who mine rare minerals, assemble devices, drive trucks, load packages, translate text, label and evaluate objects and faces,” says Cole.

Attempting to take in this underground reality is as jarring as Neo’s first look at the vast array of baby-powered fuel cells.

“No one wants to hear about the costs: the enormous carbon footprint of computing, the draining of community water and power supplies, the reliance on taxpayer-funded infrastructure, sewers, gas lines, fiber optics, you name it. There’s a reason these megaservers are hidden away in remote locales.”

There’s another analogy for the brave new world of humans being sucked and suckled by the machine: the way in which our data is used to power the economic monopoly of a few.

---

The humongous stores of data scraped off the internet — our faces, habits, health, finances, kids, lovers, favorite actors, vacations, conversations with your Roomba — are funneled into mega computers that tweak what you see in order to keep you hooked, sell you stuff. It’s a one-way street. We’re transparent to the megaservers, but they’re opaque to us.

Distant corporations use the data to change our lives “in unfathomable ways,” Lanier writes. “You never really know what might have been if someone else’s cloud algorithm had come to a different conclusion about your potential as loan taker, a date, or an employee.”

Let’s take it for read that we know all of this. Does that alter our reality at all or are we already so far down the rabbit hole as to be passive receptors of what the machine feeds us?

It’s not as if reality is all that it’s cracked up to be anyway. Cole says, “The human mind is a mess. The reality evolution left us is an accidental collection of spare parts of make-dos. Consciousness is chattery, convoluted, slippery, spotty, fickle.

The impending era of the quantum computer (wait a couple decades) is likely to make artificial reality feel even more real. Quantum computers might be able to handle all the complexity of the human condition.

“The cat can be alive and dead in a quantum computer,” Cole observes.

The alternative is to fight back. To build Web3 or the spatial internet or the metaverse in our own ideal image. To build back better.

In the meantime, Cole jests that she’s tempted to start selling T-shirts that might tap into a new kind of identity politics: This device is analog! Or maybe: Analog (and only analog) Lives!”

 

UHD, immersive audio, 5G and virtualised OBs: all the broadcasting innovation at the Beijing 2022 Winter Games

IBC

Beijing 2022 will offer an exciting glimpse into the immersive and virtualised future of Olympic broadcasting, as revealed by tech leaders at host broadcaster OBS.

article here 

The Winter Olympics features a UHD HDR production and immersive 5.1.4 audio; live virtual reality; 70+ hours of 8K and an experimental virtualised OB van as part of a record 6,000+ hours of content produced for the event.

For all that, it is the use of 5G in Beijing which may prove most game-changing for wider live production.

Host broadcaster OBS is promising, as it did with the Tokyo Games, the most immersive televised Winter Olympics yet. Many of the same tech advances deployed in Japan pop up again.

Here’s everything you need to know.

UHD

Tokyo 2020 set a new benchmark for an Olympic broadcast in being a full UHD HDR production. This is being doubled down in Beijing with a single UHD HDR workflow from which HD SDR will be derived. OBS claim this to be a ‘full native UHD’ set up but admits that – as in Tokyo – it will also rely on several specialty cameras “that at this time can only operate in HD 1080p SDR.”

In fact, the number of HD contribution feeds (41) it is creating outweighs that of UHD (31) with presumably HD to UHD conversion of numerous cameras taking place to get to a master UHD format. It will prepare 43 HD feeds for international distribution compared to 36 UHD.

The HDR to SDR conversion includes a set of look-up tables composed in-house to enable better interoperability between the two co-existing HD SDR and UHD HDR workflows. The UHD production will adhere to SMPTE 2036-1 at 50 Hz with HDR in Hybrid-Log Gamma (HLG).

“Although our production workflow is greatly simplified by having a single workflow model, achieving consistency across all HDR and SDR sources remains quite a complex undertaking on a job of this scale,” says Isidoro Moreno OBS’ head of engineering.

Immersive audio

This will be first Winter Olympics captured through a 5.1.4 audio configuration, ideally giving viewers a more realistic audio experience. To the 5.1 surround sound mix is added an overhead sound layer, from four hanging ceiling microphones with adjustable heights. In total, OBS will be using more than 1,600 mics (40 different models). Two audio QC rooms installed inside the IBC (international broadcast centre) will guarantee quality consistency across all sports. The 5.1.4 audio configuration will be provided for both HD and UHD.

“In Tokyo, this immersive audio set-up helped mitigate the absence of spectators,” says OBS CEO Yiannis Exarchos. “However, in Beijing, there will be limited spectators in the venues and their presence will certainly enhance our coverage.”

IBC infrastructure

The International Broadcast Centre (IBC) acts as the nerve centre for all broadcast operations during the Olympic Winter Games. Feeds from all competition venues are sent to the technical facilities at the IBC to be then accessed by broadcasters, many of whom have a physical base of operations in the IBC. Due to the size and complexity of the IBC, for the 17 days of the Games, the IBC serves as the largest broadcast centre in the world.

OBS and eight rights holders operate from the Main Media Centre (MMC) in the newly-built China National Convention Centre.

A secondary broadcast centre in Zhangjiakou is operated by OBS for alpine events, additionally hosting five broadcaster facilities.

Digital transformation

Underpinning all the technical capability of OBS’ move to remote and virtual workflows is the switch from traditional broadcast hardware to one that’s fully IP-based. This change was completed a couple of years ago. Equally important is a broadcast-specific cloud-based platform that OBS created with Alibaba. Content delivery, signal processing, post-production and many other elements are now based in the cloud.

Excharcos says, “For the first time, OBS will distribute the multilateral signals in HD and UHD via the cloud, and it will also be the first time that rights holders will have the ability to edit the content available on our online distribution platform [Content+] remotely. It means a more efficient way of working and addresses broadcaster’s huge demand for content to share on their digital platforms, without having to multiply their resources.”

Virtualised outside broadcast test

An important project for Beijing 2022 is a virtualised OB van which has a control room based on virtualised “cloud-ready” technologies. “We are trying to replicate the way an OB functions with reducing the physical broadcast footprint to the minimum,” explains Exarchos. “This pilot programme will be based at the curling venue. Depending on the results, we could use such a set-up at future Games. Not only does it offer greater flexibility and scalability, but the OB operations could be performed in a much more sustainable way than having a huge number of trucks coming from all around the world.”

The first stage of this project prioritises functionality and interoperability, as well as ingesting and processing of 1080p50 SDR video feeds from 18 cameras used for the coverage of one of the sheets at curling. Four additional native IP cameras, dedicated to the virtualised OB van project, will be connected to the network stack, eliminating the need for camera control units.

“Virtualisation will redefine broadcast production requirements and workflows and simplify them,” says CTO Sotiris Salamouris. “It will also reduce the set-up time to the minimum, given that the systems can be configured remotely before moving to the host city. In future, one could imagine having more flexible production crews, not necessarily based in the venue compounds, but perhaps operating from their own country premises, eliminating the need for dedicated full production crews on-site and avoiding the long-term rental of worldwide broadcast equipment for the live production of the Games.”

Nonetheless 15 conventional OB vans plus one flyaway system and nine ‘field datacentre production units’ will be deployed in Beijing.

The operation in numbers

Additional coverage is provided by 12 beauty cams dotted all over Beijing capturing iconic landmarks such as the Great Wall, Tian’anmen Square, and the Forbidden City. Helicopters will shoot ‘fly-bys ‘ at the start of competition. Half a dozen drones will complement the live coverage, some being tethered.

OBS will also generate content with smartphones, providing 8000 short video clips from back-of-house athlete areas designed for publishing on social media.

6000+ Estimated hours of content produced by OBS for Beijing 2022 - more than 6x than Torino in 2006

1.7 Tbps – of international connectivity – the most ever for an Olympics (Pyeongchang had 500Gbps)

660+ camera systems – including 13 railcams, 11 cablecams, 148 speciality systems, 38 high speed

4300 – OBS personnel (34% hired in China)

5700 – accredited personnel from rights holding broadcasters

130 – broadcasters including sublicensees taking the feed

0 - figures for the carbon savings, sustainability goals, or total carbon footprint of the operation (although reduced footprints, reusable IP kit and remote workflows will be contributing to reductions)

Primetime for 5G

OBS is able to expand its traditional camera positions at these Games because of more widespread 5G coverage.

“5G has now reached a level of sophistication where it can be applied to production in earnest,” says Mario Reis, director of telecommunications. “Thanks to the full-scale implementation of a 5G network across all Beijing 2022 Olympic venues, we can now use its capabilities in our live coverage. It provides our production teams with greater flexibility than having to tether and plug in wired cameras, and the added mobility of camera positions will help OBS capture the action from unique angles.”

Where, in Tokyo, 5G-connected cameras were only used for ENG coverage at the Ceremonies, for Beijing 2022 production teams will deliver live signals over 5G from more than 30 live and near-live cameras, including those fitted on snowmobiles at crosscountry skiing and those at the start and finish areas at alpine skiing. 5G-connected cameras will also be used as part of the virtualised OB van project to capture the action from curling.

Intel has helped select the 5G transmission and receive technology (encoders and modems) that goes with those cameras. China Unicom has established the telecom networks required to send the 5G signals back to the IBC, as well as tweaking the established network to allow OBS to do all this in real-time with low latency transmission from cameras to production units.

“5G has great capacity to support low latency and high bandwidth live broadcast transmissions over a public infrastructure,” says Salamouris. “This is certainly a key enabler for field production, especially considering the limitations of the legacy broadcast solutions that rely on dedicated radio frequencies that become scarcer and scarcer.

“However, there are several challenges for this approach to take off, since it requires certain network configurations that the carriers should adopt to secure the necessary availability levels that broadcast requires. On that front, China Unicom, with Intel, was really instrumental helping us engineer a solution appropriate for our quite demanding needs.”

Going remote

For Beijing 2022, the majority of the broadcasters will be using remote production to run all or part of their production outside of China. Driven in part by the necessities of the pandemic, but also the shift to remote production workflows, the size of broadcast teams being sent out to the Games has fallen dramatically - nearly 40 percent fewer broadcast personnel on-site in Beijing compared to PyeongChang 2018. For the first time, more than 20 broadcasters will receive feeds in real-time at their centralised production house back home through the cloud.

“The fact that broadcasters have shown significant interest for this new service is indicative of the growing integration of cloud-based workflows and how broadcasters are offloading more of their traditional video infrastructure to the cloud,” says Exarchos.

The OBS Video Server is now fully hosted in the cloud to provide “a more efficient and scalable system, while reducing on-site hardware and all costs associated.” This is also first time that OBS’ Multi-channel Distribution Service will be distributed via cloud as well as satellite.

All in all, this means that for the first time in Olympic broadcasting, the distribution of live signals over the cloud will be of equal volume as through standard delivery models.

8K

China Media Group (CMG) and NHK (rights holders for China and Japan, respectively) will collaborate to produce 8K coverage from select venues as well as from the Bird’s Nest Stadium. NHK is producing the figure skating events; CMG takes charge of the Ceremonies, freestyle skiing/ snowboard big air and speed skating in 8K. A feature of the Main Media Centre (MMC) is a 15 x 8 metre 7,680 x 4,320 pixel display for showing off the results.

“Early Olympics trials of 8K were simplistic,” says Salamouris, “in that we were only using a few cameras, and only one type of production unit. Now the whole 8K production has really matured and the technology around it is moving quickly so that it will eventually become an option for more and more broadcasters.”

8K VR

Beijing 2022 marks the first ever multi-sport event to be covered in 8K VR. 70-80 hours of the format will be produced from sports selected on the ability to place cameras close to the athletes.

Up to six 180-degree monoscopic cameras and one 360-degree camera will be in action. Viewers will be able to choose camera perspectives of live streams or watch a produced stream on VR headsets Oculus Quest/Quest 2 and Pico, and also on mobile. VR VOD content will be available. Broadcasters are further invited to use the 8K VR feeds as presentation backdrops.

5G plays into the evolution of VR at the Games. High downstream bandwidth from 5G will “free” consumption of content to be happening anywhere, and not just where Wi-Fi is available.

“A few years ago, it would have been hard to discern faces and other details while using VR, but now with 8K resolution the experience is far more lifelike,” Salamouris says. “However, that increased resolution can only be available to the user if they have the network to support it. With 5G you have the necessary quality to transmit to final users, be it on a 5G enabled VR device.”

AI / ML

AI tools will be used in figure skating and ice hockey to speed highlight package creation. “We are moving closer to being able to integrate AI technology as part of our toolsets and using it for our video tagging workflow,” says Salamouris. “If this content isn’t properly tagged, then it is very difficult to work with. For a long time, we’ve employed students to tag our live content. While we wouldn’t replace those students with an algorithm, AI would allow us to tag far more content and offer ultimate flexibility and expandability compared to the capabilities of human-only loggers.”

Although some athlete tagging is currently achieved by OBS loggers, it is practically impossible to tag all athletes in all available video frames, says Salamouris, but this is often what rights holders require.OBS in-house technology, which it began developing ahead of Tokyo, is called Automatic Media Description. “We train the system to automatically search for specific content/video sequences, and once indexed, stitch this content together to produce quick highlights packages which are made available to OBS producers,” he explains.

“After Beijing 2022, we will start experimenting with automatic switching, which would mean using AI in live broadcast operations,” he says.

Data visualisation

Specifically for Curling, data gathered from an overhead camera will visualise stone trajectories, contact points, and distance between stones for on-screen analysis. In other disciplines, real time ‘Jump’ data such as speed, height, length, duration and angles of skis is collected from motion sensors and processed by computer vision analysis.

2D image tracking

After being introduced at Tokyo, 2D image tracking (also referred to as athlete ‘pinning’) technology crops up again in Beijing, for the biathlon and crosscountry skiing events.

A ‘patch’ (a square) is defined on selected video frames to identify each of the athletes. The computer then creates a ‘label’ that is attached to each of the identified athletes which is maintained even as the image changes. This captured data is then made available to a graphics rendering platform for on-screen presentation, displaying the exact location of the athletes.

Additional data captured using more traditional GPS positioning can be combined with the ‘labels’ to identify athletes, their speed, distance to finish or relative position to the leader.

Live speed measurement

Speed measurement in the coverage of alpine skiing events was introduced as part of the TV graphics at Sochi 2014. Until now, the speed was measured from a very specific position and limited to only snapshots, or just a few seconds of data. This was due to the limited sensor coverage. For Beijing, OBS will deploy a multitude of antennae with increased reception capacity that allow for the capture of more data throughout a much larger portion of the downhill course. The use of such technology will also guarantee an overall better speed measurement in terms of accuracy and frequency of updates.

Multi-Cam Replays

Arrays of 4K high speed cameras will be deployed at ten venues including at Figure Skating, Ice Hockey, Freestyle Skiing/Snowboard Halfpipe and Short Track Speed Skating. Another system will be near the end of the take-off ramp of the space-age looking ski jumping facility to capture the first seconds of the skier’s flight.

These replays can be paused at different points in motion, an effect “similar to action scenes in the The Matrix,” says OBS. A rig with 120 4K cameras will be used at the Ice Hockey venue.

For curling and speed skating, OBS will process multi-camera replays in the cloud, the first time this has been attempted live. All frames captured by the array of cameras installed at these venues will be sent to an edge server and reconstructed in the Alibaba Cloud to generate the replay clips. Those will be up-converted to 4K in the cloud before being sent back to the production unit in the venue compound.The rigs are remotely operated by a single operator who can freeze the action, manipulate the replay from side to side around the athlete, as well as zoom in. Since the system simply stitches together these feeds and does not have to virtually create filler frames, no rendering is required, allowing clips to be ready in under five seconds.

“In some cases, you need multiple cameras that are placed around an object and fly the cameras around, but you also have the volumetric technology that Intel has developed, which records and recreates a model of a solid object,” informs Salamouris. “In the first case, the flight camera pattern is more fixed, whereas in the second case, the producer has much greater flexibility in terms of the flight camera trace that you can build around it.”

While the volumetric approach is much more flexible and richer in terms of produced results, it comes with significant computational complexity that also leads to much higher turnaround times, compared to the simpler ‘stitching-based’ method. In Beijing, OBS will deploy both technologies.

Gender equality

Beijing 2022 will feature the closest ratio of female to male participation at a Winter Games, with female athletes making up about 45 percent of those competing. New mixed gender events include snowboard cross mixed team, ski jumping mixed team event, and freestyle skiing mixed team aerials. More gender balanced coverage is on editorial minds at OBS; the term ‘women’ will be used across all sports, rather than ‘ladies.’

“Sport media has traditionally been male dominated and we will continue to keep up our efforts to open new opportunities for females to work in sports broadcasting so that eventually we have a 50-50 split between women and men,” Exarchos says. “We are not there yet, but we are on the right path and would encourage our fellow broadcasters to follow our example.”