Where Are We With Audio Asset Management?

NAB

Cloud storage is getting stuffed full of video but there’s a surprising amount of data related to audio assets, too.

article here

The reason behind this is the dramatic growth in the audio book and podcasting markets, according to a new report from tech vendor Backlight.

In figures quoted in the report, the audio book market was valued at $4.2 billion in 2021, with an expected revenue of $35 billion in 2030. The global podcast market is also projected to be worth $98.78 billion by 2030, with a CAGR of 30.65%.

The report itself is based solely on data from users of Backlight’s media asset management (MAM) system Iconik, but the company believes it reflects the direction of trends in the wider market.

Iconik is a MAM built for the hybrid cloud, meaning it can manage media on both cloud and on-premise storage. Since the pandemic, cloud storage has accounted for nearly 70% of iconik’s data.

As the cost of cloud storage drops, more companies are taking advantage of this to become more agile. This year the hybrid cloud data split is 67% cloud and 33% on-premise — which is consistent with previous years.

Numerous industries are using the cloud to store assets but the largest is Media & Entertainment, which makes up 62.5% of Iconik customers.

Of course, even in M&E there are countless sub-industries which Iconic identifies as, “Companies that engage in producing and selling entertainment products and services, including companies engaged in the production, distribution and screening of movies and television shows, producers and distributors of music, entertainment theaters and sports teams. Also includes companies offering and/or producing entertainment content streamed online.”

Use of AI services for transcribing and auto-tagging media has risen 40%. AI transcriptions are being used to save time on manual work, while adding huge amounts of metadata into assets with entire transcripts being made searchable.

Per the report, AI transcription technology is more affordable than visual AI technology, which makes it a good option for organizations who want to start using AI to enrich their media.

“However, the majority of our users are not using this technology yet. We believe that visual AI needs to improve so that it can be trained to the needs of each organization. Current pricing models are also not ideal for large media archives.”

Because of this, the company says it will invest in more custom AI tools in 2023 so businesses can train AI to fit their tagging needs.

According to figures in the report, the Global MAM Solutions Market is expected to reach $12.2 million by 2027, growing at 23.95% during the forecast period.

No, AI Can’t Create a Film — But Here’s How Filmmakers Can Create With AI

TechInformed

With all the hype these past few months you’d be forgiven for thinking that AI is on the verge of taking over Hollywood, but execs, directors and scriptwriters needn’t worry about their day jobs just yet.

article here

“Don’t drink the Kool-Aid,” says Yves Bergquist, Director of AI & Neuroscience in Media at studio-funded think tank ETC, who has written a paper, “Generative AI in Media: What it Means For The Industry,” on AI’s creative and business implications. “The technology is still experimental, highly dependent on large and clean training data (which is expensive to collect and maintain) and ungodly amounts of computation.”

In particular, generative AI is currently very bad at producing anything like a decent story.

“ChatGPT is fresh and still being iterated, but the capabilities displayed so far aren’t a threat to anybody in the media industry. The AI apps that would create (or even optimize) deep narrative content haven’t been invented yet.”

That said, the tech is progressing very fast, and Bergquist thinks it only a matter of time until applications and use cases emerge that will transform the media industry.

In which case, “media executives need to be educated about what AI is and isn’t. Technical executives need to learn to talk about AI to non-technical executives. New skills are needed, such as managing the data-business interface (and the trust issues around it).”

The document is meant to help media professionals think more clearly about AI and its implications for the industry, especially around what skills will be needed in an AI-augmented future.

Good AI Needs Humans

The paper puts forth that the technique that has unlocked the AI genie in tools like ChatGPT is in fact a clever fusion with human moderators. OpenAI, as Bergquist explains in an article in the ETC etcentric blog, used a method called Reinforcement Learning through Human Feedback.

This is essentially a Neural-Symbolic process of continual tweaks and fine-tuning by human teams to train and re-train the AI model.

“When it’s good (often) the machine-output text is indistinguishable from human output, which is probably why the education industry is (justifiably) freaking out,” says Bergquist. “When it’s bad (also often), it betrays inescapable yet fundamental flaws in OpenAI’s methodology: it is hyperscale parroting, not intelligence per se. But it works, which is why it’s garnering so much attention.”

What AI is Good For

How does this translate into practical opportunities or threats to the media industry?

Because ChatGPT is very skilled at outputting basic, low-narrative content, such as an email, website copy, or a press release, ETC thinks the main opportunity for it right now is SEO, “where writing a lot of copy (blogs, websites, etc.) is key to create a lot of links and jack up search engine rankings. Another use is creating bot content on social media on a massive scale. In the media industry, ChatGPT can be used in combination with other generative AI systems to more quickly create previs assets like storyboards and schematics.”

Creative iteration could get a lot easier and faster. And that could be a new way to work. Generative AI models will be powerful (but still pretty dumb) assistants for creatives. Putting a look book together will become a lot easier and faster. Some parts of writing, such as log lines, will become semi-automated.

Marketing functions pivoting on search engine optimization or social media campaigns will need to adjust how they measure success. ETC says they will need to beef up their ability to detect bots and fake accounts.

A little further ahead, opportunities abound, from automating micro-workflows within editing, post-production and CGI to letting machine learning models loose on creating entire scenes from text or image prompts, or automating large chunks of the CGI workflow.

The technology also opens up new creative avenues: voice synthesis can replicate any voice, facial replacement technology can make face blurring in documentaries a thing of the past. Music synthesis can create a score in mere hours. De-aging opens up new narrative possibilities.

What AI Can’t Do

However, ETC also provides a reality check. For while all generative AI tools can automate some of the grunt work process out of the system, its creative capabilities leave a lot to be desired.

“ChatGPT doesn’t comprehend the concepts or symbols present in its output. It can’t abstract or reason in very basic ways. It doesn’t understand anything about the world it’s in, especially causality,” says Bergquist.

“ChatGPT doesn’t have any ability to even understand the basic tenets of a story, let alone build one. Its output is too basic to be even interesting,” he continues.

“ChatGPT cannot perform highly in domains that it has not explicitly been trained and tuned on. It cannot write a novel, or a script, or any kind of text that requires an understanding of the world and its nuances. It’s a complex and highly tuned statistical engine that doesn’t actually ‘know’ or ‘understand’ anything. It can’t write engaging or even slightly narrative text beyond a few paragraphs. It can only have ‘good ideas’ by accident.”

While synthetic image generation is “staggeringly good, and can produce images that could be true works of art,” Bergquist thinks audiences will find its output meaningless as entertainment.

“The point of art is equally the art itself and the artist’s persona and brand. What makes art valuable is the rebellion and talent of a human reaching into deep and previously unknown areas of the human condition.”

In sum, a handful of creators will leverage generative AI models to take their workflows to the next level. As ETC puts it, those creators who can leverage ChatGPT into their workflow to laser-focus their time on the core craft of exploring higher-level human narratives will win the era of augmented content creation.

Studios Need AI Talent

Other winners will be studios, which own their distribution platforms and detailed audience data, and which can use machine learning to understand which content attributes resonate with various audience segments.

“Hollywood will continue to rule,” Bergquist insists. “In an ocean of content, the value lies with curation and personalization. Time being the world’s most precious commodity, exhausted digital denizens will pay a high premium for a service that can deliver them the exact content that they need or that inspires them. And nobody is better at sorting this signal from the noise than the giant talent-filtering algorithm called Hollywood, which also knows something Silicon Valley keeps ignoring: people actually hate technology.”

To this end, studios and content producers are advised to partner with startups to leverage their own data to increase their competitive advantage in the most valuable area of the generative AI.

ETC also suggests that studios should be hiring talent with AI smarts. These include AI engineers with math and computer science skills who can build products, as well as AI engineers with a business (or product development) background who are the interface between the data science function and the C-Suite.

“They can effectively break down business problems in terms of data (for the data science team), as well as communicate to business stakeholders about AI (for the C Suite). We call them ‘hackers’ because they’re scrappy, computational thinkers obsessed with solving business problems with data and computation.”

Senior technical executives with a deep understanding of AI are also important to guide on how budget, plan and staff for it, as well as how to talk about it and which ethical considerations need to be present.

In addition, there will also need to be a greater focus on digital watermarking to ensure content authenticity, ETC finds.

AI Is Also Great at Generating Chaos

NAB

After a period lasting about a year when all things AI were generally considered a benign boost to human potential (the worst that could happen was you could lose your job to ChatGPT), the dystopian winds have blown back in.

article here

A number of commentators are calling for a brake on AI development while the risks of its use are assessed — an unlikely scenario since there appears no appetite for it among governments, legislators, big tech, or the AI coding community itself.

With tech companies rushing out AI models without any apparent safeguards (like, for accuracy), cognitive scientist Gary Marcus thinks this is a moment of immense peril. In The Atlantic he writes, “The potential scale of this problem is cause for worry. We can only begin to imagine what state-sponsored troll farms with large budgets and customized large language models of their own might accomplish.

“Bad actors could easily use these tools, or tools like them, to generate harmful misinformation, at unprecedented and enormous scale.”

He quotes Renée DiResta, the research manager of the Stanford Internet Observatory, warning (in 2020) that the “supply of misinformation will soon be infinite.”

Marcus says that moment has arrived.

“Each day is bringing us a little bit closer to a kind of information-sphere disaster,” he warns, “in which bad actors weaponize large language models, distributing their ill-gotten gains through armies of ever more sophisticated bots.”

His comments coincide with the results of a annual survey of machine learning researchers called AI Impacts. In its latest report, 14% of participants said they expected AI outcomes to be ‘extremely bad’ and nearly a third (31%) think AI will, on the whole, make the world markedly worse.

Just to be clear, AI Impacts chief researcher Katja Grace, says that they are talking about human level extinction.

She added, “The most notable change to me is the new big black bar of doom: people who think extremely bad outcomes are at least 50% have gone from 3% of the population to 9% in six years.

In another op-ed, The New York Times columnist Ezra Klein expresses dismay at what he sees as the lack of urgency among AI developers to do anything about the Frankenstein they are unleashing.

He says the people working on AI in the Bay Area inhabit “a community that is living with an altered sense of time and consequence. They are creating a power that they do not understand at a pace they often cannot believe.

“We do not understand these systems, and it’s not clear we even can,” he continues. “That is perhaps the weirdest thing about what we are building: The ‘thinking’ for lack of a better word, is utterly inhuman, but we have trained it to present as deeply human. And the more inhuman the systems get — the more billions of connections they draw and layers and parameters and nodes and computing power they acquire — the more human they seem to us.”

Klein seems less concerned about machine-led mass human extinction than he is about having even more of his data — and therefore media consumption — turned into advertising dollars by Microsoft, Google and Meta.

Since these tech giants hold the keys to the code he thinks they will eventually “patch the system so it serves their interests.”

His wider point though is to warn that we are so stuck on asking what the technology can do that we are missing the more important questions: How will it be used? And who will decide?

“Somehow, society is going to have to figure out what it’s comfortable having AI doing, and what AI should not be permitted to try, before it is too late to make those decisions.”

Marcus offers some remedies. Watermarking is one idea to track content produced by large language models but acknowledges this is likely insufficient

“The trouble is that bad actors could simply use alternative large language models to create whatever they want, without watermarks.”

A second approach is to penalize misinformation when it is produced at large scale.

“Currently, most people are free to lie most of the time without consequence,” Marcus says. “We may need new laws to address such scenarios.”

A third approach would be to build a new form of AI that can detect misinformation, rather than simply generate it. Even in a system like Bing’s, where information is sourced from the web, mistruths can emerge once the data are fed through the machine.

 “Validating the output of large language models will require developing new approaches to AI that center reasoning and knowledge, ideas that were once popular but are currently out of fashion.”

Both Marcus and Klein express alarm at the lack of urgency to deal with these issues. Bearing in mind that the 2024 Presidential elections will be upon us soon, Marcus warns they “could be unlike anything we have seen before.”

Klein thinks time is rapidly running out to do anything at all.

“The major tech companies are in a race for AI dominance. The US and China are in a race for AI dominance. Money is gushing toward companies with AI expertise. To suggest we go slower, or even stop entirely, has come to seem childish. If one company slows down, another will speed up. If one country hits pause, the others will push harder. Fatalism becomes the handmaiden of inevitability, and inevitability becomes the justification for acceleration.”

One of two things must happen, he suggests, and Marcus agrees: “Humanity needs to accelerate its adaptation to these technologies or a collective, enforceable decision must be made to slow the development of these technologies. Even doing both may not be enough.”

 

You Need to Consider AI Your Creative (Capable) Assistant

NAB

In a theme that will resonate across Hollywood, the majority of users at stock photo site Shutterstock are using generative AI “for inspiration.”

article here

“They’re using it to get rid of that blank page problem,” said Meghan Nally, chief product officer at Shutterstock. “They’re using it as a starting point.”

The idea of AI as “a universal basic intern” (an assistant that can be used in your daily work, whatever your line of work) was the overall theme at SXSW, where Nally was speaking. The partnering with AI tools as co-creator seems to be the overriding opinion of those companies pushing AI tools in the media and entertainment industry.

Shutterstock launched generative AI capabilities earlier this year and now has two million new images being generated every week.

“Generative AI: Where Creative and Tech Innovation Meet” at SXSW, with panelists Jake Kwon of LG AI Research, Sarah Hoffman of Fidelity Investments, and Fred Werner of AI for Good, moderated by Meghan Nally, Chief Product Officer, Shutterstock

“These are images that didn’t exist before,” Nally said. “These are things that only existed in people’s imagination that then they’ve been able to bring to life in a matter of seconds.”

At a live presentation at SXSW titled “Generative AI: Oh God What Now?“ two technologists, Romain Bogaerts from Real Chemistry and Sameer Grover from AbbVie, pondered how many creativity-driven jobs will get taken over by machines. In a Shark Tank-esque pitch session, entrepreneurs proposed new ways to integrate AI into entertainment, such as by splitting audio stems or visualizing film scripts automatically. A SoundCloud executive told another audience that people who categorically reject AI-generated music sound “a bit like the synthesizer haters” of electronic music’s early days.

“AI could be an amazing tool to help democratize a lot of the aspects in filmmaking,” actor Tye Sheridan told Brian Contreras at the Los Angeles Times. “You don’t need a bunch of people or a bunch of equipment or a bunch of complicated software with expensive licenses; I think that you’re really opening the door to a lot of opportunity for artists.”

Along with VFX artist Nikola Todorovic, Sheridan founded Wonder Dynamics, a Hollywood-based company focused on using AI to make motion capture easier.

In a demo Sheridan and Todorovic showed the LA Times, the software took an early scene from the James Bond movie Spectre — of Daniel Craig walking dramatically along a rooftop in Mexico City — and scrubbed out the actor to replace him with a moving, gesturing CGI character. The benefits, to Sheridan, are straightforward.

“I mean, you don’t have to wear those silly-looking motion capture outfits anymore, do ya?” he said.

BuzzFeed chief executive Jonah Peretti believes this isn’t another bubble like NFTs or stereoscopic filmmaking that is destined to burst. The rise of AI is more akin to mobile phones or social media, he said, “massive trends that changed the economy and society and culture.”

Also at SXSW, Amy Webb, CEO of consulting firm the Future Today Institute, imagined a world in which AI programs are used to mass-produce many different versions of a single TV pilot, either to focus-test them before release or to show different ones to different viewers after.

“I bet sometime in the next handful of years that there becomes this horrible industry practice where you have to have multiple variations before things are greenlit,” Webb said. “And then there’s a predictive algorithm that tries to determine which version has the highest likelihood of grossing the most [money].”

The rise of AI in writing has also raised concerns by unions representing screenwriters, who fear studios might replace experienced TV and film scribes with software. According to the LA Times, the Writers Guild of America will demand studios regulate the use of material produced by artificial intelligence and similar technologies as part of negotiations for a new pay contract this year.

Fred Werner, who works for the UN information and communications agency ITU on a program called AI For Good, was eager to talk about inherent bias in AI data sets.

He said, “When you’re developing these tools, do they work equally well on men and women or on the elderly, on children or on people of different skin colors or people with disabilities or in low resource settings and in the least developed countries? These are not questions that occur naturally to the fast tech startup industry [where] it’s more like build it first and fix it later.

“This is important because you need to create a kind of common framework of understanding if you’re going to connect AI innovators with problem owners.”

A “problem owner”* could be a local mayor, a doctor, an NGO, a teacher, a scriptwriter. The question for AI For Good is how can people engage with AI tools in a way that they know data sets are inclusive.

“You’re mindful of things like gender and disabilities and where people live. And right now there’s a digital divide. As useful as these tools are as feeling like having ten interns in my pocket, that’s just going to increase the digital divide for people who don’t have it.”

Understanding that outputs are only as good as the inputs, Nally explained that Shutterstock is attempting to address that concern. She said the company is applying “logic” between the input and the output that allows them to filter for things like hate speech.

“It ensures that we’re generating results that are diverse. That we’re mitigating bias. It’s not humans that we’re [using] to help get to the place where you’re generating visuals that are representative, that aren’t full of hate,” she said.

The industry is going to have to learn how to apply that filter of machine logic on top of the basic AI models, Nally said.

Nally also addressed the way Shutterstock is handling royalty payments on generative content.

“Whenever a piece of content is generated and downloaded on our platform, we pay a royalty to a contributor fund just like we would direct to contributors. That’s incredibly important to us because our business model does not exist if we’re not inextricably linking supply and demand creator, artist, contributor. So we’ve been really thoughtful to ensure that whether this is 50% or 5% of content budgets in the future, that our contributors will have the ability to continue to make money off of their content in a variety of ways on our platform.”

Shutterstock has more than 600 million assets in its library not all of them get sold. But AI promises to help monetize the long tail.

“We have contributors who are uploading their work and have not ever made a dollar on Shutterstock. Now all of a sudden they have the opportunity to make money because of this technology. It’s [no longer] just how effective you are direct selling to a consumer,” said Nally.

“I think we are the only company in the world that’s created this ecosystem today that truly connects generative outcomes with contributor compensation.”

 

FIFA World Cup Qatar 2022’s Record Live Streams Set Out Future of Fan Engagement

IBC

The FIFA World Cup Qatar 2022 was always certain to be a tournament like no other. It was the first time that the most famous competition in global sport was held in the Arab world and one which saw streaming services set a new benchmark for the possibilities of live event coverage.

article here

Around five billion people engaged with the tournament content, according to FIFA, across an array of platforms and devices across the media universe shattering the record set in 2018, when 3.575 billion people watched. On social media, according to Nielsen, there were 93.6 million posts across all platforms, with a 262 billion cumulative reach and 5.95 billion engagements.

FIFA World Cup Qatar 2022: Unprecedented live traffic volumes

In terms of scale, the FIFA 2022 World Cup Qatar was content delivery network Akamai’s single-largest traffic event. “During the 2022 World Cup we set a peak traffic record with 261Tbps on the platform due to the combination of the France v Morocco semi-final and a major video game download,” explained Harish Menon, Senior Director, Global Broadcast Operations and Customer Events, Akamai. “This is why we plan for these large events, accounting for peaks in traffic and regular organic growth.”

The epic final in which Argentina’s GOAT Lionel Messi went toe to toe with French superstar Kylian Mbappé was one of the most dramatic sporting events ever, generating unprecedented internet traffic volumes worldwide for an individual game.

In Argentina, which won on penalties after the game ended two goals each, the final was aired across three channels (TV Publica, TyC Sports and DirecTV) – with a combined audience of 12.07 million viewers, per FIFA stats.

In France, the final on TF1 attracted an average audience of 24.08 million viewers, 81% of the audience share. This was 24% greater than the audience for the France v. Croatia final in 2018 (19.38 million) and an all-time viewing record in the country.

In the US, the final attracted a combined audience of almost 26 million – with the coverage on FOX being the most watched English-language broadcast of a FIFA World Cup in the USA. The final was also the most-watched match of the tournament in Spanish, with a Total Audience Delivery of nine million viewers – a 65% increase compared to the 2018 final.

Coverage of the final aired live across the middle east region on beIN Sports. The match achieved an audience reach of 242.79 million viewers – equivalent to 67.8% of the channel’s potential television audience.

In Mexico, viewing peaked during the Mexico-Argentina match when 20.96 million viewers, about 67.9% of the total viewing population was engaged. It also became the largest FIFA World Cup Group Stage match in Spanish-language history when televised by Telemundo in the US.

Not only were there significant increases in streaming VOD and live streaming during matches, but surges of traffic within matches as digital engagement was triggered by goals, yellow cards, and penalties.

A North American client of traffic management solutions provider Sandvine saw traffic steadily climb in 10-minute increments of the final, reflective of the excitement generated by each goal and each yellow card as the match grew more contentious. Twitter traffic surged from 60GB to 130GB once the World Cup started, Sandvine reported.

FIFA World Cup 2022: Exponential engagement

In Japan, youth-focused streamer Abema procured streaming rights for all FIFA World Cup games in Japan. According to Sandvine, Abema didn’t reckon on the fact that each match Japan won would spur more interest for the next match, growing from 10 million to 30 million active daily viewers by the time Japan played Croatia at the end of November. To keep up, Abema engineers reportedly had to temporarily restrict access, which caused a flood of Twitter complaints from its younger viewers.

In Brazil, the tournament attracted an overall reach in the country of 173 million or 81% of the population.

Over 51 million UK viewers were reached across the entire tournament, representing 83.9% of the potential market audience, according to FIFA. Service provider Red Bee Media tells IBC365 that it even had occasions which included 68 EPG changes during the same day to serve its international broadcasters.

BBC figures reveal the matches across the tournament were streamed 92 million times on iPlayer, while also reaching 38.8 million on BBC TV.

Coverage of Portugal’s round-of-16 match against Switzerland delivered the most-watched FIFA World Cup broadcast ever recorded in Portugal, with an average of 3.89 million viewers – 71.8% of the broadcast share.

FIFA World Cup 2022: Quality the key metric

The clear trend is towards richer, higher-quality online consumption. Broadcasters and streaming services providers will be looking now to the FIFA Women’s World Cup from Australia and New Zealand this summer, then the Paris Olympics 2024. Further ahead, during the 2026 World Cup hosted in Canada/Mexico and the USA, fans will want an even better experience with higher resolution and low latency mere table stakes.

Fabio Murra, SVP Product & Marketing at V-Nova observed: “For World Cup 2022, certain matches available in UHD HDR with surround sound on streaming platforms were only available in HD with stereo on broadcast channels. On the other hand, a shift to streaming highlighted the latency issues that are still a challenge for this delivery method.”

Jason Friedlander, Sr. Director Product Marketing, Media Platform, Edgio stressed the need for operational scale. “During major live events like the World Cup, there are so many moving parts behind the scenes you have to be as efficient as possible. I’ve heard of some companies deploying up to 100 people on live events, which is just not sustainable and creates significant overlap. The complexity involved is huge, and to be honest I don’t think streaming providers are taking on board the challenges they will face as live streaming and programmatic grows from here.”

By 2026 sports content will be primarily streamed. According to Murra, “High-quality, low-latency services will deliver a reliable quality of experience for traditional 2D viewing. Perhaps, one of the most significant changes will be the use of more advanced technologies for extended reality (XR). These technologies will enable viewers to experience sports events in a more immersive way, give them the feeling of being at the venue, and enjoy it with friends and family that are physically far away.”

Red Bee expect greater personalisation in presentation – “particularly in the way that more personalised and local ‘shoulder’ content is used to augment mass audience moments. Serving the audience wherever they are, while maximising the opportunity to unite the nation.”

FIFA World Cup 2022: Showing the way for future engagement

Julie Souza, Head of Sports, Global Professional Services, AWS said, “There’s a desire by many leagues and teams to be closer to the fan, so we could potentially start seeing them launch services and experiences to complement the streams on more traditional broadcasts and OTT streams.

“We’re heading in a direction where the viewer holds the producer’s reins versus producers creating an experience they think works for the masses.”

By 2026, in-venue, 5G and edge computing will help ensure lower latency content to mobile devices of spectators at the stadium so that there’s more continuity between what’s they’re seeing unfold live and what they’re seeing on their phone.

A survey from Grabyo found that nearly two thirds (65%) of sports fans watch video on their smartphone (46%) compared to 43% of those that watch on their TV. Red Bee says these numbers highlight the need for sports content delivery to be more flexible and cater to market diversity as media companies expand across rights holding, content acquisition, packaging and delivery – for cable, satellite, digital TV and mobile.

“Consumers, more than ever, now demand customised and regionalised versions of live matches,” said Rick Young, SVP, Head of Global Products, LTN Global. Players like Amazon and Apple have forever changed the landscape causing traditional media players to rethink their strategy on how live sports fits into future plans. The 10-year deal by DAZN to license NFL rights across their global OTT platforms shows that platform-specific, digital-first delivery continues to grow quickly.

“In order to meet viewer expectations, experiences must be tailored for the specific platform or regional audience with the right graphics, language, shoulder content and advertising.”

FIFA World Cup 2022: Augmented reality

This is on FIFA’s mind too. New technologies and productions designed to be consumed via various devices were some solutions that FIFA proposed to its broadcasters in order to appeal to younger generations in 2022. For the first time at this event, social media content in vertical format was captured via mobile phone and distributed to broadcasters to use in their coverage.

An Augmented Reality app was developed by FIFA, offering viewers content through mobile devices, “transforming living rooms into a 3D data centre,” FIFA said. New tracking data allowed fans to get insight into a specific player or team to get stats and in-depth analysis of their performance, in real time.

The organisation also developed services to populate broadcaster services, such as VOD clips, near-live multi-angle clips or near-live statistics.

As Romy Gai, FIFA’s Chief Business Officer, summarised in a release: “Viewers are becoming increasingly active and less passive in how they consume content and in the future will have the option to choose how to enjoy an event like the FIFA World Cup – whether that be via livestreaming solutions more suited to VR or through gaming and the possibilities the metaverse will bring.”

Destination Cloud: Here’s How Live Broadcast Can Get There

NAB

SMPTE’s master plan for moving broadcast production into the age of IP didn’t anticipate COVID (how could it?) and the cracks are beginning to show.

article here

The pandemic has made such a time jump on the use of remote distributed video over the internet and cloud-based technology that grounding live broadcast studio workflows in specifications that mirror the gold standards of SDI may outlive its usefulness even before the industry fully transitions.

It was long thought that it would be at best compromised if not downright impossible for live programming to be produced in the cloud, yet this is what some corners of the industry, including SMPTE itself, are contemplating.

One of the innovators pioneering this change is Mike Coleman, a veteran broadcast engineer and co-founder of Portland, Oregon-based Port 9 Labs.

In an excellent blog post written by Michael Goldman for SMPTE and in a public demonstration of Port 9’s proposed cloud-based live switching technology, Coleman explains how SMPTE is working to develop new architectures for live remote video broadcasting — in the cloud.

He argues that the industry has by necessity begun moving parts of its production equation to the cloud, but that this is to a large extent piecemeal.

“If you examine how a cloud-native service would be built, it would be radically different than the architectures you are seeing for these lift-and-shift kinds of things. In other words, for now, there is a big disconnect,” he says.

Coleman admits it is still early in the company’s development of its own cloud-native architecture for doing production in the cloud, and that the industry will be slow out of necessity to evolve in that direction generally, but he nevertheless believes such a transition is inevitable.

“Right now, we have lots of lift-and-shift going on,” he explains. “That means people are moving existing ground-based solutions into the cloud. Since the [pandemic], people have been under a lot of pressure to take what they already have on the ground and incrementally change it to somehow make it work in the cloud. But they are starting to realize their limitations, and the industry is starting to understand it needs to adapt.”

Coleman believes it is now possible to build IP-based media systems that can be used via public cloud services and says his company has had success moving uncompressed video on multiple public cloud systems using multi-flow UDP (User Datagram Protocol).

“Cloud IP media services would be managed as SaaS [software as services]. Broadcasters would control the programming from anywhere they choose, but the underlying service will be maintained by the service provider,” is how he and his colleagues describe it in a separate article written for the SMPTE Motion Imaging Journal.

“It’s definitely an over-the-horizon thing and will likely take many years to get there,” Colman says. “But, in our opinion, cloud architecture, if done correctly, would be totally different from how things are done on the ground, since the whole point obviously would be to leverage the strengths of the cloud.”

A number of critical issues need to be addressed. They include addressing broadcasters chief concerns about quality — of image and of synchronization both of which are fundamental to the SMPTE 2110 family of standards.

Coleman says it should be possible to maintain quality by working with compressed media in the cloud and effectively only using uncompressed media at the point of transmission (or perhaps even rendered at display if edge compute is built out).

He picks out NDI — once anathema to broadcast engineer purists — as a robust and proven solution for sharing lightly compressed AV and metadata across IP networks.

“Generally, it is pretty good for its purpose and pretty easy to move up into the cloud, but even so, the video quality isn’t quite up to modern broadcast standards since it still requires 8-bit compressed video,” Coleman says. “Studios typically would prefer to compose video in the highest possible quality and then use compression later only for the transport phase.”

He thinks this is still a hybrid of the “lift and shift” approach and therefore not ideal. A better solution, to Coleman, is Grass Valley’s AMPP, “which is more cloud-native but still kind of in the middle between lift-and-shift and where we think it has to go.”

Coleman says one key to creating a true cloud-native architecture for broadcasters to use when producing live content involves approaching the concept of an IP-based workflow differently by taking an “asynchronous rather than a synchronous approach.”

“Today, in an IP-based studio, like with most IP-based things, you need extremely tight timing,” he explains. “Everything has to be synchronous using the PTP (Precision Timing Protocol) to [synchronize all devices on a computer network]. In the cloud that is really hard to do and we have begun to realize you don’t need to do it, because you typically have a huge amount of bandwidth and tons of CPU available in the cloud [when using a major cloud provider]. So, instead, we want to work in an asynchronous model, only synchronized on the edge if you need it to be.”

He says Port 9 is working on an architecture that works without being synchronous because everything is time-stamped: “We call this having a looser time model so that we can work on uncompressed video in the cloud.”

Another problem is egress — transferring material, and particular data heavy media, out of the cloud. That’s not a problem, per se — but the cost is.

“Cloud providers will charge you a lot of money in terms of data transfer fees,” Coleman says. “Therefore, typically, you do not want to send your uncompressed video back down to your facility on the ground. Our solution for that is to send only proxies down to the ground — that’s where we would use compression. Broadcasters are already very familiar with using proxies in their workflows.”

He says that SMPTE ST-2110 “is simply too tight in terms of timing” to work as a formal standard for live media production in the cloud, but adds that the Video Services Forum (VSF) is already at work with its Cloud to Ground/Ground to Cloud (CGGC) working group, which launched in 2020.

Among other things, that working group is examining where a common set of requirements for ground-to-cloud and cloud-to-ground data transfers would be necessary, and how best to establish a common technical approach for such data transfers.

Coleman also adds that working group “is embracing the idea of the timing model being looser in the cloud, so in my opinion, they are moving in the right direction by focusing on the data plain, or data transfer area.”

All of this has quite a way to travel before it would ever become ubiquitous in the wider industry. For now, he says the primary initial goal is to simply “sensitize people so that they can become aware that something like this is possible.”

“Broadcasters are still in this process of continuing to try incremental changes to their workflows in order to keep working as they move into the cloud,” he says. “What I’m saying is that an incremental approach won’t ever get you where you want to go. At some point, you have to make a big break. Before they can make that big break, they have to understand how it could work using [a cloud-native process]. I expect there may be a transition period of about five years before broadcasters are really using the cloud the way it ought to be used for live production. But I do think it is inevitable that it will happen.”

 

Behind the scenes: John Wick: Chapter 4

IBC

It is fitting that scenes in the latest John Wick movie are set in a cathedral since the look of the film is often that of light through stained glass.

article here 

This is the work of cinematographer Dan Laustsen ASC, DFF who also shot numbers 2 and 3 in the franchise in the same vein and whose goal with Chapter 4 was to dial up the colour. A lot.

“Everything we do in number 4 has to improve on the first three - better story, more powerful images, stronger colour,” he told IBC365. “The nice thing is we can do what we like. Nobody is interfering so we can try to do some things in a new way.”

Neon-soaked darkness is indelible to the series’ brand and something that Laustsen, with director Chad Stahelski and production designer Kevin Kavanaugh, discussed at length before shooting.

“Chad knows that Keanu looks so good when lit by single light sourcing. His face is so strong with a single source that this became the whole concept of John Wick. We don’t want to be afraid of the darkness so we use colour to lift it up while staying black, black, black.”

Stahelski picked Laustsen to shoot John Wick: Chapter 2 in 2017 after seeing the trailer for Crimson Peak which the Danish cinematographer had lensed for Guillermo del Toro. His direction to the DoP at the time was to shoot action like a Bernardo Bertolucci movie. The Italian director’s most praised films were photographed by the legendary DP Vittorio Storaro.

“Chad wanted bold colour and subtle movement,” said Laustsen by way of explanation. “Colour is an incredibly powerful asset but there’s a thin red line between getting it right and going too far in one direction. The danger is that it gets cheesy and out of control.”

In Chapter 4 Lausten chose to emphasise green and amber (“burning rust,” he calls it) in contrast to the blue and magenta that play in JW2 and JW3.

“Colour is integral to the story and each city has its own palette. We drew on colours from street photography in Hong Kong or Japan.”

The production travelled from New York to Osaka, Berlin and Paris. “Whereas Guillermo likes to be in the studio much more, Chad likes to do it for real,” he said.

“We were shooting the Louvre, where there are so many restrictions as you imagine, and I woke up the next day scarcely believing we had been allowed in there.”

A scene set at the famous Sacré-Cœur was scripted for sunrise but was shot night for day lit by giant lamps on cranes of the type that only a big budget can afford.

“The first time we see Bill Skarsgård (who plays Wick’s nemesis, the Marquis de Gramont) it is at sunset and we had to have this feeling of golden light.”

A massive brawl shot at the Kraftwerk nightclub in Berlin was the most challenging scene and required weeks of prelighting and discussion about how colour and light should interact.

Scenes set in Jordan’s Wadi Rum desert and in snow with leaden skies drain the palette and provide contrast with the neon saturation of the rest of the film.

“Some scenes were very green and when I started shooting I got nervous,” Laustsen admitted. “You have to trust your instinct that it’s the right way to do it but I am happy with the result. What we film is the final palette. We’re not changing colour in the DI. Windowing, yes, but the concept of colour is the same from dailies to final movie.”

With Reeves performing most of his own stunts Laustsen deliberately keeps the camera fixed on him, often in long takes. It’s a style of restraint that the filmmakers have learned from Sergio Leone’s Once Upon a Time in the West and The Good The Bad and The Ugly.

“These classical westerns are touchstones for the blocking and the framing of big wide shots,” he said. “A lot of action movies use handheld to give the action more energy or use cuts to hide some of the action but we are not that movie. Chad told me from the start that Keanu is so good I want you do shoot more wide and not go crazy handheld.”

Stunt coordination is Stahelski’s expertise (he was Reeves’ double in The Matrix) and they typically shoot 3-4 takes per fight sequence.

“We try to do as few as possible but he and Chad are perfectionists and won’t stop until it’s right.”

Laustsen changed up the camera package from ARRI Alexa XT with which he shot JW3 to Alexa LF and a set of ALFA anamorphics.

“It was very important to shoot anamorphic since the XT’s larger sensor means the depth of field will be shallower and we want to get powerful close ups of Keanu.”

They carried five LF bodies including an A and B cam, one on a Steadicam another on a crane and another shooting higher frame rates but never rolled with more than two at a time. Almost everything was shot by the main unit; “It’s nice to do everything yourself,” the DP said.

Most cinematographers take still photographs of everyday things that catch their eye but for someone with such a pronounced sense of colour it’s a surprise to learn that Laustsen’s stills are all black and white.

“For me, it’s important to make the most powerful image as possible if that is what the scene suggests, or to make it softer or grittier as appropriate for another scene.

“If we make another movie [a fifth in the series has not officially been greenlit] I would have the same feeling as after ‘3’ when I wondered how it was possible to go further than shooting in a glass house.”

The climactic sequence of Parabellum takes Orson Welles’ The Lady From Shanghai shootout in a hall of mirrors to a whole other level.

“Maybe we go to the North Pole,” he mused, “and everything will be covered in snow and pure white and we go in the opposite direction.”