Here’s What’ll Happen When AI Gives Consumers Total Content Control

NAB

The rise of hyper-realistic synthetic media may also be accompanied by AI technologies that enable infinite versions of content to be generated and controlled by individual consumers, according to panelists speaking at Cannes Next conference.

article here

Speakers including Hovhannes Avoyan, co-founder and CEO of Picsart, suggest that the border between consumption and creation will end up being very thin, even nonexistent.

“Infinite could be generated and rather than algorithms pushing content to us, individuals will be able to control hyper personalized experiences,” he said as part of the panel discussion, “AI Apocalypse or Revolution? Rethinking Creativity, Content & Cinema in the Age of AI,” hosted by The Hollywood Reporter.

“If you really do come up with personalization of the content, it ultimately could mean that you can watch the same movie a different way that I watch the movie,” Avoyan said. “It means you will be in control of what you’re going to see. You can make your own movie and you don’t even need distribution hubs to get this. You can generate a movie on-the-fly and you don’t even need to go to Netflix to do it.”

Picsart is a suite of online photo and video editing applications. Avoyan said he believes that AI tools like these are not going to replace people but instead will enhance productivity.

“Think about AI is like a co-pilot,” he explained. “It’s a mentor, a helper, an assistant to get jobs done. It can take the most labor-intensive work and let you [get on] with the more fun, cool and creative part — making creativity more affordable and accessible. We can say [generative AI] is democratizing creativity.”

However, whether and how you use AI is up to us, he insists.

“I believe the competition will be between people who are using AI versus those who are not. And people using AI are going to be 10 times more productive, versus people that are not using AI tools. [those with AI skills] are going to be taking the jobs of other people.”

Also on the panel was Anna Bulakh, head of ethics and partnerships at Respeecher, an AI startup behind the voice of Darth Vader in recent editions of the Star Wars universe. She explained how Respeecher created a synthetic voice based on two to 30 minutes of a person’s voice recordings. Respeecher has a library of voices available for use on anything from audiobooks to ads with permissions.

“What it means that we preserve all emotions, intonations and language agnostic, so all languages are covered. It means that your vocal is part of biometric data,” Bulakh said.

The speakers were also keen to label their work as “synthetic media” — that is, digital content created for the creative media industry, as distinct from deepfakes, which have more negative and possibly illegal connotations.

Behind the Scenes: Prehistoric Planet II

IBC

BBC Studios and MPC worked with biologists and palaeontologists to translate scientific research into photoreal VFX and the visual grammar of a blue-chip wildlife doc.

article here

Inspired by BBC Natural History docuseries Planet Earth, Apple TV+’s Prehistoric Planet takes viewers on a safari of the Cretaceous 66 million years ago. The first set of 5 x 40’ won multiple nominations for its docu-like realism in resurrecting dinosaurs and was quickly recommissioned for a second run.

“Season two ups the ante in every way,” said series producer and showrunner Tim Walker (Ice Age Giants) in a promotional trailer. “It’s a wonderful combination of science and art.”

Prehistoric Planet 2 is produced by BBC Studios’ NHU led by Creative Director, Factual Mike Gunton alongside Hollywood’s photo-realist pioneer Jon Favreau as executive producer. Visual effects are by MPC and includes many of the heads of department who previously teamed with Favreau to create The Jungle Book and The Lion King.

Prehistoric Planet II: Hyper realism

“The work that we did on The Lion King itself used BBC NHU docs as a point of reference,” said Elliot Newman, MPC VFX Supervisor. “Now our collaboration with Jon uses techniques learned on The Lion King but fed back into the style of a blue-chip wildlife documentary.”

Shooting on series two overlapped with series one with some environments and assets including the more familiar dinosaurs like Triceratops being able to be reused. However, Prehistoric Planet 2 introduces 57 new dinos and animals and required 1183 shots from MPC (with some additional non-dino VFX by Moonraker VFX in Bristol).

“The main goal was hyper realism, to bring these things back to life,” said Newman. “If you could take a natural history crew back in a time machine to film these creatures in the wild that was the visual style and format we were going for. You’ve got to convince people they are watching a documentary not an animation.”

The illusion is complete with narration from Sir David Attenborough. However, the animators consciously dialled down the CGI perfection and even reined back on the technical advances that are being deployed on the current production of Planet Earth III.

Prehistoric Planet II: VFX extravaganza

“With VFX by default you have a lot of time to finesse and overthink but if the end result looks too polished that defeats what we were trying to achieve,” said Newman.

“The reality of location-based filming is that you don’t always end up getting everything you want. There are so many variables in a narrow shoot window when you’re filming wildlife. You might turn up and the weather is foggy. Maybe you planned to film with a drone but it wasn’t working that day. Perhaps the focus isn’t right on a shot but it is still the one that works best for the story in the edit. We were trying to emulate that lack of perfection as if created in the edit from thousands of hours of rushes not something that looked like it had been orchestrated over many months.”

The process began in previz and creation of 1-minute-long 3D animations of creature behaviour. These featured proximations of the type of environment needed for the scene and a draft version of the assets (dinosaurs and animals).

“They look like a video game,” said Newman. “The creatures are proportionally accurate but not fully detailed. You are not thinking about camera position at this point, just animating behaviour.”

These 3D animated sketches were used by camera teams out on location as a guide to the coverage needed from particular environments in Iceland, Saudi Arabia, Zambia, Montana, New Zealand, Maldives, the Cayman Islands and Sweden.

Given that there were no actual creatures to shoot, the photographic process was more procedural than would occur on a conventional natural history shoot.

The previz were also used to storyboard VFX and shots using virtual cameras by teams at MPC’s London and LA facilities. Episode directors used virtual production techniques, for instance while wearing VR headsets, to move around the scene and work out how best to tell the story. This was done with guidance from BBC wildlife crew expert in how similar animal behaviour would be photographed in real life.

Prehistoric Planet II: Logistics of the Quetzalcoatlus

“Based on the actual logistics of the scene were you actually present we are selecting a camera position and camera movement,” Newman explained. “Would you film this from a drone or on a long lens from a distance? Would you pitch a tripod over there and wait for the animals to come?

“In a T-rex hunting sequence you are not going to be 2ft away. You’d be a long, long way away making sure your crew is safe. The collaboration with BBC Studios is a massively important part of the grammar of making it feel like it was shot as doc.”

For example, the flying predator Hatzegopteryx would be too dangerous to film close-up but when in a mating ritual they are distracted. “You can take advantage of that,” Newman said. “You don’t have to be quite so cautious and it’s a justification for why you can get closer to predators in some situations.”

The series includes episodes set among islands and swamps and oceans. We see a Tyrannosaurus and two Quetzalcoatlus fight over an Alamosaurus carcass; a pack of Imperobator pursue a Morrosaurus onto a frozen lake, a male Beelzebufo attempts to attract a mate and a female Phosphorosaurus hunts lanternfish under the ocean’s moonlit surface.

To virtually film the small crocodile-like Shamosuchus, BBC experts advised that creatures like these have paths that they tend to choose when fleeing from prey. “So you can predict where they are going to run,” said Newman. “We simulated that by putting a camera low down by the bushes, just as you’d use a trap cam in reality, to get really close.”

As for series 1, Jellyfish Pictures (The Book of Boba Fett) had created character packs for each creature. These contained a wealth of information about creature size, skeletal structure, fossil record research and current day species that might exhibit similar characteristics to the dinosaur in question. The packs included 2D concept images on top of musculoskeletal diagrams providing references, for example, of the colour of a velociraptor’s feathers. MPC used this to build the game engine assets in previz and for the final 3D construction.

Prehistoric Planet II: Scalify

The facility rebuilt its feather system to better ‘groom’ feathers and developed ‘Scalify’, a tool that allowed it to quickly add and alter tiny scales to skin.

Newman explains, “Adding hair to characters would be too time-consuming to do hair by hair so we develop tools to define hairstyle, density and length like a sculpting system. We took that logic as the basis for Scalify. It allows you to plot and control curves on a surface. You can paint black and white portions of the skin to dictate that you want very small scales here and larger ones there. It means the workflow is more non-destructive. You could make changes without it causing our artists to go crazy and have to redo a lot of work.”

The production involved close collaboration with the scientific research team led by chief scientific consultant and palaeozoologist Darren Naish.

“We want to portray these creatures in a way that science thinks is accurate and that entailed unlearning, from an animation point of view, some of our preconceived ideas,” Newman said. “It’s not how to make a Triceratops walk in a cool way, it’s an understanding based on science that they walked in a certain way even if it’s not necessarily how you’d animate them.”

Working on the project also seems to have benefitted the science. For the first time, scientists were able to see behaviour for extinct species previously modelled on fossil study or a theory, now visualised in photorealistic animation.

“There was this interesting feedback loop when we had all the scientists and heads of animation in the room,” said Newman. “You’ve still got to adhere to physics. How do you make a 20-tonne Sauropoda feel heavy and big? That’s still the skill of the animator but in terms of the mechanics of saying this is how we believe they behaved and moved required a lot of collaboration and a certain amount of discipline from the animator.”

Prehistoric Planet II: The Devil Frog

Prehistoric Planet II returns to the period as series 1 but when your timeline is 79 million years long (spanning 145 million to 66 million years ago) you have huge scope to play with in the Cretaceous alone.

“There were solid reasons the producers went for that particular point in prehistory since it helps to feature the classics like T-Rex – the golden age of dinosaurs,” Newman said.

He particularly enjoyed recreating a ‘devil frog’ which proved technically tricky because the sequence required close ups of mud and water interaction “which we were very much concerned about our ability to pull off.”

More broadly, though, he is proud of bringing all the project’s constituent parts together given the complexity of content.

“On a typical show you tend to have a lot more continuity,” he said. “In The Lion King you’ve got Simba. You figure out Simba, you’ve got the movie locked in. This was a lot of story vignettes that don’t necessarily relate to each other and a massive number of animals with totally different environments exhibiting never seen before behaviours. Sustaining a high standard of quality was the biggest challenge we faced.”

What’s Behind the $100 Billion+ Creator Economy (Hint: It’s Not Creativity)

NAB

The creator economy is maturing into an industry worth north of $104 billion globally by the end of 2023, according to data compiled in a new report.

article here

There’s been a 314% increase in the number of people earning a living as creators worldwide, up from 50 million in 2021 to 207 million this year, per the “2023 Creator Economy Report” from The Influencer Marketing Factory.

This includes amateur creators whose number has almost tripled in size since 2021, while the demand for creator mentorship and monetization opportunities is “drastically increasing.”

Creator-led brands have emerged as valuable partners for well-known companies, sports teams, leagues, and other entities looking to expand their audience by tapping into the content creators’ niche audience.

“The industry is starting to recognize that creators are businesses,” says Sima Gandhi, Co-Founder, CEO of Creative Juice, one of a dozen industry execs quoted in the report. “We’ve seen that when creators leverage business and tax tools, they can make more, save more, and grow faster.”

TikTok and YouTube are the top favored and top earning platforms at a relatively even rate in 2023. For both TikTok and YouTube, 26% of creators say they are their favorite platforms and 26% say they earn the most on either platform.

The report supports other research that being a full time creator will net you a decent living but by no means one you can retire on. The majority of content creators interviewed for this report make between $50,000 and $100,000 per year. More than 75% of creators earning less than $20,000 annually have less than 150,000 followers.

The report suggests that this may be due to the fact that they are smaller creators just starting out in their creator careers, or they may use influencer marketing and user-generated content as a side hustle.

To start earning $1 million a year, creators will need more than five million followers. Even then, only 4.8% of creators interviewed with more than five million followers reported earning more than $1 million per year in 2023.

Cr: The Influencer Marketing Factory

Avi Gandhi, Founder of Partner with Creators, points to the rise of the “grown up” creator.

“Ten years ago, when you said ‘creator,’ the mental image would be of a young teen or 20-something taking Instagram photos or making YouTube videos,” he says. “Then COVID-19 happened, and every gainfully employed adult in the world was sent home. Since then, hundreds of thousands of professional adults have augmented their incomes or gone full time as creators.”

Being a creator is no longer a young person’s game: “As tools, platforms, and services businesses have arisen to feed relatively new revenue models – like masterminds, coaching, courses and more — small audiences have started to yield large dollars.”

In addition, creators don’t need to appeal to the masses to make a living; they can carve out a niche and find people willing to pay for their content.

“Content creation isn’t just about creativity and entertainment anymore,” Ghandi adds. “Now, more than ever, it’s about utility.”

The report provides an overview of every notable social media platform and creator tool. For instance, it notes that multi-language audio tracking is now available for creators on YouTube.

YouTube reported that dubbed videos made up approximately 15% of watch-time in the channels’ non-primary language.

One of the world’s highest earning creators, Mr. Beast, tells the report that multi-language audio is extremely useful for creators like himself who have several subchannels for content translated to a different language in that their spread of content can be condensed to one main channel.

“You can imagine if you take twelve channels like those and instead of doing them all separate you combine them on one, it supercharges the heck out of the video,” says Mr. Beast.

This in turn makes it easier for global viewers to locate content and simply select their language dubbing preference from the video settings menu. Mr. Beast advises that creators dub their old content as well as their new content so that fans may binge watch your content.

“Just as word-of-mouth marketing has always been the most effective form of advertising, creators harness that power, but at scale,” Brendan Gahan, Partner & Chief Social Officer at Mekanism says in the report. “Individuals are the trusted media outlets.”

He goes onto argue that the relationship creators have with viewers isn’t so much a fan relationship but one more akin to friendship.

“Influencers form powerful, parasocial bonds with their audiences. It’s a one-to-many, scalable friendship. Celebrities may be recognized, but it’s creators who are truly loved and trusted by their communities.”

 

How AI Is Already Democratizing Production and Post

NAB

Generative AI is changing not only the economics and logistics of film production, but also the entire creative process as a whole.

article here

In an insightful discussion held at NAB Show with Pinar Seyhan Demirdag, the co-founder of Seyhan Lee, and Yves Bergquist, director of the AI and blockchain and media project at the Entertainment Technology Center at USC (the think tank funded by all of the major Hollywood studios and tech companies like Google, Microsoft and Amazon) consider the implications. Watch the full session, “Generative AI: Coming Now to a Workflow Near You?” in the video at the top of the page.

Demirdag kicked off the conversation by talking about Cuebric, the AI filmmaking tool developed by Seyhan Lee. “It is to my knowledge, the first virtual production tool that runs on generative AI,” she explains. “It’s a tool that enables filmmakers to go from concept to camera in minutes. Other services that our company offers involve include a generative AI VFX for films and advertising films.”

In developing Cuebric, Dermirdag explained that Seyhan Lee spotted several gaps in the current film and TV workflow that could use an AI boost:

“For example, on a film set, it feels always like everybody’s hurrying up… and then there’s always this constant waiting, everybody waits. Then, in the VFX process, there’s a bunch of repetitive tasks like rotoscoping. I don’t know how many of you rotoscoped in your life but if ever I offer you a solution to push a button, none of you will regret not rotoscoping.

“The third benefit of AI is a barrier with virtual production workflow — it’s a completely new technology and directors of photography especially are terrified of it. 3D real-time engines require an expert to operate them and it’s quite costly.”

There’s another problem that generative AI could help solve, too: the cost of reshoots. Seyhan Lee’s research found that reshoots, even on “healthy movies,” can cost anywhere from 5% to 20% of its budget. “For a medium-size movie, it costs the production $375,000. And for a big-budget movie, it costs up to $24 million for reshoots alone,” Demirdag added.

“How about we make a tool where we can save the production even a fraction of that and bring the mental health of the producers back?”

Bergquist explained that AI is making immediate inroads into enabling faster and cheaper pre-production. The use of generative AI for storyboarding is among the most accessible use cases for filmmakers.

“Down the road a lot of visual production is going to be extremely disrupted [by AI]. The Adobes of this world are already integrating AI into their software,” he said.

Bergquist predicted that many VFX companies will integrate generative AI into their pipelines. Not only would this make content faster, and cheaper, he said, “but it opens up a lot of opportunities for new creatives, riskier content and riskier content formats to be produced.”

Forecasting what might lie beyond automating workflows, he commented that AI would be liberating as well as disruptive for all aspects of moving image storytelling.

“The macro trend is giving individuals extremely powerful tools that used to be reserved for institutions. Information used to be in the monopoly of institutions, now in the hands of everybody with the internet. This is really putting the tools of large-scale high value production into the hands of everybody,” he said.

“We’re heading into a world where everyone including TikTok influencers and Instagram influencers and all sort micro content creators are going to have the tools to make a full anime series, for example. This is not something the entertainment industry is ready for.”

As a neat example, Dermirdag pointed out that that bullet time sequence developed painstakingly from research into photogrammetry by Paul Debevec in 1998, and employed in The Matrix a few years later, “was a megalithic invention” at the time but that now “you can basically have a cheap iPhone, and you get a similar effect as The Matrix.”

Bergquist continued the theme, theorizing that taking the tools of production out of the hands of large, expensive organizations and putting them in the hands of everybody means “we’re at the cusp of just an explosion of creativity. And that’s really, really exciting.”

However, both were in agreement that just having generative AI doesn’t mean it will produce a great story. “If you look at the history of independent cinema — most of independent cinema is just garbage,” Bergquist said.

“There are very, very few independent filmmakers who are very, very talented. So is AI going to take creativity to a completely new level of quality in general, in terms of how deep we reach into the human condition and tell stories that never been told before? Or is it just going to be painting over a lack of talent?”

Dermirdag dubbed this “the danger of normalization of mediocrity.”

The pair then held an interesting discussion about the merits and possibility of AI as a creative force in its own right. Dermirdag argued that much of the fear about using generative AI in the creative arts is because we don’t call it what it is: a tool.

“This tool is great, this tool doesn’t work. This tool will help me, this tool will ameliorate my workflow, this tool is going to help me make more money. This tool is complicated too to understand, but I’m going to read some books and understand it,” she said.

“Unfortunately, our collective subconsciousness is tainted by [negative] stories. 2001: A Space Odyssey Robocop, Terminator, Blade Runner, like we’re all entering forcefully, very fast into a zone that does not serve the elation of humankind. But it is our responsibility as every single human being to research into what AI does do. It’s actually quite simple. There’s a data set and then there’s an algorithm and it produces results in order to serve your creativity.”

Bergquist made the distinction between the craft of production and the decision making that leads to the craft. While AI could vastly improve the efficiency of production, it would allow for humans to make decisions about what to make for audiences.

“As all the crafting part of creativity becomes a lot more automated, a lot faster, a lot more optimized, my question is this. Does the quality of creativity lie in knowing about the craft, or does it just lie in the kind of creative decisions that you’re making?” Bergquist asked.

“Are there elements of knowing about the craft of being trained in the craft versus the decisions that are material to good creativity? How much of quality of how much of knowing about the craft of what aspects of Visual Arts just disappears. If production becomes just the push of a button what impact would this have on art or does telling the algorithm what to make pack all the creative decision making up front? Does knowing about the craft make you a better decision maker?”

These are great questions which will horrify editors and cinematographers and production designers.

Many, including cinematographers and directors, enjoy the creative discussion up front with a script before principal photography as the most inspiring part of the process. It’s where many believe the show is designed and created (editors would disagree), but surely all great filmed art is the outcome of a multiplicity of talented people and of the happenstance of coming together with technology to fulfill a vision.

Demirdag is in no doubt: “Generative AI has nothing to do with creativity. It has everything to do with being your parallel processing, never tiring, just your assistant constantly giving you options for you to curate, review, and select.”

You could probably program an AI to factor in artifacts and deviations from an original idea but would this produce, say, Flowers of the Killer Moon?

It may not be long until we find out.

Hollywood Thinks AI Will Automate Workflows and Allow More Creativity… But Let’s Be Real

NAB

AI will have a profound impact on film and TV production and as its use increases what everyone in town wants to know is: Is my job safe?

article here

The market for AI in video production is expected to grow from $362 million in 2021 to around $1.5 billion by 2028 as tools are adopted across the board to speed workflows and cut production costs.

Owen Harris, the director of Peacock’s AI-focused comedy-drama Mrs. Davis, told TheWrap that Hollywood desperately wants AI to be a “force for good.”

“Certainly, the attraction and the amount of investment we put into technology financially, but also in terms of our own time, reflects that we get something out of it,” he added. “But the question is how much of ourselves are we giving away to get that and is trade-off worth it? At the moment, it feels very much like we’re at the crossroads.”

TheWrap has canvassed the opinion of artists working in various industry roles and of those developing Gen-AI applications and broadly concludes that AI will be widely adopted as a tool, but with humans still in charge of the creative process.

Using AI for previs and storyboarding is widely anticipated to be a benefit to getting productions greenlit and then into photography.

Dan Cobb, CMO at OneDoor, which is developing an AI-generated storyboarding and previsualization tool, told TheWrap, “Instead of making a storyboard that looks like sketches and pen and ink, now we’re going to have storyboards that literally look like the scene from the movie. And they’re going to be so close that once you’re on set, you’re going to try to match it because the quality of that image is the vision of the artist.”

He emphasized that storyboard artists who “stick with the same old marker boards” could risk becoming “irrelevant.”

At the same time, he doesn’t believe that AI will take storyboard artists’ jobs away outright and instead sees the technological advancement as an opportunity to help them do their jobs better.

The argument is that AI will enable storyboard artists to co-conceive movie collaboratively and in real time, “whereas in the past, the artist would have to go back to their closed door, spend a few days drawing, come back and show you another iteration.”

Digital makeup can also be supercharged by AI. VFX studio and AI technology startup Monsters Alien Robots Zombies, for example, has an tool called Vanity AI that it claims enables the delivery of cosmetic VFX 300 times faster than traditional pipelines. It’s apparently been used on 27 major Hollywood productions, shaving 100 weeks off schedules and saving nearly $8 million in costs.

CEO Matt Panousis told TheWrap that far from sucking up manual jobs, Vanity AI will enable better working conditions for VFX artists by slashing the time it takes to complete work.

“So from the artist’s perspective, it’s fantastic because you are increasing their output by orders of magnitude. Like the same artist that could do one shot to maybe only half a shot a day is now pumping out 20, 30 shots a day.”

What about makeup artists? Panousis doesn’t think they’ll be put out of business either.

“There’s always a creative component to makeup and that’s not something our solution handles today. Our solution is there to make cosmetic fixes really, really fast and on a really affordable scale.”

But, in the future, he added, “do you need a world where makeup artists are doing the very simple stuff like getting rid of eye bags, crow’s feet, laugh lines and acne? Maybe not.”

Shows with particularly heavy shot ratios — like reality TV series requiring rapid turnaround — could benefit from AI that gives the editor a jump start in tagging, filtering and even rough assembling rushes.

Although not in use yet for feature film or high-end TV drama, there’s no reason to suppose AI tools won’t become sophisticated enough to the same job. An editor, though, would still use their experience and skill to craft the final work in accordance to script and directorial intent.

Other AI tools at an editor’s disposal today include those for microtasks like matching style and color and auto-tagging.

For sound editing, AI can help quickly scan through large sound effects libraries to create soundscapes, Robert Harari, a music professor at Stevens Institute of Technology, told TheWrap.

“You used to sit there and have to go through thousands and thousands of banks of sounds to try and identify something that’s going to have a certain sonic texture like a harsh sound for a storm versus a quiet rain,” he noted. “It’s helping speed up processes, but at the end of the day, it always comes back to the human curation of what’s right.”

Although films are a collaborative medium many are also “authored” by a director. Is their role under threat?

Director Owen Harris probably speaks for many — perhaps in hope against hope — that films made entirely by AI won’t be films that he or anyone else will want to watch for the simple reason that they will lack humanity.

While AI can be used to help break and communicate ideas visually, he argues that storytelling is still a “very human interaction” that’s going to be “very difficult” for AI to replicate.

“I’d always hope that there will be a human intelligence that can then interact with that and be the bridge back into the creative discussion… but it’s all happening so quickly that it’s really difficult to sort of gauge,” he said. “I’d be amazed if there’s a piece of AI that could tell a story that is absolutely unique to you in a tone of voice that’s absolutely unique to you. Maybe you could mimic a bit of Quentin Tarantino but that’s not the storytelling that I want to make.”

Once thing is for sure; it won’t be just Hollywood’s writers that will be severely impacted by AI. The International Alliance of Theatrical Stage Employees, which has more than 168,000 members from all sides of the entertainment industry, is launching a new study of AI technologies, with a particular focus on how they might reshape jobs under IATSE’s jurisdiction.

It will also consider how contract provisions, legislation, and training programs can be adapted to ensure the fruits of increased productivity through AI are shared equitably among all stakeholders.

 

New Legislation on AI Creation

NAB

As one headline put it, in the battle between artists and AI, the Supreme Court just became Hollywood’s new BFF. That’s because of a recent ruling in a case involving Andy Warhol that experts say has huge implications for the right of human creators to be recognized when their artwork is used by Generative AI.

article here

At the heart of the debate about AI’s impact on creative fields are questions of fair use. Namely, whether AI models trained on copyrighted works are covered, at least in the US, by that doctrine.

With that in mind, it’s worth asking what the Supreme Court ruling in the Andy Warhol Foundation for the Visual Arts Vs. Goldsmith case may mean for AI moving forward. The answer is simple, according to Jonathan Bailey at Plagiarism Today. AI companies should be worried as none of its implications bode well for them.

Christopher Parker at Smithsonian Magazine explains the background to the case. In the 1980s, Warhol created an illustration of the musician Prince for Vanity Fair, based on an existing image by photographer Lynn Goldsmith. The Supreme Court has just ruled that Warhol infringed on Goldsmith’s copyright.

While the court acknowledged that Warhol’s piece was “transformative,” it said that the piece competed directly with Goldsmith’s work and served as a replacement, citing the fact that Vanity Fair licensed the painting rather than Goldsmith’s original photo.

It’s ironic that Warhol is the artist in question here. He played with the idea of mass media by taking existing images (brands of soup cans, photos of Elvis) and reproduced them in the factory style of a Henry Ford.

What’s interesting is what the judges had to say. Seven of them found against Warhol’s estate, saying that to find otherwise “would potentially authorize a range of commercial copying of photographs, to be used for purposes that are substantially the same as those of the originals.”

A judge who disagreed with this verdict argued that it “will impede new art and music and literature. It will thwart the expression of new ideas and the attainment of new knowledge. It will make our world poorer.”

Plagiarism Today itemized what the Warhol ruling may mean for AI. Bailey believes it is inevitable that this case will shift the way future fair use decisions are reached in lower courts.

“Nearly all data that AIs have ingested, including text and image AIs, has been without permission from the original creators. This means that AIs are built on large volumes of copyright-protected material that they are using without permission,” he says.

AI companies have long argued that their use of that source material is allowed because it’s a fair use. Their argument for that, primarily, has been how transformative AI-generated works are.

The ice under that is now thinner. Now, “transformativeness” must be contrasted with other elements, most notably how the new work competes with and/or replaces the original in the marketplace.

As Bailey points out, stock photographers should have little trouble proving that the new works are used to compete with stock photos.

“Things get even worse when you realize AIs often are tasked with producing works that are ‘in the style of’ a particular creator, making works that are designed to directly compete with that artist’s work.”

This will be tested in a number of other lawsuits in the coming months.

The lawsuit from Getty Images against Stability AI, creators of Stable Diffusion, alleges that the company copied 12 million images without permission or compensation “to benefit Stability AI’s commercial interests and to the detriment of the content creators.”

Stability AI, DeviantArt and Midjourney are also being sued by artists alleging that the companies’ use of AI violates the rights of millions of other artists.

Matthew Butterick, a lawyer and computer programmer involved in the lawsuit, told CNBC: “These images were all taken without consent, without compensation. There’s no attribution or credit given to the artists.”

Butterick is involved in another class-action against Microsoft, GitHub (which is owned by Microsoft), and OpenAI (in which Microsoft is a major investor) alleging that GitHub’s Copilot system, which suggests lines of code for programmers, does not comply with terms of licensing.

To be clear, this doesn’t mean that those suing AI companies are a lock to win. “The issue is still complicated, and the Supreme Court made it clear that ‘tranformativeness’ is still very much a factor and AI companies still have arguments in their favor,” says Bailey.

“However, the argument that AI companies have largely based their businesses around has been severely weakened, and that should give them pause.”

Indeed, pending litigation has prompted Gen-AI developers to shore up their defence.

As reported on CNBC, OpenAI CEO Sam Altman has said stock footage library Shutterstock was “critical to the training” of its generative AI image and art platform DALL-E — and has set up a contributors fund. It compensates content creators if their IP is used in the development of generative AI models. Moreover, creators will receive royalties if new content produced by Shutterstock’s AI generator includes their work. Contributors can opt out and exclude their content from any future datasets.

Microsoft said content generated by ChatGPT and embedded in Microsoft 365 apps will be clearly labeled, encouraging users to review, fact-check and adjust. It will also make citations and sources easily accessible by linking to an email or document the content is based on or a citation when a user hovers over it.

Google said Bard, the company’s competitor to ChatGPT and being embedded within Gmail and Google Docs “is intended to generate original content and not replicate existing content at length.”

How virtual reality is changing medical care

IEC 

The global market for augmented reality (AR) and virtual reality (VR) in healthcare is expected to reach USD 14 billion by 2030, growing at a rate of 21,5% a year, according to market research and strategy consulting company Emergen Research.

article here

The rising need for better medical training is one of the major factors driving this industry. For clinicians and patients, AR/VR technologies provide a more immersive and participatory experience that can improve results. “Training and simulation are a major part of what hospitals are looking to do in order to overcome challenges in staff turnover and to find a better way to train surgical teams,” explains Maayan Wenderow, VP of Marketing at an immersive surgical training provider, which is accredited by the American Association of Orthopaedic Surgeons and the Royal College of Surgeons of England.

Analysts at Frost & Sullivan say that VR accelerates skills acquisition and medical product adoption “by smoothening the learning curve”.  Fairly common is the use of VR to review cases before a surgical procedure. Dr Rafael Grossmann, a full-time general, trauma, advanced laparoscopic and robotic surgeon who practices in the US explains, “You can view images of a patient’s specific organs in VR and navigate the map you are going to cross. If your VR system has haptics, you also get some muscle memory to take into the operating room.”

Surgical simulations have been used by surgeons for decades, but VR goes one step further than practicing on mannequins or human cadavers. Kinesthetic haptics, such as those developed by Wenderow’s company, mimic the physical touch of surgical actions, and accurately simulate the sensations of bone textures, muscle and soft tissue. “You can practice certain operations with haptics such as drilling the bone or injecting the eye and master it before moving onto a human,” Grossmann explains.

VR is also being introduced to help patients understand the procedures they will undergo. “VR is a way to enhance the process of informed consent,” says Grossmann. “Patients are less afraid if you can show them in VR what you are going to do.”

Great potential for relieving pain

VR/AR is also being widely used in certain forms of treatment. Trials at Chelsea and Westminster Hospital NHS Foundation Trust report particularly good results for VR applications in reducing anxiety and pain experienced by children attending accident and emergency units. “We also saw an improvement in patients’ sleep quality in the Intensive Care Unit, and positive effects on anxiety, stress and pain in cardiac patients,” reports Tom Carlisle, digital innovation fellow at CW+, the charity of Chelsea and Westminster Hospital NHS Foundation Trust.

The hospital is extending the use of VR as a distraction tool and an alternative to pain relief. For example, headsets are being used to support women who experience loss in the early stages of pregnancy: they can observe a relaxing scene or follow a guided mediation while undergoing a manual vacuum aspiration procedure. This can reduce anxiety. “There is so much potential,” Carlisle says. “We can imagine a world where VR could be used for oncology patients in the Chemotherapy Day Unit as a form of distraction and pain relief. AR could be used to educate patients and provide faster and more effective remote diagnosis.”

Simulated environments can help people overcome social anxiety disorders. One such therapy is delivered to patients through weekly half-hour sessions. Each user puts on a VR headset and is greeted by a virtual coach who asks them to complete a range of tasks, such as travelling on a bus, buying groceries or going to a doctor’s office. These kinds of situations are common triggers for people with social anxiety. “The immersive nature of VR provides a powerful new way to engage users and helps them to regain confidence, feel safe and overcome trigger situations,” says Director of Clinical Partnerships, June Dent.

Social engagement therapy is being deployed by the UK’s NHS while several private UK healthcare providers are also using the service. Other VR-based clinical trials include treatments for those with a fear of heights and for treating post-traumatic stress disorder (PTSD), depression and obsessive compulsive disorder (OCD) .

Perhaps the most important aspect of this approach is that the therapy is entirely automated, meaning it doesn’t require the presence of a qualified clinician – the session can be delivered by a staff member who has been trained to set up the VR headset.

Pioneering surgery using AR

Neurosurgeons at Johns Hopkins performed one of the first augmented reality surgeries on living patients in 2020. “When using AR in the operating room, it’s like having a GPS navigator in front of your eyes in a natural way so you don’t have to look at a separate screen to see your patient’s CT scan,” says Timothy Witham, M.D., Director of the Johns Hopkins Neurosurgery Spinal Fusion Laboratory who led the spinal fusion surgery.

A London-based startup recently launched a lightweight, wearable ‘smart glasses’ device that allows surgeons to share a ‘first-person’ perspective of open surgeries and minimally invasive procedures. The first-person perspective allows those observing remotely to have a far clearer picture of the surgery, providing opportunities to advise or learn from surgical techniques. 

According to the company, pilot procedures with surgeons wearing its AR glasses were conducted in the UK and US last year for colorectal, otolaryngology, orthopaedic and urology surgeries. “The fact that it’s lightweight and intuitive makes it easy and convenient for anyone to deploy in surgery, even for the first time,” says Stella Vig, Consultant General and vascular surgeon, Croydon University Hospitals NHS Trust. “Healthcare overall is trying to figure out how to do more with less, and technology like this gives us new ways of delivering care.”

According to Emergen Research, the market is confronted with several obstacles, including the still high cost of AR and VR equipment, the lack of technological know-how among healthcare professionals, and worries about data security and privacy. Additionally, regulatory barriers and a lack of standardization in AR and VR healthcare applications can restrain market expansion.

That is where IEC work comes in. Several IEC Technical Committees are developing standards for AR and VR devices.  IEC TC 100 prepares standards for audio, video and multimedia devices. It aims to standardize forward looking technology areas like haptics or even the Metaverse. It has recently published a technical report to clarify the conceptual model of haptics in multimedia systems and has set up a group to prepare standards for multimedia equipment relating to the Metaverse.

IEC TC 110 publishes standards for electronic displays. One of its working groups has developed the first edition of IEC 63145-20-20, which establishes the measurement conditions for determining the image quality of eyewear displays. IEC has also formed a joint technical committee with ISO, JTC 1, which prepares standards for information technology. One of its subcommittees publishes documents which specify the requirements for AR and VR. 

With the correct regulatory framework and appropriate standards in place, AR and VR technologies have virtually unlimited potential in the field of healthcare.