AI to revolutionize sports broadcasting

IEC

article here

Artificial intelligence (AI) is breaking the conventional frame of broadcast production and transforming the fundamentals of content delivery and viewers’ experience.

In less than a year the torch will be lit at the Olympic Games in Paris with record breaking audiences expected to watch the competitions online. The International Olympic Committee’s broadcast services arm, OBS (Olympic Broadcasting Services) has been experimenting with new digital technologies and in 2024 promises a visual and data-rich feast.

The previous Olympic games in Tokyo 2020 proved to be a watershed moment in the history of Olympic broadcasting with digital output up 34 % on  the previous ones in Rio.  It is part of a wider trend in which the distribution and consumption of TV is shifting from broadcast TV to streaming over the internet.

With broadcasters reluctant to invest further in the (satellite or internet bandwidth) infrastructure needed to lift picture quality to 4K ultra high definition (UHD) resolution, attention has turned to using innovations in data and artificial intelligence (AI) to bring the sport closer to home.

The IOC’s ambition, unchanged for a decade but thrown into greater urgency during the Covid-affected 2020 Tokyo Games, is to deliver “the most realistic experience one can get from viewing a sporting event without actually attending in person.”

Using the power of data

Data from Paris Olympics venues will be used on screen to inform and engage fans and behind the scenes to speed up content production. Trialled at Tokyo and again at the Winter Olympiad in Beijing, the IOC will expand the use of specialized algorithms – or AI – applied to data in Paris.

This includes an Automatic Media Description system which combines existing data from the TV signal with image recognition based on an athlete’s bib. This will be used to make the process of tagging footage for use in media production much more efficient.

“If we were to tag all our content, it would require an incredible effort and a large number of human operators,” explained OBS Chief Technology Officer, Sotiris Salamouris. “By using trained AI systems, this could be accomplished in a fraction of the time (and cost), while increasing both the speed of searches and the accuracy of the search response.”

The same system can be used to customize content for broadcasters, or even for individual fans, wanting to view the action of a particular country or athlete.

In Paris, AI-based motion tracking technology will help commentators and viewers keep track of the position of the athletes during events like marathon, road cycling, triathlon and canoe sprint. This has proved tricky before given the nature of the course terrain.

IEC Standards can help

ISO/IEC JTC 1/SC 42 is the joint IEC and ISO committee that develops international standards for AI. The committee works on horizontal standards that provide a foundation for developing AI solutions across a wide and diverse set of industries and applications. For example, ISO/IEC 23894 shows users how to manage AI-related risks effectively in order to achieve objectives and improve performance.

Sport climbing returns after making its Olympic debut in Tokyo and to help audiences understand the challenges involved, OBS will use augmented reality technology to overlay live shots of the wall with virtual ones to display information about the holds, angles and routes.

IEC is paving the way for the use of virtual and augmented reality technology in a variety of different fields. IEC TC 110 prepares standards relating to VR viewing, including VR, AR and MR eyewear displays, for instance IEC 63145-1-2. The IEC and ISO have formed a joint subcommittee, SC 24, which works on the standardization of interfaces for information technology based applications relating to computer graphics and virtual reality.

5G, cloud and biometrics combined

Alibaba Cloud is the IOC’s official cloud services partner and will provide cloud-based broadcasting, big data analytics, logistics support and  tracking of athletes’ performance on the field. Its contract runs until 2028 at the Summer Games in Los Angeles.

In Paris, expect this to include biometric data such as the heart rates of competitors in archery. From cameras focussed on the athlete’s face, the system analyzes slight changes of skin colour caused by the contraction of blood vessels. Audiences will be able to chart the adrenaline rush experienced by archers as they shoot their arrow in an on-screen graphic.

Also in Paris, expect greater use of 5G wireless technology to open up more of the same real-time performance data from other sports. While data from sensors can also be gathered by other types of wireless technology, 5G can offer ultra-high data collection speeds with very low latency.

Although the Olympic Games tends to push the technology boundary more than any other live broadcast, similar innovations are being deployed across elite sports.

Other sporting events use AI

At the Wimbledon tennis championship this year, raw ball and player tracking data was ingested into an AI-driven software from which a range of new data points were extracted.

“To date, all tennis data has centred on the first and last hit of a rally, but you could have 42 shots in a rally and lack the ability to capture every shot – until now,” explains James Muir, Wimbledon’s Broadcast Technical Manager.  

“In a game as nuanced as tennis you want to use data to help viewers understand, for instance, why a forehand won the point, or how the tennis champion Alcaraz is able to apply extra spin to a slice backhand,” he adds.

Automation is on the cusp of making significant inroads into sports production. Indeed, the most high-profile events are already benefitting from AI-empowered tools. At the Qatar World Cup, FIFA used semi-automated offside technology in support of match officials to help them make faster, more reproducible, ideally more accurate, decisions.   

Leading soccer clubs are adopting AI/ML as training aids. An AI player tracking software is being used by AFC Ajax to help coaches make snap analysis during training sessions or matches.  

Fully automated sports production, including remote operated cameras and the software to automatically record and cut video, offers huge cost savings in terms of equipment and technicians for sports like US college basketball or second-division Scottish Professional Football which do not have mainstream media rights partners.

AI not quite ready for all applications

Computer vision techniques are not yet smart enough to understand every sports action. Sports with predictable movements, like track cycling, are relatively easy to automate and so too are distinct moments in soccer (like ‘goal’ or ‘red card’ or ‘corner’) for an AI to track, record and playback as a sequence. It is the myriad of other permutations which prove harder to programme. 

“For example, if there’s an injury to a player the AI will have no clue about what’s going on,” says Michel Bais, Chief Product Officer at a US developer of automatic sport production systems. “The AI might instead start filming the goalkeeper because that is where the ball is at that time. After 30 seconds it may realize something is wrong but not know how to correct it.” 

Developers of AI sports production solutions believe that their technology will be used alongside experienced human crews for some time. Familiar production roles will change to accommodate the growing use of AI. An ‘AI production technician’ might be someone who installs the technology onsite, ensuring the data is processed correctly. An ‘AI production director’ might be responsible for supervising the AI generated output of a highlights reel, perhaps adding a more human emotional layer to the presentation.  

“Lots of time is needed to train AI to be good enough to be relied on for top tier programming, so by default, it’s quite expensive,” says Bais. “You’d need an innovator like Sky or Fox Sports to put AI front and centre.”  

During the Tour de France this year, a 3D graphic helped show viewers the gradients faced by the peloton that are often so tough most people couldn’t walk up it let alone cycle it. “This is where the audience expects to go in sport,” said Scott Young, SVP content and production at Warner Bros. Discovery Sports Europe. “They want to be on the field of play. They don’t want to be in the grandstand anymore, from a television viewing perspective. Trying to bring the enormity of what the athletes are actually up against into our studio is the biggest challenge we face.”