FEED
Mobile Viewpoint's AI-driven production solution makes broadcasting possible for any sport anywhere.
The evergreen popularity of live sports is part symptom and
part cause of the rapid adoption of video streaming services but while the
demand is huge, only half of the opportunity is being seized. Accelerating
output to keep up with content-hungry sports fans is the next big challenge and
one that automated production systems can help deliver.
“In the same way we have seen costly satellite trucks
replaced by backpack-sized live video transmission units, AI will deliver
similar cost savings to live production and streaming,” says Michel Bais, managing
director, Mobile Viewpoint, developer of a system for sending video over the
internet using mobile networks.
Whether that’s by using AI to analyse audio signals, video
images, people or objects to identify which cameras to switch to and control,
removing the need for an expensive camera crew, or from algorithms that can
generate replays and graphics and then live stream content, AI (augmented or
assisted intelligence, however you describe it) looks set to revolutionise the
industry.
Mobile Viewpoint is not alone in looking to extend its
capabilities beyond the competitive cellular uplinks market. Canada’s Dejero,
for example, has a mobile app to helps bridge the gap between traditional
remote production and cloud production while TVU Networks has also identified
automatic sports logging and production as a related field for its core bonded
cellular units.
The Dutch developer, though, has arguably gone further than
any in co-opting AI into the workflow. It’s worked with research institute TNO
to retune an off-the-peg AI brain from Google’s TensorFlow to fit the patterns,
behaviours and relevant data sets of live streamed football.
The package, branded IQ Sports Producer, comprises two parts
and includes a 32MB dome camera which contains four lenses, the feed from which
is stitched into a 180-degree panoramic field of view. De-warping and stitching
technology provides a corrected image with straight line markings. This image
can be cut out, panned, or zoomed into as directed by the AI-software which is
trained to follow the action as a human director would. All this takes about 30
frames, or 1 second delay.
At its crudest the AI will track groups of players (being
most likely where the action is) but more sophisticated applications can be
taught to anticipate where a ball might be passed to.
The camera itself was originally built by China’s Hikvision
for surveillance purposes but has proved ideal for this solution, not least
because it is relatively cheap (around Euro 5000 per module) and has been
designed to withstand all weather as well as vandalism. Plus, the camera’s 4 x
4K feeds are reliably in sync which, Bais says, is not always the case with
similar cameras.
The second part is hardware containing three Nvidia GPU
boards, the AI software and the mobile connectivity which for Mobile Viewpoint
is a no-brainer. “IP links are our bread and butter,” says Bais.
The AI software further combines motion-tracking with
positional and other biometric data gathered from sensors (RFID,
accelerometers, GPS) worn on special vests by the players themselves.
The first (currently only) system with the 32MP camera in
Europe is installed at AFC Ajax, the biggest club in the Netherlands and part
of the UEFA Champions League. Ajax is using it to film and stream its matches
to club web channel Ajax TV. The footage is also captured to gather data on
player performance analysis during training.
The Ajax training academy can monitor why a player missed a
goal, why they failed to make an assist, and help improve their performance.
Image detection means the AI technology can recognise different players and
follow them, or, detect a ball on a pitch and follow its movements.
“The potential for AI in this respect is huge, especially in
the production of live sports content,” says Bais. “As algorithms develop, AI
can detect faults (yellow or red cards) or injuries as it learns how to make
productions more interesting and story-like. There are plenty of smaller sports
that could use this technology to become content owners in their own right at a
low cost, and then monetise it.
“Only 10 per cent of professional sports are premier or
first leagues. The majority are second or third tier sports many of which are
played at grounds or parks with limited internet connectivity, even in middle
of a city.”
That’s where the bonded cellular links come into play. It’s
3G and 4G capable today, encoded in HEVC H.265, and primed for 5G which telcos
worldwide including local giant KPN will rollout in 2020.
The Netherlands Pro women’s football league, for example, is
keen to install the system at six venues throughout the country as a
budget-friendly means of launching a new online matchday streaming service.
However, to completely automate live sports streaming at the
top level without the need for a production crew and director is perhaps a
bridge too far—for now. AI used for assisted production alongside humans is the
immediate goal. One of the obstacles to fully automated streaming is that it
simply takes time for an algorithm to learn the nuances of what is interesting
or important for each sport. For example, an algorithm may think that capturing
a fight breaking out during a football match is the same as capturing a punch
being thrown during a boxing match—but for the viewer, these are two very
different experiences—one that is normal, and one that isn’t.
“Getting the algorithms up to speed requires time, so there
is still very much a role for humans,” says Bais. “It tends to look for people
walking or running rather than people on the ground so it doesn’t necessarily
zoom in on a player who went down following a tackle which is maybe what the
viewer would like to see.”
Higher image resolutions help too, since the more pixels the
AI has to play with it can, for example, more easily differentiate the ball
from a piece of white paper on the pitch.
While most budget-conscious customers want a single camera
solution, more cameras around the venue are needed to capture different angles
and more data to achieve greater accuracy.
Planned enhancements to the software include AI advertising
insertion, auto playback of replays following a goal, auto summary (highlights)
generation and to train the system on other sports including motorsport and ice
hockey.
It has even created a means of translating the game into 3D
VR graphics within which a user can select the point of view of any player to
analyse decision making during a game. Currently used by Ajax as a training
tool, “our ambition is to bring this to the home too,” says Bais. “With KPN we
are working to bring a layer of interactivity to live streamed games running on
KPN’s standard set-top box.”
No comments:
Post a Comment