SVG Europe
article here
World Athletics’ new Virtual Finish Line (VFL) system made
its debut last week, offering host broadcaster HBS a near‑real‑time 3D
reconstruction of the final metres of sprint races. The project, developed by
Sony’s Sports Entertainment Technology Development team, combines optical
tracking, computer vision and rapid 3D rendering to give directors an
additional visual option during live coverage.
At the heart of the system, shown exclusively to SVG
Europe ahead of the event, a 26‑camera optical‑tracking array installed at
gantry height around the track. Each PTZ unit is motorised, synchronised and
network‑controlled, forming a continuous capture line around the stadium. “From
those cameras, we capture the video, and we also capture the motion from video
analysis. With this motion data, we create the visualisation,” explained Sony
engineer Koshino Sota, who oversaw the camera deployment.
The cameras feed into a Sony OB hub, where captured video
and motion vectors are aggregated and converted into 3DCG. HBS receives the
generated graphics in its OB truck, where directors can choose to integrate the
VFL output into replays or analysis sequences.
The system is designed to feel like having “1,000 cameras
everywhere,” Sota said — giving directors multiple angles and perspectives that
would be impossible to achieve with physical cameras alone.
A new workflow for track and field
Hiro Hattori, General Manager of Technology for Sony’s
Sports Entertainment Technology Development team, described the R&D project
as a departure from traditional athletics coverage.
“This is a brand‑new approach for track and field. So that’s
why my R&D team is involved,” he said. “The focus is on finish‑line
reconstruction of sprint races, where margins are smallest and outcomes are
decided in milliseconds. We want to create 3D visualisation footage for
broadcasting, to help fans understand all of the drama.
“Our system captures every motion on the track, mainly
around the finish area. Soon after capturing such motion live, we generate the
3D visualisation for the broadcasting system.”
Development and technical challenges
The proof of concept premiered in Budapest is the result of
18 months R&D and further discussions with World Athletics during the Tokyo
Championships last September. Additional behind closed doors tests were then
made in Japan. Sony already had experience with optical capture for football officiating,
but adapting it to track required new engineering.
“We had two main challenges,” Hattori explained. “We needed
to capture data with absolute precision. Even if the difference of distance [between
athletes] is very, very small, it matters.”
The second challenge was to process the VFL output quickly
enough to be made available like a replay. “If we create this kind of footage one day
after the event there is no value. But there is tremendous value if we can
compress the workflow,” he said.
“Our motion capture is almost live. Within one second we can
capture the motion data. After that we integrate the official timing data from
Seiko and then create the visualisation.”
Currently this takes less than 30 seconds and the aim is to
reduce that further.
The graphic visualisation workflow was developed jointly
with HBS. “This part was mainly based on discussions with HBS, because they are
professionals of sports broadcasting and also of fan engagement,” Hattori said.
Stefan Koch, senior director at HBS, described the process
as a genuine co‑development effort with Sony and World Athletics. “It’s
definitely a joint venture,” he said. “We sat together before World
Championships Tokyo and asked what could we do, or what would make sense? And
then we came together to this idea.”
For Koch, the appeal of the VFL was simply that the finish
is the defining moment of a race, yet traditionally one of the hardest to show
clearly.
“The winner of the 100m is the athlete whose chest cross the
line first. That is the rule. If you
think about it, this actually means a virtual wall. Now, for the first time, we
can actually show this virtual wall in reality,” he said. “What is the most
important moment on track? It is that moment. Now we visualise that.”
Koch sees it as a way to give viewers a clearer, more
intuitive understanding of what happened. “This is a visualisation, a new ‘aha’
moment. Seeing this, fans can understand the result and the performance so much
better. If it’s a tight finish, we will always show the official Photo Finish
because you want to see what the judges saw.”
HBS worked with Sony to define the angles and replay
patterns directors could use. Two main options emerged: a single camera move
from front to diagonal to top, or a sequence of three angles showing the finish
from different perspectives. Koch said the choice depends on the race format:
“In the semi‑finals, the single camera move works best because we have four
[qualifying] athletes to cover. For the Ultimate Champion — the visualisation
could work better by showing just their performance from three angles because
it’s always about who wins.”
Camera control and presets
The camera array is fully remote‑controlled. “Each one has a
motorised head, so we can control it from our video village,” Sota explained.
Sony prepared multiple preset angle patterns — multi‑camera sweeps or single‑camera
arcs — which HBS directors can select per race.
“We show the director a couple of examples - shorter
versions, long versions, multi‑camera and single‑camera. Then they pick, for
every race, what they want for the broadcast. It gives them a lot of
flexibility.”
However the number of cameras trained on the athletes is
less important than their angle. Most of the cameras in Budapest were
positioned towards the finish line, since this is the focus of the first
application though in theory an entire circuit of a race could be tracked and
data-captured for CG visualisation.
“It’s important for the cameras to have line of sight onto
the athletes on the track,” Sota said.
It’s also not yet practical to create 3D visualisations of
races like the 1500m where athletes will bunch together onto a single lane –
partly because the data gathering of movement from individuals would be
obscured, partly because the on-screen presentation could look confusing. In
addition, many longer duration races tend to be more strung out with a clear
winner so there’s less of a compelling case for 3D storytelling at the finish
line.
The motion‑capture pipeline uses computer vision and AI,
trained on general human movement rather than athletics‑specific datasets.
“For this Championship, we are just showing one kind of
visualisation. Soon— maybe after the next Ultimate Championship — we will add
more. You can overlay more data. You can extract more data. It will expand to
another discipline,” Hattori said.
Sony also hopes to migrate parts of the processing pipeline
from GPU-powered workstations into the camera itself, creating a more compact
edge‑processing platform.
The next staging post for the tech could be September 2027
when the World Athletics Championships returns to Beijing 12 years on from the
city’s successful staging of athletics’ flagship event and almost two decades
on from the Beijing Olympic Games.
Sony is an official global partner of the World Athletics
Ultimate Championship and is also providing Hawk-Eye’s video replay service for
officiating.
No comments:
Post a Comment