Monday, 10 December 2018

Behind the scenes: Mary Poppins Returns


IBC
Cinematographer Dion Beebe explains how he mixes digital production techniques with hand drawn animation and London location work to recreate Disney’s whimsical caretaker.
The blending of animated penguins alongside actors Julie Andrews and Dick van Dyke in the 1964 classic Mary Poppins was as ground breaking technically as it was charming but could Disney’s sequel capture the same magic more than fifty years on?
Devising a style which paid homage to the original but also updated the story with a visually fresh take was the chief challenge facing director Rob Marshall and cinematographer Dion Beebe ACS ASC in preparation for Mary Poppins Returns.
“Going into the movie there was a lot of discussion about the long gap between the two films,” explains Beebe. “There is a lot of nostalgia for the original and love for the main character but we are also looking to reach a new audience. The audio-visual language of cinema has shifted a lot over the decades and an old-style movie wouldn’t necessarily appeal to today’s kids.”
This is Marshall and Beebe’s fifth feature collaboration and their fourth musical project following Chicago (2002), Nine (2009) and Into the Woods (2014). Beebe was Oscar nominated for Chicago and won for Memoirs of a Geisha in 2005, directed by Marshall.
“The funny thing was that in tests when we pushed the visuals too much in a stylistic way to try and modernise the look, it didn’t resonate,” says Beebe. “It didn’t feel true to the character and period somehow.”
The new film takes place 25 years after the original in the 1930s and finds the Banks family, which includes a grown-up Jane and Michael (Emily Mortimer and Ben Whishaw, respectively) as well as a new generation of children, struggling to cope with the loss of their mother. Whereas the original was filmed in gloriously eye-popping Technicolor, the sequel has a more nuanced palette.
“We enter the world of the film after the big crash of 1929, it’s Depression era London and the story revolves around their house being repossessed,” he explains. “The bold primary colours have faded a little to a cooler shade. But on Mary’s arrival things change. [Emily Blunt]’s Poppins introduces vivid colour into the picture as the kids start out on a fantastical journey of the imagination.”
Beebe shot multi-cam with two ARRI Alexa XTs and an Alexa Mini paired with Panavision G Series anamorphic primes as a nod towards the original.
“The optics lent the film a certain nostalgic look,” he says. He deployed ARRI’s Skypanel LEDs rather than hefty theatrical lights as the best way to control lighting cues and mood changes on location and in studio.
 “Digital is a fantastic capture medium because it gives you an ability to explore looks in post but just because of all those choices it’s even more critical for me to find the look of a movie in pre-production,” he says.
“Testing is really key. I want to create the look in test to use on set while shooting.
This is important for production design in terms of how it affects the colour of walls, or for costume design in their choice of texture and fabric. Everyone can respond to the look of a film if they get a chance to see those tests in pre-production.”
The heart of the film is the songs, several of which pay homage to the original. For a reworking of the ‘Jolly Holiday’ song and penguin scene Marshall lured a number of Walt Disney animators out of retirement to create a number which mixed old style hand-drawn animations with live action.
“You’d be on set in discussions about the scene and one of the animators would grab a pen and paper and sketch a character,” explains Beebe.
“Then you’d use this to visualise the sequence understanding, for example, that an animated character is only 2 ft tall so you have to ask the actors to relate to that. Rob began the sequence by choreographing the actors and in production we filmed them against green screen. [Editor] Wyatt Smith fine-tuned the footage and then the animators took that and we worked in their designs.
The whole scene was an elaborate visual storyboard in which rehearsals, recorded songs, editing and animation was iterated again and again.”
Another scene has actors travelling in a hand-drawn animated carriage. “We rigged a mechanical device that our actors could sit on which would provide various physical motions and shot that against green screen.”
A set-piece action sequence around Big Ben and through central London was story boarded to gauge how much of the set would be physical, how much a VFX extension.
For the song ‘Stuff and Nonsense’, in which the characters swim underwater and is rendered in a totally CG environment, the scene was pre-visualised almost to the frame.
The more traditional song and dance numbers were evolved during rehearsals by Marshall and his cinematographer.
“I film the rehearsals and then talk with Rob about camera placement and specific shots,” says Beebe. “We’ve worked this way on musicals since Chicago.”
The set for Cherry Tree Lane was built on stage H, the largest soundstage at Shepperton along with sections of Big Ben, created by production designer John Myhre. A number of actual London street locations including around Bank and Threadneedle Street were also incorporated and used for scene transitions and the opening number ‘Underneath The Lovely London Sky’.
“Rob’s mantra when it comes to the musical genre is that you have to earn the song. You can’t just break into song. It’s got to flow out of the emotional content of the movie so it becomes part of the storytelling not just a musical interlude. The trick is to find ways of making the transitions into musical numbers so that it doesn’t feel forced.”
The most challenging sequence in this regard was for the song ‘Trip A Little Light Fantastic’, the sequel’s version of ‘Step In Time,’ the chimney sweep song from the original.
“The kids are walking the streets of central London and get lost in fog before finding Jack, the lamp lighter (Lin-Manuel Miranda), whose job it is to light the street’s gas lamps,” Beebe explains. “We begin this slow transition into what will become the film’s biggest dance number involving 40-50 dancers by devising ways in which he would turn on the lights by seemingly pointing at one lamp then another. So, we start from a real street location and as the scene builds we blend into the stage shoot and into a more fantastical landscape.”
Marshall’s affinity with the musical stems from his own background as a Broadway dancer and stage producer.
“He understands how to represent a show, says Beebe who says the director pays great attention to cast rehearsals of scene and song commencing six weeks before principal photography. “He understands how to frame a scene from his background in visual theatre. Movement is crucial in terms of both the actors and the camera.
“Ultimately, filmmaking is about choreography. When I move a camera or when he directs an action or for the actors everything is a bit of a dance.”


Friday, 7 December 2018

The major tech trends of 2018

Broadcast


The shift to IP took a big step forward this year, while broadcasters found a use for AI in everything from subtitling to assembling highlights packages.

The majority of artificial intelligence (AI)/machine learning (ML) applications to date have been proof-of-concept, but there is little doubt that the technology will ultimately have an impact on virtually all parts of the content creation and broadcast chain.
“Almost every media business is focused on speed, agility and intelligence,” says Digital Production Partner ship (DPP) managing director Mark Harrison.
“AI – or whatever term people use for process automation that exploits complex data faster and better than humans – will start to become a commodity business tool. It will be interesting to see what new partnerships and relationships form around this, and what the unforeseen consequences might be – there are always some.”
It’s clear that AI/ML is not an off-the-shelf solution from Microsoft or IBM but one that needs to be tailored to specific broadcaster databases and workflows. That requires expertise, time and expense – something the European Broadcasting Union (EBU) aims to tackle in 2019.
“You need a critical mass if you want to deploy AI/ML,” says EBU director of technology and innovation Antonio Arcidiacono. “We want to unite the resources of our members to develop applications together.”
One example is speech-to-text. “Very often, you find the best AI solutions for this are in American English, but they won’t necessarily be the best for Dutch or German,” says Arcidiacono.
“Since broadcasters’ resources are often limited, we can cross-fertilise their teams’ efforts to solve AI applications – dealing, for instance, with archive or the newsroom.”
According to international media and tech trade body IABM’s Buying Trends survey, AI/ML adoption in the broadcast and media industry leaped from 2% to 13% in the six months to 30 September, with another 68% of those surveyed vowing to deploy it over the next two to three years.
“Imagine a scenario where you drop in a piece of content and specify whether and how you want it dubbed or subtitled in any language you choose,” says Rian Bester, chief executive of online broadcaster Insight TV.
“There’s no need for scripts since the machine will recognise metadata on the fly. Finer aspects like tonality and context will be considered. This will make a very big difference to the speed of versioning.”
This year, IBM’s AI technology Watson was used at the Wimbledon tennis tournament to automate the tagging and assembly of two-minute highlight reels for online publication; Endemol Shine Group used a Microsoft AI to speed up production of the Spanish version of Big Brother; and the BBC experimented with AI/ML to mine its archive and help schedule content for BBC4.
However, BBC4 editor Cassian Harrison stressed in a blog: “Humans have nuance, taste and judgement. These are qualities no algorithm or machine can replace.”
Discovery’s Eurosport plans to use ML to produce personalised sports content but executive vice-president and managing director of digital Ralph Rivera told the IBC conference in September that the company wants to “balance the algorithm with editorial curation”.
The risk is that if viewers are only served what they think they want, they will miss out on all the other content that Discovery wants subscribers to consume.
This was the year Reuters revealed that almost three-quarters of news publishers surveyed were already using AI to optimise marketing, automate fact-checking or speed up tagging and metadata.
China’s state press agency Xinhua took this to its logical conclusion by unveiling a ‘photorealistic’ AI newsreader on Chinese television (speaking Mandarin and English).
Either as an aid to composition or as an automated soundtrack generator, AI is already a force to be reckoned with in the music industry, with ‘functional music’ – for everything from adverts to stock music for TV – considered the prime market for AI-produced compositions.
Experimental film Zone Out, released in June, was scripted, directed and edited entirely by an AI called Benjamin, with its score generated by algorithms.
“We see AI as a tool to augment creativity,” Nathan Leong, director of channel strategy at production music library Audio Network, told Broadcast in October. “But we can’t ever imagine a scenario where an algorithm would replace a composer.”
IP breakthrough
Two very different conversations about internet protocol (IP) are taking place. One is about IP as a means of moving signals in live production; the other concerns IP as a means of moving files. In the former, there’s been significant progress in the definition of common standards and specifications.
The latter is more about the non-live environment – and particularly the consumer demand for streamed video, often on the move.
“Where they come together is that consumers are now expecting broadcast-level quality of service around streamed, mobile video,” says the DPP’s Harrison.
“The biggest impact of IP this year has been that broadcast and IT professionals have come together around that challenge.”
The finalisation of the SMPTE 2110 suite of standards in May means that every element that has been part of the traditional SDI studio can now be put into an IP studio. This has given impetus to IP technology adoption, according to IABM chief executive Peter White.
“The transition to IP is a gradual one and it will take years to achieve full deployment since it’s as much about transforming culture as technology change,” he says.
The EBU calls SMPTE 2110 a “promising technology” but cautions that more work needs to be done. “We’re in the process of going from a standard on paper into the interoperability of equipment that needs to exist in practice,” says Arcidiacono. “We probably need to look three to five years out before it goes mainstream.”
The impact of SMPTE 2110 is already being felt in remote live production. Leading the way is Australia’s Andrews Production Hub in Sydney, which opened in March. Fox Sports is using it to produce Aussie rules football and other sports from venues 4,000km away.
Service supplier NEP also deployed IP technologies at Wimbledon this year as a replacement for traditional SDI routers, in a system that was able to support UHD production.
While projects like these use dedicated fibre to send signals to the production centre, technical advances are opening up live remote coverage of lower-budget, second-tier events.
SRT, an open-source protocol developed by streaming specialist Haivision, goes some way towards ironing out the glitches in sending video over the public internet, a point recognised with the award of a technical Emmy.
Sony’s launch of a cloud-based production switcher this year – part of its Virtual Production system – is a step towards replacing complex and typically expensive outside broadcast (OB) hardware with software apps. Since cameras are the only physical kit required, it alters the business model for OBs too, as a producer need only pay per use.
IP is also being adopted to enable virtualised playout and all the benefits this delivers in terms of channel launch agility. Red Bee Media’s debut of a completely IP and software-based system in 2019, claimed as a world first, will be watched closely. The BBC is moving this way too.
Opening in 2019, regional hub BBC Cymru Wales will be based around SMPTE 2110 and is seen as the template for the corporation’s future development.
Yet the ability for millions of users to live-stream concurrently remains an issue. The rollout of Formula 1 streaming service F1 TV got off to a stuttering start in May when “live playback issues” resulted in some viewers being unable to watch the race.
And Australian telco Optus had to hand its rights to the Fifa World Cup over to free-to-air broadcaster SBS in July after subscribers reported being unable to watch games.
However, Nick Moreno, director of strategy for satellite and media at communications infrastructure company Arqiva, says: “The odd stories about streaming failures for live events are outliers.
“Though clearly serious for the rights holders involved, these are noteworthy events only because they are now relatively rare. Multiple millions of streams are delivered every day without hitch, but this doesn’t make headlines.”
Latency, the delay incurred when streaming live –and noticeable in contrast to the speed of satellite broadcasting – has long been an OTT bugbear but one the BBC’s estimable R&D team may have cracked.
In a prototype for which it won an award at IBC, BBC R&D showed that it could eliminate the delay by reducing the duration of each segment of video file sent over the network.
“The biggest bottleneck to streaming UHD is bandwidth to the home and the content delivery cost,” says Insight’s Bester. “You need the scale of Netflix to make it work, but that’s rapidly changing.”
In part, that’s because 5G is on the horizon. Bester predicts a battle in 2019 between traditional cable and satellite providers and rival mobile network technology providers delivering premium content to consumers, as SVoD competition hots up with the arrival of Disney+.
Elsewhere, Ronen Artman, vice-president of marketing at US-based portable live video solutions acquisitions company LiveU, predicts continued growth of users streaming their real-life experiences online.
“YouTube may have begun the trend,” he says, “but Twitch TV is now reaching beyond its eSports/gaming community to embrace artists, musicians, podcasters and comedians.”


Raw production takes off

IBC


IBC365 takes an in-depth look at an increasingly popular choice for acquisition, with new methods trading cumbersome, data heavy workflows for virtually identical image fidelity but at lower data rates and file sizes.
Choice of camera and shooting format are intricately linked to post production, but getting from A to B with the best possible picture is fraught with decisions and trade-offs.
“In general, no matter what you choose, you will take a hit,” says Avid’s director of provideo marketing Matt Feury. “There is no best way, just the best way for your particular project. Do you have more time than storage? More money than time? The key is production agility. Depending on the type of programme there will be discussions between production and post regarding camera/codec options and their impact on storage/costs.”
All compression algorithms work to make files smaller while retaining resolution, colours and dynamic range to the best of its ability. An increasingly popular option for cinematographers, and one becoming available on a wider variety of cameras, is to record the Raw camera sensor data.
Sometimes referred to as ‘digital negatives’, Raw image files carry a far wider dynamic range and colour tonality from which to construct an image in post.
Lightly if processed at all, Raw files don’t bake in parameters like white balance and the images tend to be in native pixel count. The result provides greater flexibility for the creative process.
Proprietary codecs designed to get the most out of a camera manufacturer’s particular imaging sensor include ARRIRAW, Redcode Raw and Canon Cinema Raw. Sony offers 16-bit Raw out of its flagship cine camera Venice. Both the Panasonic VariCam line-up and EVA1 can output Raw data to external recorders.
Red cameras use the proprietary Redcode format which can be selected at recording ratios that range from 3:1 to 18:1. The closer the ratio gets to 1:1, the lower the compression, the higher the quality of the image but the higher the data rate.
While some cameras can only record one format (the ARRI Alexa 65 only captures ARRIRAW) most models offer several different flavours for recording video.
Choice of codec
The most popular two for broadcast TV acquisition are the H.264-based Apple ProRes, offered in flavours such as 444 XQ, 444, 422 HQ; and AVC with variants such as XAVC used by Sony, XF-AVC used by Canon or AVC-Ultra in the case of Panasonic’s cine camera range. For Panasonic users wanting to work in 4K resolution AVC-Intra Class 4:4:4 is offered on the VariCam, while on the EVA1 there’s AVC-Intra Class 4:2:2. Another codec, Adobe Cinema DNG, is available in several cameras including Blackmagic Design’s URSA Mini Pro.
“Choice of codec and data rate will play the biggest hand in dictating quality,” says Jai Cave, head of operations at Soho facility Envy.
For example, footage shot at 50Mbps data rate using an MPEG2 codec (HD422) is the minimum acceptable for UK broadcast. High end drama destined for Netflix, Amazon or for domestic broadcasters like Sky will require a higher spec, usually delivered at 4K and increasingly with an HDR grade.
That tends to require the best colour and dynamic range you can generate out of the sensor which is why Raw is an attractive option.
“The benefit of working with Raw is that you’ve got full control of what can be extracted from the original file,” explains Francisco Lima, visual effects technology supervisor, Gramercy Park Studios. “You are reassured that there’s not been any extra processing that could degrade the footage and cause issues when grading and performing VFX.”
If the project requires minimal colour-correction and no VFX, or is destined for the web, then you can probably get away with lower bit-depth, chroma subsampling, and macro blocking that come with lower-quality capture codecs.
“If a production needs no, or minimal, grading and/or a quick turnaround, a camera like Canon’s XF705, with its direct HDR recording capability, would be a great choice,” suggests Paul Atkinson, pro video product specialist, Canon Europe.
“Certainly, if your production is really effects-heavy, then you should be filming in a full 4:4:4 colour space since chroma key requires super-accurate colour information,” confirms Barry Bassett, managing director at camera rental shop VMI. “If you are planning on mastering with an HDR version, then you will require recording in Log format and for this shooting Raw is ideal.”
The Raw downsides
In spite of the image-quality benefits of Raw, there are two major downsides: larger file sizes and extra processing requirements. That means more memory cards, more hard drives, and more time spent copying files. Raw video also takes significantly more processing power in order to view, edit, or transcode.
To pass them through a post workflow without incurring massive storage costs or bottleneck delays in throughput, Raw files typically need transcoding (converting) into proxies for editing before being replaced with the original footage for effects and mastering. For example, Panasonic offers a proxy compression method within AVC-Ultra which records a low-rate, high-resolution proxy video for a straightforward edit file or streaming purposes.
“Working with smaller resolution, data rate and file size will allow the editor to work faster as the transcoded files are lighter, especially in jobs where the editor is having to cut and overlay multiple videos,” says Lima.
Cave advises, “You want to try and keep the files native for as long as you can to keep as much of the original quality as possible.”
Apple and Blackmagic Raw
And that’s where the latest Raw innovations from Blackmagic Design and Apple come in. Last April Apple released a Raw version of ProRes, and last month camera maker Blackmagic announced its own version of Raw in direct competition to ProRes (though Blackmagic Design say that they began work on their Raw codec several years ago and its 2018 release with ProRes Raw is coincidental).
What’s different about both is that they’ve worked out a way of retaining all (or most) of the quality of Raw but in smaller file sizes associated with compressed formats.
Some manufacturers have already gone this route but kept it within their own product range. Sony devised X-OCN, which features in the Venice camera, to retain everything the sensor sees but at lower bit rates; a Light version of Canon Raw (available on the EOS C200) records internally to CFast cards offering a reduction in file size, while still allowing for extensive grading possibilities.
Apple claims ProRes Raw (which comes in an even higher quality HQ version) offers superior performance in both playback and rendering to codecs like Redcode Raw and Canon Cinema Raw and supported this claim with benchmarks tests, albeit only within Apple Final Cut Pro.
With most video codecs, including ProRes, a technique known as rate control is used to dynamically adjust compression to meet a target data rate. This means that, in practice, the amount of compression varies from frame to frame depending on the image content. In contrast, Apple designed ProRes Raw to “maintain constant quality and pristine image fidelity” for all frames. As a result, images with greater detail or sensor noise are encoded at higher data rates and produce larger file sizes. Nonetheless, it has managed to get ProRes Raw’s data rates down to less than that of its previous next best codec ProRes 4444.
“ProRes Raw does have much smaller file sizes than even ProRes 4444 so this is cheaper to shoot (less card storage); has less media to move (both for copying to hard drive and also for cloning); less data to move in post; store; less render time etc,” says Bassett.
“ProRes is not as efficient as ProRes Raw, but remains popular and reassuring when production teams do not want to take a risk with their work,” says Oliver Newland, regional marketing manager, Panasonic UK.
Blackmagic’s solution to the problem of processing power requirements is to move part of the process into the camera, which is able to provide hardware-based acceleration. By performing this in-camera, software like Blackmagic’s DaVinci Resolve won’t need to work nearly as hard to decode the files, the firm says.

There are two variable bit-rate levels of ProRes Raw being introduced. ProRes Raw HQ and ProRes Raw record at similar data rates as their ProRes counterparts as seen left.
This efficiency allows Raw data to be stored with similar memory space as common video files, meaning that you can edit and colour natively in Raw on FCPX on a MacBook Pro. ProRes Raw files will also output from FCPX to video finishing formats faster than other Raw formats.
Raw on a laptop
Both Apple and Blackmagic’s Raw innovations promise to speed the transfer of data files from camera through post but they also have the potential to eliminate the need for proxies. Raw editing on a laptop could soon become a practical reality. Working with, storing, and coming back to projects using ProRes or Blackmagic Raw should be seamless.
Kees Van Oostrum, president of the American Society of Cinematographers, even claims that Blackmagic’s tech “could entirely change the workflow going from camera through post production… because the editorial team can work with the camera original files, which are fast enough to use for everyday editing.”
A number of products already support recording of ProRes Raw including Atomos, maker of the Shogun Inferno recorder/monitor and DJI’s Zenmuse X7 camera and Inspire 2 drone. Currently, though, the codec is reserved for editing and finishing on Final Cut, isolating non-Mac OS users for the time being.
Blackmagic Design Raw, currently in beta and compatible only with its URSA Mini Pro, is - like ProRes - designed to work with third party equipment via an SDK (Apple also requires vendors pay a licence). By NAB next April you can expect more product such as recorders and NLEs to be able to decode and play files in both codecs.
Raw alternatives
These are far from the only Raw deals in town. ARRIRAW is a proprietary, uncompressed and unencrypted Raw format, which can be recorded and reviewed on the camera directly and processed with the ARRIRAW SDK.
“Because ARRIRAW is not encrypted, it is open for image processing with third party tools, such as Davinci Resolve or Colorfront OSD, which offer the ARRI processing chain as well as their own,” says Henning Raedlein, head of digital workflow solutions at ARRI.
There are also mathematically lossless schemes available for ARRI cameras, such as Codex HDE, which reduce the data footprint “with bit-identical results”.
Earlier this year, Red released an updated version of its image processing pipeline (IPP2) which is a Red-specific means of managing the RAW sensor data and notably the colorimetry to the final image output. A particular goal in IPP2 is to support HDR with new on-set HDR monitoring controls.
The new VariCam Pure meanwhile is a camera system designed by both Panasonic and Codex to capture 4K Raw at up to 120fps via Codex’s Production Suite. In turn, this delivers ProRes in addition to all other required deliverables. This system was used by Vanja Cernjul, ASC to film the feature Crazy Rich Asians.HEVC for broadcast shooting
Panasonic has also brought out an AVC-Intra LT codec for the VariCam LT that permits recording of 4K 422 10-bit at 50p or 60p with the same bitrate as 25p or 30p. “This is a smart codec that partially compresses before the de-bayering process when the data amount is still low,” explains Newland.
The move to 4K UHD requires an even more careful balance of output versus budget for post, storage and archive. A solution to this is to use HEVC, the MPEG standard which produces files with greater compression than the current standard (H.264), but enables 4K footage to be captured at high bitrates on easily available recording media such as SD cards.
Canon is first out of the hatch with the first pro-camcorder capable of working with this codec. The XF705, launched September includes the XF-HEVC format as an option. According to Canon, it can record 4K UHD 50 frames a second at around 160Mbps as 4:2:2 10-bit files directly to SD cards. The file is wrapped for post with the widely used Material eXchange Format (MXF).
Other manufacturers are expected to follow suit. “In future, we expect to see the entire industry moving towards codecs that offer efficiency compression, and improved codec algorithms to further reduce the codec data rates, especially with H265 technology,” says Newland. “This will provide the same quality for less bit-rate than H264.”
Individual preference
Selecting one codec over another for acquisition can also depend on individual preference.
Mark Toia, a director and DP of commercials for clients like Virgin Airlines and Coca-Cola uses Red cameras almost exclusively on top end jobs. “I’ll use the 8K sensor all the time now but I’ll shoot 11:1 in Red’s compressed format unless I’m using lots of VFX for keying and green screen in which case I’ll go 3:1 or 5:1.
He doesn’t set LUTs (a colour correction preset in-camera) or worry about an under exposed shot. “All I am after is the angle and the content because I can work with the Raw and pull out any detail I want after the fact,” he says.
By contrast, director John Mathieson, BSC (X-Men: First Class, Mary Queen of Scots) dislikes the whole concept of Raw, finding that the images tend to be “confused, washed out and looking indistinguishable from any other film.” He urges film school graduates to be taught how to light and expose a picture using 16mm as the basis for successfully working in digital.
Ultimately, choice of compression needs weighting the proposed resolution, frame rate and visual preferences versus time constraints and budget.
Choice can even come down to something as unscientific as the most powerful personality among the crew - “DITs with an inflated sense of their own importance,” observes Bassett, “who insist that a specific codec is crucial.”


Network Considerations for Remote Production

Content marketing for Haivision
Contribution in the broadcast sense, is the delivery of video from a camera to a studio control area for processing and production. It’s a mainstay of live production but no-one wants it to contribute any more than necessary to the cost.
Producers with tight budget constraints (and who isn’t one of those?) have been exploring ways of reducing the amount of crew and facilities on site with at-home or remote-integration models (REMI) and, by extension, the cost of transmitting signals from A to B before onward distribution.
There are four basic video contribution options, each with pluses and minuses.
Satellite
Satellite offers the ultimate in high bandwidth. Its strongest suit remains the ability to contribute links from almost anywhere with guaranteed multi-camera and audio sync. But uplinks are notoriously expensive, especially for temporary periods. Connections need booking in advance and require pretty expensive terminal gear on both ends of the circuit.
Fiber-Optic Cable
Another super-fast option with copious bandwidth and lightweight is private fiber-optic cable. Fiber circuits can provide a stable level of latency and therefore the ability to sync video and audio feeds. The main downside, once again, is the cost which can put it beyond the boundary of most budgets. Such managed terrestrial video networks also require advanced planning, and are still only available in locations where carriers have chosen to deploy regional Points of Presence or PoPs.
In addition, the majority of venues in Europe and the US simply do not have a dedicated fiber connection and there’s often a total lack of such connectivity in less developed markets.
Mobile Networks
The possibility of using mobile cellular networks has come on in leaps and bounds in recent years. 3G and 4G networks are capable of streaming live video, especially if good compression is used, and they can be cost-effective too. The main issue is reliability. Telecom providers vary in coverage and signal strength, so picking a single provider for the whole of a cycle race, for example, may not guarantee you quality of service (QoS).
Multiple network providers can be ‘bonded’ together to form a single connection which can work for ad-hoc news gathering or as part of your overall production contributing footage wirelessly, from locker rooms for example. This solution is not, however, ideal for streaming multi-camera live events at a reasonable cost.
Public Internet
What’s left is the public internet, an infrastructure notoriously fraught with limited bandwidth, security issues, and fluctuating reliability. The internet’s primary obstacle is minimizing or eliminating video drift and maintaining lip sync over tens of streams. Surely you wouldn’t touch that if you are to serve your brand and audience in the best possible way?
Well, that’s yesterday’s news. Technological advances have made use of the internet for REMI as a genuine alternative even for high-value signal contribution.
Most of the issues with the internet were associated with , the basic video over the net distribution method which slowed down the stream and dramatically increased end-to-end latency while not providing content encryption
The industry is turning to (Secure Reliable Transport), an open-source (and  Emmy®-Award winning) internet streaming protocol, expressly designed for high-performance, secure, transmission and bandwidth optimization.
Hybrid Contribution Workflows
Producers of most major US sports events still rely on satellite or fiber to contribute their main feed, but are increasingly using the internet for contribution of additional content from tunnel areas, sidelines, commentary positions, multiple in-game camera angles, or fanzones outside the stadium.
Such ‘hybrid’ workflows, in which satellite or fiber is used in conjunction with an unmanaged network, can also relay a feed of the output back to a presenter on site displaying the coverage for confidence monitoring. The bidirectional capability of the internet is further being used by sports federations to deliver video analysis to the on-field officials from the remotely located Video Assistant Referee (VAR).
A growing number of live events have gone further and made REMI over the internet the center of their production workflow. Riot Games’ 34 HD 60p source capture of a was streamed from Paris back to Riot Games’ Los Angeles control room for live cutting and production. The latency over 5,500 miles was just 175ms with “zero” impact on the production team, it claimed.
The video game developer  didn’t stop there. Riot Games also remotely transported the teleprompter over IP: actual scrolling and prompting was performed in LA and fed to a prompting portable system in Paris. Riot Games tested sharing, audio mixing, and switching from LA, proving that it can produce the same high quality shows from anywhere without having to compromise on quality or spend a fortune.
While some might think this was a niche event, the League of Legends event was streamed to millions of fans worldwide across sites like Twitch. It’s massive, and part of the reason is that publishers are able to reach audiences all over the world from any location is with the live stream.
This is not possible if cost is in any way a consideration without using the internet and the right combination of technologies. Fortunately, with the evolution of streaming technology, fans from across the world are able to access their favourite events like never before.

Wednesday, 5 December 2018

EBU tech innovation chief Arcidiacono: PSBs need to join forces


Sports Video Group

At a time when the UK moves ever closer to parting company with the European Union, broadcasters across the continent are being urged to work more closely together than ever before.
“I fervently believe that the only way public service broadcasters can continue to stay relevant in an era of rapid technology change is to join forces,” urges Antonio Arcidiacono, the EBU’s new Director of Technology & Innovation. “We must use our collective expertise to innovate.”
Arcidiacono, who replaced Simon Fell at the beginning of September, highlights the global trends of 5G, artificial intelligence and OTT services as key areas where members might pool resources.
“By combining the efforts of members we can test and benchmark standards, build better product and services,” says Arcidiacono. “Collectively, we will be more efficient, decrease duplication and have more of an impact with respect to our audiences.”
The EBU is of course not solely comprised of members from the European community. Its 73 members in 56 countries include PSBs in Turkey, Israel, Jordan, Tunisia and the whole of Eastern Europe and Russia. Associate members are drawn from as far afield as Bangladesh, the U.S, South Korea, Japan and New Zealand.
“We see that in order to put AI to work effectively and to unlock its full potential for their organisations, all broadcasters must innovate. If you look at our members, even those with larger resources are only able to devolve a limited number of people to developing AI/ML applications.
“What we are discussing is a way of identifying areas in which broadcasters are working – AI tools for live sports production, for metadata gathering, for archive or the newsroom, for example – and then to get those teams to share research and development. These teams might be two or five people at individual broadcasters but together we can multiply the effort to bring applications to market faster.
“There is some fantastic work being done by broadcasters to solve specific problems and create new services and an enormous amount broadcasters can learn from each other.”
One example is speech to text. “Very often you find the best solutions for this in terms of AI applications are in American English but that won’t necessarily be the best solution for Dutch or German languages,” he says. “All our members have decades of archive which they are eager to leverage and they need AI/ML tools to help optimise it because the task is so vast it cannot all be done manually.”
At the same time, AI can be used to help broadcasters discern fake from genuine news sources. “To maintain their role as a public service, PSBs need to guarantee that the news they report and disseminate is factually accurate and they could do that by algorithms to help identify news reports which are not necessarily genuine.”
The EBU is organising another workshop on the impact of AI on media in the New Year. Among topics on the agenda; Will television sets track a viewer’s emotions and tune content to it? In practical terms what is AI? What is the status of the technology in the hype cycle? and what are the pitfalls in terms of security, data protection and ethics?
“Perhaps most importantly, we need a clear understanding of the use cases for AI/ML applications.”
Implementation of IP broadcasting
Prior to joining the EBU, Arcidiacono worked as director of innovation, and a member of the management committee, at Eutelsat where he was responsible for launching IP-based satellite services.
He was a founding member of the Digital Video Broadcasting (DVB) project and played a central role in the convergence between the broadcasting world and the Internet world in the mid 90’s with the launch of the DVB-IP set of standards.
It’s work he is keen to continue at the EBU which has played a significant role in specifying and benchmarking IP production standards, helping bring the ST 2110 suite into being earlier this year.
“There’s still a lot of work to do,” warns Arcidiacono. “IP production is still in its early stages for video and while we see lots of activity toward practical implementation it’s likely to be three to five years before this enters the mainstream.”
The EBU is providing guidance to its members on relevant emerging standards and advising broadcasters on the interoperability of ST-2110-based IP production tools that are being brought to market by vendors.
5G tests with 3GPP
When it comes to IP as a delivery mechanism to consumer platforms, the EBU plans to ramp up its activity in 5G in partnership with the 3GPP.
“While 5G will come on stream next year it will be in very limited areas and applications,” says Arcidiacono. “It will be several years before 5G really becomes widespread and in the meantime, broadcasters will be using existing 3G and 4G LTE networks.
“If you try to use a unicast IP infrastructure to deliver a popular media service to millions of people simultaneously you might be able to do it but the cost will likely be prohibitive, plus the infrastructure might not be able to cope with peaks in traffic. That’s why we think a combination of broadcast and unicast is best. You also need to be respectful of the law of physics and use the right network for the service.”
Arcidiacono points to the European Championships in Berlin last summer during which Italian broadcaster RAI and research group IRT trialled 5G for distribution of public service content.
“The trials showed how mobile technologies deployed on a conventional terrestrial broadcast network infrastructure can be configured to meet specific broadcast requirements, such as free-to-air delivery, wide-area coverage, and cost-efficient distribution to mobile devices,” he says.
The IRT demonstration used a combination of unicast and broadcast features, both of which will be inherent in 5G, to simultaneously deliver live TV broadcast as an MPEG-2 transport stream and HbbTV-like services.
RAI, although switching to second-generation DVB technology to deliver full-HD quality for all 13 terrestrial channels by 2022, is also seriously investigating the potential of 5G technologies to serve mobile devices from high broadcast towers.
Beyond the technical aspects of 5G, EBU members are increasingly interested in how 5G networks will be deployed and operated, and how broadcasters’ specific requirements can be met.
The EBU project group 5G Deployments is addressing issues related to business arrangements, deployment models, and regulatory conditions for 5G mobile systems.
The EBU has proven its ability time and again to converge the efforts of multiple members but it will only be by continuing to do so that it will achieve anything like the economies of scale of the major pay-TV streaming rivals.
“The fundamental role of the EBU in terms of technology and innovation is to propose and develop new solutions and to lead as a reference for its introduction. For broadcasting in the broad sense and for PSBs in particular this role remains more relevant than ever as the industry progresses.”
Before working at Eutelsat, Arcidiacono worked for the European Space Agency and started his career at Telespazio and Selenia Spazio. He has a Doctorate in Electronics & Telecommunications Engineering from the University of Pisa. Arcidiacono is fluent in Italian, English and French.


Monday, 3 December 2018

From post to playout: Advances in the cloud

IBC
From playout to post, the cloud is upending business models – but the most quantum leap is still to come.
Red Bee Media is building the first all software playout service with commercial launch as soon as Q2 2019.
First showcased at IBC2018, the solution is the first in the world to use uncompressed video over IP, without requiring any dedicated infrastructure.
It’s the result of two years R&D for the former BBC owned facility which still manages playout for all BBC channels. Now owned by Ericsson, headquartered in London and with eight media international media hubs, Red Bee also plays out the linear services for Sky, ITV, Channel 4 and Canal+ as well as VOD services including All4 and My5.
“Just lifting what we do into private or public cloud would be an interesting engineering challenge but one that would not necessarily benefit customers,” explains Richard Cranefield Head of Portfolio, Playout.
What broadcasters want, he says, is the ability to launch new channels quickly – such as an extra children’s channel for Christmas - and to adapt services for existing ones to target different audiences or regions for a short period of time.
That type of business model isn’t possible with a traditional playout model reliant on single-purpose machines which need purchasing and installing many months in advance.
“For us the benefits of moving playout to the cloud would only be realised if you can move all of it,” says Cranefield.
Playout has evolved less quickly to the cloud than other areas of the business, like media management and post production. Elements of the playout chain like automation have been virtualised (made software-only) but Red Bee’s wholesale move would not have been possible a couple of years ago.
“To deploy a new channel quickly we need to have all the tools needed to run a channel in software rather than physical video servers, or logo inserters sitting in frames in a rack, or compliance recorders, transcoders, graphics insertors. All of those have been discrete appliances but we needed them all in software.”
All of this had to operate together and for any channel or service that a Red Bee client wants at the drop of a hat.
While significant advances have been made by vendors in replacing SDI with IP and adapting hardware-centric product to software, there were gaps.
Red Bee worked with long standing partners including Imagine Communications, Grass Valley, and Pebble Beach but says not necessarily all of them delivered for its cloud deployment.
“The vendor community are doing a great job at running interoperability tests of their own ecosystem and sometimes between each other’s product but you need that tech to work in your own environment,” he stresses.
That’s where the interop promise of IP is sometimes not matched by reality.
“Interop is nowhere near as simple as plugging one piece of kit isn’t another with an SDI cable. Now your software may not directly interface with another software. It will need connecting over the network fabric passing through two or more other vendor’s infrastructure. This is what our two years of testing has enabled us to achieve.”
Indeed, figuring out what vendor’s equipment interoperates well with another is part of Red Bee’s “intellectual property” he says and something Red Bee hope will give them a competitive edge.
“We’re also not vocal about saying which kit specifically we are deploying because we don’t see this as a design set in stone.”
Red Bee wants the ability to swap software in and out depending on whether further software advances make better use of CPUs or network bandwidth or are more cost effective.
Its service will combine public cloud (from Microsoft Azure among others) and private cloud (built at some of its data centres) although it will use public cloud for the more bursty OTT services and private cloud for playout due to the level of complexity and bandwidth needed for a playout.
“Because of the amount of live events we carry which need to work in low and predictable latency we need to work with uncompressed video,” says Cranefield. “We have got uncompressed streaming at 1.5 Gbps so doing compressed would be easy.”
There is no launch customer although it is talking with all its existing clients about migration. UK broadcasters are ahead of the pack.
“Devising the right commercial model was as hard, if not more so, than getting the actual technology to work,” he says.
It has persuaded vendors to switch from selling product as a one-off capital outlay to being an ongoing operational expense. At the same time, it had to work out a rate card of predictable pricing for clients.
“They will rent capacity at a known price to help them make business decisions much quicker.”
Rebuilding post production in the cloudLong anticipated, the ability to collaborate in realtime with teams in multiple geographic locations is a reality that is altering the post production landscape.
It is VFX which has led the way. The computing muscle of banks of computers has been routinely called on to render frames but it has only been the largest facilities which have been able to either accommodate the horsepower on site or to pay for spinning up additional machinery in the cloud.
MPC did this on its way to winning an Oscar for The Jungle Book, thanks to being able to run millions of Core hours on Google’s cloud.
So established has the cloud rendering process become, that even this once ground-breaking capability has evolved to encompass a whole host of post production workflows from pre-viz to transcoding. In doing so, the field has also opened up to smaller shops and even freelancers.
One of the key benefits is not needing to buy or maintain expensive hardware to run a business: everything runs in the cloud, with only the display being streamed to the user.
“The cloud can essentially offer a pop-up studio, accessible anytime, anyplace across the world,” says Jon Wadelton, chief technology officer at Foundry.
This year Foundry launched Google cloud-hosted platform Athera intended to make it possible to run a visual effects studio using nothing more than a laptop and a standard web browser.
“The lightbulb moment for me was having a 2 Gig Lidar scene loaded to Google Cloud and running on Nuke,” explains Lindsay Adams, VFX Supervisor at Sydney’s Future Associate which is working on VFX for HBO’s serialisation of graphic novel Watchmen. “When I animated a virtual camera and sent that to the compute stack a result came back within 10 minutes. I had access to more than a thousand machines from my hotel room from my Macbook Pro.”
Sohonet has a competing service, used by Milk in its Oscar-winning work on the sci-fi Ex Machina.
CEO Chuck Parker says the technology has now evolved to a point where a post company might employ only a dozen managers and project supervisors on its books. “They can bid with confidence on jobs of any scale and any timeframe knowing that they can readily rent physical space in any location, anywhere in the world, to flexibly take advantage of tax breaks and populate it with freelance artists,” he says.
With file sizes set to continue to increase and as immersive media like VR emerges, leveraging the cloud will not only be routine for the highest budget projects and largest vendors.
“You can have big budget artists and use big budget renders but not have the big budget facility,” says Austin Meyers, Associate Director at LA post house Steelhead. “It will be a major turning point in the industry.”
Widescale adoption of this technology may not happen overnight, but we’re already seeing the beginnings of it with early pioneers like Inversion Studio.
“The most useful thing is being able to access our files and footage anywhere in the world and use talent anywhere in the world,” says Arvind Sond, co-founder of Inversion Studio. “Working on the cloud is the future.”
Quantum cloud computingWe are on the verge of an era of unprecedented data in which the volumes generated will be so incalculably large that current computing technology will buckle and break under the strain.
The race is on to build the first generation of quantum computers which promise souped-up calculation speeds over existing computers and to supercharge developments in artificial intelligence.
Quantum engineering began a few years ago in anticipation that Moore’s Law, a pretty consistent theory of computing power, was about to hit its shelf life.
Moore’s Law predicts that the number of transistors on a microprocessor would double every 18 months but in the next few years, and perhaps as soon as 2020, it will be practically impossible to build semiconductor transistors any smaller – using current technology.
Quantum computers will be measured on the atomic scale using the power of atoms and molecules to perform memory and processing tasks.
“Microsoft and others are working on quantum computing,” Microsoft co-founder Bill Gates wrote on a Reddit blog. “It isn’t clear when it will work or become mainstream. There is a chance that within 6-10 years that cloud computing will offer super-computation by using quantum.”
In 2015, Google claimed it had developed a quantum computer capable of processing 100 million times faster than today's systems. A year later boffins at Massachusetts Institute of Technology (MIT) and the University of Innsbruck in Austria announced the creation of the first scaleable quantum computer.
Last November IBM released a quantum computer cloud service, only to be trumped in March by Alibaba Cloud and the Chinese Academy of Sciences, which also launched its first cloud-based quantum computing.
IBM, though, has an upgraded version in the works as do Intel, Microsoft and more.
At CES 2018 in January, Intel CEO Brian Krzanich predicted that quantum computing will solve problems that today take months or years for the most powerful supercomputers to resolve.
Quantum computations use quantum bits (qubits), which like the famous Schrödinger’s cat thought experiment, can be in multiple states at the same time – quite different from digital computing’s requirement that data be binary (either in one state or another, 0 or 1). Running a large number of calculations in parallel in the cloud opens a future where complex problems can be solved in much less time on a quantum computer compared with a traditional digital device.
But while quantum computing has great potential, the field is in its infancy. And it will take many generations of qubit increases for quantum computers to begin solving the world’s challenges.
“In the quest to deliver a commercially viable quantum computing system, it’s anyone’s game,” said Mike Mayberry, corporate vp and md of Intel Labs. “We expect it will be five to seven years before the industry gets to tackling engineering-scale problems, and it will likely require 1 million or more qubits to achieve commercial relevance.”
With current quantum computers such as Alibaba’s capable of a mere 20 qubits you begin to realise the magnitude of the task. We won’t be using a quantum PC on the desktop anytime soon.