Codecs 101: A Post-Production Primer

NAB

If you’re confused about the plethora of codecs used from capture to delivery as video passes through editorial then a handy primer has arrived from Shift Media.

article here

The workflow tools developer has laid out all the main codecs in use in postproduction today, explains what they do and their limitations.

The reference guide divides up the uses of codecs into three categories: capture formats, intermediate (or editing) formats, and release formats.

As Charles Haine explains in an introduction to the paper, the job of a capture format is to capture as much of the on-set information as possible. You want the brightest brights and the darkest darks and the full range of colors in front of you captured as best you can.

The job of an intermediate codec is to be simple to work with; you want your editing process to feel easy, with applications opening quickly, timelines whizzing by, and quick exports for client approval.

For final release, the goal is different; the file will generally only be played linearly (forward from beginning to end), without skimming or image manipulation, so all you care about is making the best-looking image in the smallest file possible.

The paper dives into the detail of the codecs in most common use today and how they are applied to perform different jobs.

Apple ProRes is currently the most widely used codec in all motion picture post production. One of the factors that keep Macs on top in motion picture post is the functionality and ubiquity of ProRes.

ProRes is used all the way from image capture in major platforms like the ARRI Alexa and editing in any of the four major NLE platforms all the way through to delivery, with streamers and major networks accepting ProRes file for delivery.

The basics to understand is that Apple ProRes isn’t just a single codec, but a family of codecs built around the same technology, available in multiple implementations.

As the paper explains, you can think of these as “flavors” or “strengths” of ProRes. These flavors refer to both the method of encoding the image, 422 or 4444, and the data rate, how many Mb per second are allocated to creating the image. The higher the data rate, the higher quality the image reproduction will be, with fewer artifacts, but on the flip side, the larger the file will be.

The codec is great for post, but things break down at the consumer level. If you are delivering a file to a client, there still isn’t an easy way to get a non-tech savvy Windows user who defaults to Windows Media Player to playback a ProRes file.

Avid DNx is another common codec range available at a variety of data rates and encoding for a variety of workflows. Widely supported on both PC and Macs, DNx “can be a great codec to use if your facility has mixed platforms or you are collaborating with others working in a variety of different formats. This has been its greatest strength,” the report says.

“However, it’s not particularly easy to install for the less technically savvy, so it again doesn’t make a great format for delivering cuts to clients since it requires installing a professional application for support.”

H.264/H.265 are “consumer-facing” codecs mostly used in delivery, especially on web platforms.

“You aren’t going to send an H.265 file to Netflix or HBO, but if delivering to Instagram, YouTube, Vimeo or a work-in-progress review platform, you are going to be using H.265 all day long to get a file that is both small enough to quickly upload but still looks good enough to share with the world.”

These codecs are built around Long GOP technology — great for viewing something linearly forward in time and therefore for delivering video over the web. However, Long GOP can be very awkward in the editing room, since it requires your video software to recreate individual frames by looking at the group of frames. If you are scrubbing around, it can be laggy, and if you cut in the middle of a GOP group, the software has to recreate the missing picture information by holding those other frames in memory.

It is highly recommended that footage is transcoded into ProRes or DNxHR for an easier post-workflow experience.

The Shift Media guide also dives into the use of RAW formats for capture. 

Because of the unprocessed nature of RAW files and the massive file sizes of capture video codecs, as soon as we get into post, the first step is often doing a demosaic (sometimes called a debayer). This process takes a video file from one format (RAW) and translates it into a codec for use in editing.  

There are two major categories of RAW, open RAW formats and proprietary or closed RAW formats.

Open RAW formats are designed for many different platforms to capture to or work with. Proprietary formats created by a camera company are often only supported by that one company, with varying support from post-production software.

The paper details the basics of the most common Raw capture options including ProRes Raw, Blackmagic RAW, RED RAW and ARRIRAW.