Is All of Your Media Worth Keeping?

IBC

http://www.ibc.org/page.cfm/action=library/libID=2/libEntryID=183/listID=4

In a file-based IP-enabled environment we are told that we can squeeze more and more value out of more of the media we capture. However, just because we can capture and store images and related information as data, should not automatically mean we have to. Every bit we store has a cost. So what is the true value of media considered from the perspective of deep archive?

The problem of media obsolescence has been an issue for the industry for many years. Just how do you preserve media decades from now, given the risk that neither the physical mechanisms nor the software codecs to replay it may exist?
Already, the first iterations of existing archive standard LTO tape are obsolete. Broadcasters are required to coordinate a migration of their archive every few years just to keep it up to date. LTO is, in any case, viewed as more of a short to mid-term storage system, useful for near line access on robotic systems and favoured because of its remarkably cheap basic cost of around €30 per cassette. Questions are being asked, though, about whether LTO's roadmap for capacity can keep up with the sheer volumes of data now being generated.

The paranoia over irretrievable future media was arguably more of a concern a few years ago when proprietary codecs were locked into turnkey hardware. The development of open source framework FFmpeg permits files written in any mainstream codec to be decoded, in theory, at any point down the line. Then there is ISO standard Universal Disc Format written in 1986 but which essentially still unites CDs made in 1982 with optical file systems of tomorrow. Ironically, given the widespread move to digitise film archives, the only media with a proven hundred year lifespan is celluloid. What makes film particularly appealing is that all it takes to read it back is a light source and a mechanism to move it frame by frame. The BFI, custodian of the UK's film heritage, has a dual strategy in which it plans to digitise 10,000 titles by 2017 while maintaining a temperature controlled vault for long term preservation of up to 450,000 film cans.

“The fundamental long term approach is to maintain original film elements in a secure, cold and dry vault storage, with digitisation at 2K resolution providing the means to digital access as well as a further layer of preservation,” it states. “As scanning technologies evolve, it will be possible to return to these high quality materials and reproduce images from the physical materials.” However, with over 700,000 television programmes also within the BFI National Archive, it must consider the future preservation needs of video content, which presents a different set of challenges. While VTs themselves have an inherent material stability through their polyester bases with less stringent storage climates required than for film, a serious threat to longevity exists because of dependency on multiple iterations of proprietary machinery, spare parts and specialised technical skills - all of which will ultimately become exhausted due to lack of demand from the commercial environment they were primarily serving.

The BFI believes digitisation is the only viable method for preserving legacy videotape content. What though of contemporary digital born works? “It is essential that preservation systems be created with inherent self-migrating capability, in order to sustain and maintain accessibility of the vast quantities of data,” states the BFI.

While it would be unrealistic to scan to film every piece of content being shot on file, Paul Collard, VP Film and Digital Services, Deluxe 142, makes the case that high value content should be preserved this way. “As 4K becomes the standard for post you could record high resolution content back to film in 4K either as colour stock for 100 years or as black and white separations for a life of 200 years,” he says. Alternative archive technologies are being devised. These include holographic techniques which use light to read and write data in three-dimensions for increased storage density and rapid data rates, though development appears to have stalled.

DOTS (Digital Optical Technology System) – a Future Zone exhibit at IBC2011 from Group47 - vouchsafes archival for 100 years and contains within it a microscopic 'Rosetta Stone' of instructions on how the data is encoded and on how to construct a reader.
Then there is M-Disc, already available as a Blu-ray, which engraves data by laser into a mineral layer with a claimed shelf life of 1,000 years. The catch is that it is intended for consumer rather than professional use and may not have the storage capacity required of media companies transporting Gigabytes and Terabytes daily.

Perhaps though, we should turn the argument on its head and look not at the technology but the business model. “What is the value in keeping it all?” poses Niall Duffy, Mediasmiths CEO. “For me, the cut off is 50 years, after which there is only a historical and socio-anthropological reason for archive.” This approach should substantially reduce costs, but more weeding is needed. “Today's fashion is to store everything, all the little bits and pieces, not just the master,” he says. “Part of the problem is that there is no structure to what data you really need to keep and what you need to throw away.” Second guessing the business models of a decade's time is futile perhaps, but the issue of knowing what is valuable and being able to find or delete it, is bound up with metadata and the intelligent use of it.