IEEE: Hollywood Archivists Can’t Outpace Obsolescence

Post news stories and home video release announcements here.
  • Author
  • Message
Offline
User avatar

Mike Gebert

Site Admin

  • Posts: 5670
  • Joined: Sat Dec 15, 2007 3:23 pm
  • Location: Chicago

IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostFri Apr 28, 2017 6:34 pm

http://spectrum.ieee.org/computing/it/t ... solescence

A good (long) piece on the subject of digital preservation; NitrateVillain Dino Everett is quoted:

Digital technology has also radically altered the way that movies are preserved for posterity, but here the effect has been far less salutary. These days, the major studios and film archives largely rely on a magnetic tape storage technology known as LTO, or linear tape-open, to preserve motion pictures. When the format first emerged in the late 1990s, it seemed like a great solution. The first generation of cartridges held an impressive 100 gigabytes of uncompressed data; the latest, LTO-7, can hold 6 terabytes uncompressed and 15 TB compressed. Housed properly, the tapes can have a shelf life of 30 to 50 years. While LTO is not as long-lived as polyester film stock, which can last for a century or more in a cold, dry environment, it’s still pretty good.

The problem with LTO is obsolescence. Since the beginning, the technology has been on a Moore’s Law–like march that has resulted in a doubling in tape storage densities every 18 to 24 months. As each new generation of LTO comes to market, an older generation of LTO becomes obsolete. LTO manufacturers guarantee at most two generations of backward compatibility. What that means for film archivists with perhaps tens of thousands of LTO tapes on hand is that every few years they must invest millions of dollars in the latest format of tapes and drives and then migrate all the data on their older tapes—or risk losing access to the information altogether.

That costly, self-perpetuating cycle of data migration is why Dino Everett, film archivist for the University of Southern California, calls LTO “archive heroin—the first taste doesn’t cost much, but once you start, you can’t stop. And the habit is expensive.” As a result, Everett adds, a great deal of film and TV content that was “born digital,” even work that is only a few years old, now faces rapid extinction and, in the worst case, oblivion.
“Sentimentality is when it doesn't come off—when it does, you get a true expression of life's sorrows.” —Alain-Fournier
Offline
User avatar

luciano

  • Posts: 116
  • Joined: Wed Nov 14, 2012 11:11 pm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostFri Apr 28, 2017 6:49 pm

Why in the world do they not mention the emerging horizon of DNA? Lobster film sent me a link about this a while ago.

https://www.google.com/amp/s/phys.org/n ... bottle.amp


https://www.extremetech.com/extreme/245 ... s-per-gram
Offline
User avatar

CoffeeDan

  • Posts: 908
  • Joined: Thu Dec 11, 2008 2:55 pm
  • Location: Cincinnati, Ohio

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 6:57 am

luciano wrote:Why in the world do they not mention the emerging horizon of DNA?


Because nobody is enthralled by a storage system that could easily be lost amid the sofa cushions or swept out with the daily dust . . .
Offline
User avatar

Donald Binks

  • Posts: 2856
  • Joined: Fri Jun 17, 2011 10:08 am
  • Location: Somewhere, over the rainbow

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 7:18 am

I would venture that there isn't any product currently available that can preserve film for ever - unless someone invents a way of preserving them in stone. Why, we only have to look at the fact that no film survives from the Egyptian Pharaonic or Roman Empire periods as proof of this.
Regards from
Donald Binks

"So, she said: "Elly, it's no use letting Lou have the sherry glasses..."She won't appreciate them,
she won't polish them..."You know what she's like." So I said:..."
Offline
User avatar

oldposterho

  • Posts: 476
  • Joined: Wed Jan 22, 2014 10:05 am

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 8:12 am

To be sure, the lost Cleopatra footage haunts me.
Offline
User avatar

Jim Roots

  • Posts: 2507
  • Joined: Wed Jan 09, 2008 2:45 pm
  • Location: Ottawa, ON

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 9:05 am

Donald Binks wrote:I would venture that there isn't any product currently available that can preserve film for ever - unless someone invents a way of preserving them in stone. Why, we only have to look at the fact that no film survives from the Egyptian Pharaonic or Roman Empire periods as proof of this.


You mean I should put away my hammer and chisel? Damn, and I just paid to get the chisel sharpened!

Jim
Offline
User avatar

luciano

  • Posts: 116
  • Joined: Wed Nov 14, 2012 11:11 pm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 10:58 am

CoffeeDan wrote:
luciano wrote:Why in the world do they not mention the emerging horizon of DNA?


Because nobody is enthralled by a storage system that could easily be lost amid the sofa cushions or swept out with the daily dust . . .


If you have a film as short (and physically small) as Newark Athlete what do you do? You still put it in a can. If costs keep going down, I don’t see why you couldn’t write a whole section of backup vials and store them in a large cartridge. Granted, I believe the vial itself should be more bulletproof. But to look at DNA and write it off at this stage is ridiculous. It’s showing enormous potential. Next to 35mm and 65mm negative, it’s the best storage medium in sight.
Offline
User avatar

bigshot

  • Posts: 840
  • Joined: Sat Nov 06, 2010 2:59 pm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 11:00 am

I'm a digital archivist myself, and my philosophy is to use standard formats that everyone uses, back everything up multiple times and store off site and in the cloud. You have to be ready to migrate data to a new device every few years and transcode to another lossless format if necessary. But even more important than all of that is to have convenient viewing copies in a streamable format to get the data into the hands of curators and researchers. Data pickled in a salt mine might as well not exist as far as I'm concerned. It has to be available to be a vital part of culture.
Offline
User avatar

R. Cat

  • Posts: 237
  • Joined: Thu Oct 02, 2014 11:34 pm
  • Location: The Nearest Pub

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 1:17 pm

bigshot wrote:I'm a digital archivist myself, and my philosophy is to use standard formats that everyone uses, back everything up multiple times and store off site and in the cloud. You have to be ready to migrate data to a new device every few years and transcode to another lossless format if necessary. But even more important than all of that is to have convenient viewing copies in a streamable format to get the data into the hands of curators and researchers. Data pickled in a salt mine might as well not exist as far as I'm concerned. It has to be available to be a vital part of culture.


Totally with you on availability, access being vital to culture. That said, in respect to storage I'm more trustful of salt mine pickling over reliance on dissipatory clouds for long term preservation.

European film archives somehow survived two world wars, ...a miraculous feat by any standards. Worldwide, public and private collections have been responsible for the retrieval and restoration of countless films thought lost forever. Everything recovered has been film based. Obsolescence of the tools required to restore and playback film is certainly an ongoing issue, but if car restorers can rebuild vintage classics without all original parts, then savvy film technicians should have little difficulty MacGyvering whatever is required to restore old movies.

I'm less confident about relying on streamable technologies that change faster than Prada fashions. Anything preserved in a form that can be corrupted by magnetic pulses, sunspots or the latest iteration of Windows isn't an optimum storage device. Of course, going forward, virtually all new "film" is digital, so future preservation has to reflect thinking outside the film canister.

We're all aware the clock is ticking on stored nitrate. This is where digital DNA may be relevant going forward. Thomas Dolby, take a bow! If science has devised a means of preservation that's cheaper to replicate, store, and retrieve film that's less prone to degradation & obsolescence as projector based film stock then perhaps this medium will eventually be a panacea for archivists. Time will tell.

Naturally, the jury is still out on whether anything currently on the horizon will solve long term film storage issues, but if digital DNA works, maybe some brilliant arbiter of cultural preservation will solve the A&C dilemma of "Who's on first?"
Offline
User avatar

boblipton

  • Posts: 5248
  • Joined: Fri Jan 18, 2008 8:01 pm
  • Location: Clement Clarke Moore's Farm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 1:33 pm

As a fan of obsolete technology -- which may reasonably be defined as something that works because they're not always changing it --

II don't know where I was going with this, but I'll leave it here for the joke.

Bob
The matter is complicated, and I shall proceed to complicate it still more.

-- Avram Davidson
Offline
User avatar

luciano

  • Posts: 116
  • Joined: Wed Nov 14, 2012 11:11 pm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSat Apr 29, 2017 1:54 pm

This will probably sound insane, but I do believe there is a final frontier for preservation on film. I personally dream of an extremely high resolution, 75mm stock that has a base hundreds of times stronger than Estar and an emulsion that's nuclear proof. I don’t know if anything like that could exist with modern technology but I’d like to think it could happen one day. Costs would be prohibitive and it would be considered crazy and superfluous by any archive, but it’s an interesting idea to mull over sometimes. What I really see a bright future for is DNA. It will win in the end.
Offline
User avatar

bigshot

  • Posts: 840
  • Joined: Sat Nov 06, 2010 2:59 pm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSun Apr 30, 2017 12:04 pm

There are digital formats that haven't become obsolete. Images in TIFF and JPEG format will likely be readable for a very long time, or at least convertible to other formats. The same is true of hard drives. USB is likely to be supported and easily converted for a long while. The MOV file format has been around for a very long time too. The trend now is to create "wrappers" like MKV that make multiple formats all open within a single file format. Compatibility isn't as big a problem as it used to be. It's more a problem with software than it is data files.

If you have multiple copies of digital files in a standard format, and you spin up your hard drives every year or so, you can always migrate your data to another drive in 8 or 10 years and update whatever format you need to update at that point. The advantage of digital is that all the information isn't invested in a single object. The information can be replicated and stored in multiple locations.
Offline
User avatar

R. Cat

  • Posts: 237
  • Joined: Thu Oct 02, 2014 11:34 pm
  • Location: The Nearest Pub

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSun Apr 30, 2017 2:25 pm

bigshot wrote:There are digital formats that haven't become obsolete. Images in TIFF and JPEG format will likely be readable for a very long time, or at least convertible to other formats. The same is true of hard drives. USB is likely to be supported and easily converted for a long while. The MOV file format has been around for a very long time too. The trend now is to create "wrappers" like MKV that make multiple formats all open within a single file format. Compatibility isn't as big a problem as it used to be. It's more a problem with software than it is data files.

If you have multiple copies of digital files in a standard format, and you spin up your hard drives every year or so, you can always migrate your data to another drive in 8 or 10 years and update whatever format you need to update at that point. The advantage of digital is that all the information isn't invested in a single object. The information can be replicated and stored in multiple locations.


But bigshot, aren't you concerned about magnetic storms, polar inversion, sunspots & the coming zombie apocalypse? :wink:
Offline
User avatar

Changsham

  • Posts: 793
  • Joined: Wed Feb 02, 2011 3:34 pm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostSun Apr 30, 2017 10:51 pm

The thing that spooks me about digital is that when the power goes off there is only nothingness left.
Offline
User avatar

bigshot

  • Posts: 840
  • Joined: Sat Nov 06, 2010 2:59 pm

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostMon May 01, 2017 10:46 am

R. Cat wrote:But bigshot, aren't you concerned about magnetic storms, polar inversion, sunspots & the coming zombie apocalypse? :wink:


I don't know about those natural disasters, but here in Los Angeles, the zombie apocalypse has already taken place.

Changsham wrote:The thing that spooks me about digital is that when the power goes off there is only nothingness left.


You'll want to get yourself a wind up phonograph.
Offline
User avatar

Donald Binks

  • Posts: 2856
  • Joined: Fri Jun 17, 2011 10:08 am
  • Location: Somewhere, over the rainbow

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostTue May 02, 2017 3:28 pm

bigshot wrote:]The thing that spooks me about digital is that when the power goes off there is only nothingness left.


You'll want to get yourself a wind up phonograph.[/quote]

What always amazes me is that when a storm or other impending disaster is threatened, the authorities tell us to watch TV broadcasts or listen to the wireless in order to stay abreast of events. Obviously these people are oblivious to the fact that storms and other impending disasters usually cause the power to go off, thus making it impossible to subscribe to their suggestion.

Another helpful suggestion I have found when my computer has spat the dummy is a telephone recording put out by the "fixer uppers" telling me to gain the required information from a website.

You have to laugh!
Regards from
Donald Binks

"So, she said: "Elly, it's no use letting Lou have the sherry glasses..."She won't appreciate them,
she won't polish them..."You know what she's like." So I said:..."
Offline

wich2

  • Posts: 1400
  • Joined: Tue Apr 08, 2014 11:11 am

Re: IEEE: Hollywood Archivists Can’t Outpace Obsolescence

PostWed May 03, 2017 8:12 am

Battery radio, Sir Donald... BATTERY radio.

That has long been, and still is, the best source for Emergency News.

-Craig

Return to Silent News

Who is online

Users browsing this forum: Bing [Bot] and 4 guests