Best Archival: Blu-Ray, DVD or USB?

[arobertson1]

DVD has an estimated lifespan of 10 - 20 years (depending on who you ask). Blu-Ray has an anti scratch surface and USB drives can retain information for so many reads / writes.

I am beginning to wonder if USB drives are starting to make headway with archiving. Suppose I was to archive some pictures on it and lock it away in a drawer - would it still have retained the data in twenty years time providing I didn't use it frequently (checking every so many years)? The largest I can buy at the minute is 64Gb, which is still large enough to hold several DVD's worth on it.

My main reasoning for this is I really don't see DVD's / Blu-Ray disks staying around in twenty years time. I do think that USB storage will literally expand to rediculous sizes in this time and become the main preference for storage (who wants a bulky DVD / Blu-Ray collection when it can fit on one USB drive?). I would imagine that media players will be the norm, which you literally plug in your USB drive to watch what you want / pay a fee to download and watch.

I'm really after a format that will stand the test of time to keep my precious photos and home videos alive when I'm long gone and my son is showing his children. (Just like slides / photographs used to be to me).


Alan

[videopoo]

I would say HDD would be your best option for now. Next would be DVD. 10-20 years is a long time. God knows what computing will be like then.

[MOVIEGEEK]

I think a USB flash drive is the safest(no moving parts) but a HDD is the cheapest(per GB).A USB HDD is probably the best solution because who knows what interface we will have in 20 years.

[Vidd]

Blank DVDs (Verbatim or TY), external hard drives, and USB flash drives are all cheap and go on sale often. If the data is important then you should store at least two backups. A single backup is useless if it fails. If possible, backup your pictures to all 3 devices, as well to online storage solutions (even if you email your pictures to a dedicated Gmail account).

Then make new backups once the technology changes.
Burn DVDs today, but 3 or 4 years from now when Blu-Ray blanks are cheap, copy the contents of the DVD to Blu-Ray. Do the same in another 10 years when a newer technology replaces Blu-Ray.

[guns1inger]

USB drives are incredibly failure prone. Think of them as the new floppy disc. Great for short term storage, for moving data from point a to point b, and damned handy. But they are not an archival media, and not reliable enough for permanent backup.

[edDV]

I think a USB flash drive is the safest(no moving parts) but a HDD is the cheapest(per GB).A USB HDD is probably the best solution because who knows what interface we will have in 20 years.

Broadcasters are finding HDD unreliable for archive (but rebuildable at high cost). Turns out the drive lubrication (not magnetic surface) dries out after years on the shelf with no use.

Sony is pushing Blu-Ray for 10 year Archive. See here

http://pro.sony.com/bbsc/ssr/app-postproduction/resource.demos.bbsccms-assets-micro-xd...home:avitflash

[orsetto]

HDDs fail from lubrication issues when stored unused, although the data is recoverable at great cost as edDV mentioned. USB flashdrives should theoretically last "forever" but in practice they're cheaply-made crap and they will fail catastrophically on your most important file when you least expect it. Jury's still out on recordable optical media: could be 20 years, could be 3 years. Use all three whenever possible, and make backups of the backups regularly, moving on to new storage technologies as they become mainstream.

I have Apple Mac and Windows floppy diskettes going back to 1987 that still read perfectly, but of course they rely on obsolete drives and have very little capacity. I backed most of my floppies onto CD-R back in 2001 and so far those CD-Rs are holding up fine. I do clone them to a new set of CD-R every two years, though.

[edDV]

The good news is HDD capacity keeps going up and prices down. Trick is to dub them to new drives every few years.

Optical media has failed to keep up on price/GB and flash archive reliability is unknown. Magentic media does well with proper storage.

[arobertson1]

Thanks everybody. I think I'll cover my bases and go with DVD and USB for now and later I'll backup to Blu-Ray as well.

Alan

[yoda313]

What about solid state memory? I'm not entirely sure on the concept but isn't it the next step in storage?

[orsetto]

We should maybe clarify the original poster was asking about archiving a relatively small amount of still and video family images. If these will all fit on a single, affordable hard drive then that is currently the preferred option along with secondary backups on optical media and/or flash memory. The drive should be kept in a USB2 case and read periodically to keep the bearings lubed. After a couple years, clone it to a new drive and clone the backups as well. Eventually a new "impervious" storage media will emerge and you can move everything to that.

For those with huge video collections of things recorded from off-air or cable, the answer is not so simple. Hard drive storage is still WAY more expensive than DVD-R for home video. A spindle of 50 good-quality TY DVD-Rs costs $20, a good 250GB hard drive to hold their disc images is at least $50. Of course you could assemble a huge RAID system and lower the costs considerably, but you'd still face the potential issue of eventual lubrication and bearing problems. If you can afford it, its certainly possible to replicate your collection onto a new set of hard drives every few years, but it may not be any better than simply holding onto the DVD-Rs themselves until more reliable inert storage appears.

Tough call, and tougher on a budget.

[archaeo]

USB drives are incredibly failure prone. Think of them as the new floppy disc. Great for short term storage, for moving data from point a to point b, and damned handy. But they are not an archival media, and not reliable enough for permanent backup.

That's interesting. I didn't know that they were having such problems... Do you have any web links with reviews, feedback, etc about this issue? I've been considering archiving my dvd collection onto one of these external drives, but if their failure rate is high, no thanks.

[edDV]

USB drives are incredibly failure prone. Think of them as the new floppy disc. Great for short term storage, for moving data from point a to point b, and damned handy. But they are not an archival media, and not reliable enough for permanent backup.

That's interesting. I didn't know that they were having such problems... Do you have any web links with reviews, feedback, etc about this issue? I've been considering archiving my dvd collection onto one of these external drives, but if their failure rate is high, no thanks.

Buy two drives and test them both every few years. Dub to new drives every 5.

At current progress 10TB drives will cost about $100 in 5 years.

[guns1inger]

Let me clarify - I read USB Drives to be USB memory sticks. USB External Drives are HDD with USB interfaces. When I say USB Drive I mean it in this context.

[slacker]

Internal 1 TB hard drives are under $100 now. External roughly $150.

And LTO tape or other?

[lordsmurf]

BluRay is destined for failure, so using it for a format may prove tricky when the drives disappear in years to come. Not much different that Zip discs, Betamax/Super Beta, MOD, and other formats of data/video/audio storage. It is a next-gen Laserdisc format, to be quite honest, and one that people are ignoring in the same way LD was ignored.

Hard drive is best of the choice shown here, but I would keep single-layer DVD backups too.

I don't know that flashable memory is unreliable. There are different kinds of solid-state architecture, so I guess it all depends on what kind you're planning to use.

[archaeo]

I don't know that flashable memory is unreliable. There are different kinds of solid-state architecture, so I guess it all depends on what kind you're planning to use.

I've read that the SSD's (flash drives) do have limited lifespans, based upon the number of write/erase cycles, which is based on the quality of cells used in the drive. For lower graded flash drives (MLC type), cells can wear out after 1,000 to 10,000 write cycles , and for higher quality (SLC), they can go up to 100,000 write cycles.

[crazy14muzic]

i personally would store on good quality optical media
even if the drive that wrote the data dies you can still buy another one

I have data on a harddrive that cant be retrieved unless i pay $$$$ to get it back. options being, having it sent off for data retrieval or buy a compatible controller board that may or may not work. money that would be much more than optical media if it fails.

Flash drives, a basic eeprom has been around longer than them all but still have a dependancy system that if the host cant boot up, you cant get to the user data area to retrieve your information. I had a flash drive go bad. as far as i know the data was there but I couldnt get to it because the computer didnt recognize it.

optical media if kept in good storage should be backward compatible for years to come. look at cd, is still being read by blueray drives. I have cds that have outlasted the drives that wrote them and still can read them.

quick look

optical media=requires drive to read it which shouldnt be a problem to get

hard drive=controller boards fail, bearings dry up, platters develop bad sectors

flash drives=still new to the scene regarding size. I have any old 5 year + samsung flash drive mp3 player thats still going strong today. several read/writes but if the onboard os corrupts it becomes a brick.

[therock003]

That's strange. Optical media are supposed to last 10-20 years? I thought their lifespan extended to nearly a lifetime!

So really where are we supposed to securely backup our data? Are we supposed to cycle them around every now and then in order to preserve them?

[INFRATOM]

The most reliable archive media is DVD-RAM that's what government , courts and most agencies use for archiving.
Below is an article from Stanford on long-term usability of optical media.
http://cool-palimpsest.stanford.edu/bytopic/electronic-records/electronic-storage-media/critiss.html

[lordsmurf]

That article is ancient, written in the 1990s. The only non-press optical media that existed at the time (within reason) were DVD-RAM and CD-R/CD-RW. There are a number of organizations that have totally moved away from DVD-RAM, because it was not Superman-like in durability.

Optical media will more than likely outlast the drives that exist to read them. Discs can last decades, but the lasers and mechanical drives used to read them fizzle out in just a few short years (2-5 years, in general).

[betwixt]

I use flash memory, similar in technology to that used on USB sticks and also face the prospect of limited rewrites. However, the manufacturer says they can be rejuvenated by erasing the whole memory and re-writing it again. Does anyone know if this is the case with USB stick memories?

The theory is that each time something is written to the memory, the surrounding memory cells are slightly erased. After many write operations, parts of the block may erase completely and their contents permanently lost. Reading ALL the memory contents out, erasing all the cells and writing everything back leaves the memory in 'new' condition again. I stress this is fact with the chips I use but may not be true with the NAND memory used in normal USB sticks.

Of course, if the stick is used only once to archive video or files, this is less of a problem as its the continual writing that causes the degradation.

Comments???
Brian.

[yoda313]

Optical media will more than likely outlast the drives that exist to read them. Discs can last decades, but the lasers and mechanical drives used to read them fizzle out in just a few short years (2-5 years, in general

So perhaps the best advice would be to use a combination of harddrives and optical media. Then buy a backup usb version of said optical format and keep in a cool dry place as a future replacement should any thing happen. Then that way since USB is likely to be with us forever the only factor in the future will be driver availability for the stored read/write device. of course if you wanted to get really preventative buy a standalone commercial os that supports your drive for future use in a virtual operating system - or a version of linux that supports said drive.

That should really cover all the basis there for extreme data preservation.

Of course online backup at a trusted source should be considered as well.

But my rule of thumb is if its not worth that much to me I don't go out of my way to preserve it. However I do have several dvd and cd backups of my family pictures as a precaution. I should probably do another and finding a free online picture warehouse probably isn't a bad idea either (even though I have close to 2gbs which would take a while to upload).

[VegasBud]

For general consumer level archival storage, you can't beat dvd media with a hard coat. Both Verbatim and Taiyo Yuden have their own versions, with the Taiyo Yudens being the best deal. Just a quick check at rima, with no price shopping, shows they currently have them at $43/100 disks.

For something like hd video you've shot, where the size of the material makes the use of SL dvd media impractical, I haven't been able to figure out anything better than using esata/usb external drives. Hd-dvd would have been a good option for larger capacity archival optical media, but the asinine "format war" killed that possibility. Blu-ray is a poor choice for archival purposes.

At this point, I haven't been impressed with the reliability of flash drives, at least enough to consider it an archival medium.

Redundant archiving to both hard coat dvds and hard drives should be enough to let you sleep well at night, no matter how irreplaceable your material is.

Edit:
I just noticed that the disks I linked to a rima are thermal print, here are some inkjet printable (from supermediastore)...once again no price shopping, just the first one that popped up in google.

[Valnar]

Magneto optical is the most proven storage medium created with a life expectancy of 30-50 years. However, that doesn't mean the drives will be around then! Short of that, a RAID 6 array of hard drives with DVD-R backup would be a good alternative. That is what I use. It will mean replenishing the RAID array's hard drives and DVD-R discs every now and then, but I don't think that is what the OP wanted. If you desire a single form factor that can literally be stored away for decades and have a device readily available to read it many decades later, that won't happen. The closest possibility is DVD, but DVD±R writables are not proven to work that long. Although finding drives will be easy 20-30 years from now.... just like one can find VCR and cassette tape decks easily.

Intel has the MTBF of their X25 solid state drives at 2,000,000 hours. That might be the "next big thing" ™, especially as drives get bigger. Now whether we still use SATA interfaces 20-50 years from now is another question....

[Constant Gardener]

So... if seldom-used hard drives can fail with lubrication problems, how much should they be run and how often? I have some external drives that only get used for less than an hour, a couple of times a year. Do they need more exercise?

[yoda313]

Hd-dvd would have been a good option for larger capacity archival optical media, but the asinine "format war" killed that possibility. Blu-ray is a poor choice for archival purposes.

Just for the sake of technical arguments its still valid just that there won't be any new recordable media avaiable on a large scale. The existing information stored on hddvd is still valid of course provided you have a hddvd reader.

And another thing - why would you say bluray is poor for storage? Isn't it similar in structure to hd-dvd in that it is multilayered and smaller pits and ridges to fit more data than dvds?

[edDV]

So... if seldom-used hard drives can fail with lubrication problems, how much should they be run and how often? I have some external drives that only get used for less than an hour, a couple of times a year. Do they need more exercise?

I'd be happy to hear if you find any research.

[VegasBud]

yoda313,

You're right that hd-dvd could still be used for consumer archival storage. When the "format war" ended the wrong way (at least for my purposes), I seriously considered buying several years worth of hd-dvd drives and media, and using it anyway. In the end, it just seemed ill-conceived to lock into technology that went obsolete before the equipment was even bought...especially for archival use.

Here's the in-a-nutshell version of why blu-ray is not like hd-dvd (for archival purposes):

Hd-dvd was basically a dvd which used a blue-violet laser instead of a red one. Just like dvd, it had the data layer safely nestled in the middle of the 1.2 mm thick disk, with 0.6 mm of polycarbonate above, and 0.6 mm below. Hd-dvd only increased the numerical aperture slightly (from .6 for dvd to .65 for hd-dvd). Most of the increase in data capacity/density was a result of using the shorter wavelength blue-violet laser. Hd-dvd was a conservative first step into the new technology.

Since both hd-dvd and blu-ray use the same wavelength laser, the only way for blu-ray to gain an "advantage" in disc capacity was to increase the numerical aperture to .85, which gave them 25 gb/layer versus the 15 gb/layer capacity of hd-dvd. In order to make it work, though, the data layer had to be moved closer to the read surface...much closer. With blu-ray disks, the data layer is 1.1 mm from the top, and only 0.1 mm (for single layer, 0.075 mm for layer1 of a dual layer) from the read surface (the bottom). Besides providing substantially less protection for the data layer, that also increases read errors from dust, small scratches, fingerprints, etc., which is why initially blu-ray was in a caddy. Blu-ray in a caddy couldn't compete with bare disk hd-dvd, however, so the caddy was removed, and they applied a hard coat to the disk as a substitute means to reduce the problem of greater sensitivity to dust, scratches, etc., which is more a "cost effective" than "effective" solution. This increased susceptibility to "contaminants" (or defects) on the read surface is exaggerated by the much greater areal (data) density of blu-ray, which means that if there is a scratch, speck of dust, etc. on the read surface, it is not only more likely to be a problem, it also blocks access to a larger section of the data layer (more data is lost for the same size defect).

As an interesting historical note, Sony did try marketing a more robust blu-ray (in a caddy) for archival use, called Professional Disc for Data (or PDD). It didn't last long, and was discontinued.

In an article about using bare disk media for archival purposes, Steve Tongish of Plasmon put it more simply:

Plasmon’s experience with lower density bare DVD media and the recommendation from Blu-ray drive vendors for cartridged media strongly suggests that bare Blu-ray media is a dangerous proposition.

To keep everything out in the open, Steve Tongish was Plasmon's Director of Marketing Europe, and Plasmon's UDO was certainly a major factor in the demise of PDD, but to be fair, PDD was never much of a competitor to UDO, and since Plasmon also marketed bare disk DVD archival systems, I tend to think that he wasn't just bad mouthing the competition. At any rate, it's my personal belief that he was being accurate.

[meldavid]

I don't know how good the ink technology is now with DVD-Rs but I have lost some data that was burned on CDRs using brands like Sony, TDK, etc. after only 5 years because the ink just faded away over time.