HDD reliability

[hello_hello]

I had my first Raid-0 volume failure yesterday (consisting of 2x WD 320GB drives). Aside from one that failed years ago due to brand new drives being faulty. I had a pretty good idea one of the drives was on the way out so I was extra careful with backing up. It started dropping hints a few months ago (transfer speed slowing temporarily and a couple of BSODs). Still, they're about 7 years old with a total power on time of over four years (could be 4.5 by now, I haven't checked the remaining drive yet). Not bad for a couple of standard WD drives with a 2 year warranty. Not as good as the other drives, all purchased at about the same time and all with similar usage. 4x Samsung 500GB and 2 more WD 320GB. All with only a 2 year warranty and all still working. I'm starting to replace them all with newer, faster drives now, but I'll keep them for extra storage etc.

Ironically, the only drives I've purchased (until recently) with a 5 year warranty were a pair of drives that needed a 5 year warranty. Damn Seagate 500GB Barracudas. Mind you I can't fault Seagate's customer service, just the need to keep returning them. I still only use them for external storage, and always with the same files copied to both drives, because even sitting idle they still make "odd" noises at times.

[SameSelf]

Missed mentioning this because they weren't drives that have failed after usage, but yeah, should have included this in my post as a big drive killer - shipping.

I get about almost 10%-15% DOA - very frustrating. Yes, I always get them replaced, but it's a real pain trying to make them work, finding out sometimes after hours of testing or overnight failed formats they don't work, sometimes even losing a complete backup, etc. Sometimes it's not obvious right away that they're dead, because they're "only defective" at first.

Then, requesting an RMA, printing the label, repacking it, driving to the nearest courier station, waiting again for the replacement, testing again, etc, etc, yaddy-yah - It feels like a part time job to do all this work!

10%-15% DOA is not surprising. And, yes, while it is super annoying going having to go through the RMA process when a brand new drive only makes strange clicks, if that was the only problem, I would probably still buy all my drives online. However, it is the lurking, unseen problems from shipping abuse that only show up some months down the road that worry me more. When a store bought drive fails, the RMA process unavoidably results in the mfr shipping a new drive to you anyway, so shipping can't be completely avoided, sadly. Needless to say, I am worried about one drive in my arsenal that is an RMA replacement.

The good news is with 4 TB ssd's on the market now, spinning rust is going the way of the floppy drive. Just like my son who recently asked my why the computer starts with the letter c:\ drive, the next generation will ask, what is a hard drive, daddy?

[SameSelf]

I had my first Raid-0 volume failure yesterday (consisting of 2x WD 320GB drives). Aside from one that failed years ago due to brand new drives being faulty. I had a pretty good idea one of the drives was on the way out so I was extra careful with backing up. It started dropping hints a few months ago (transfer speed slowing temporarily and a couple of BSODs). Still, they're about 7 years old with a total power on time of over four years (could be 4.5 by now, I haven't checked the remaining drive yet). Not bad for a couple of standard WD drives with a 2 year warranty. Not as good as the other drives, all purchased at about the same time and all with similar usage. 4x Samsung 500GB and 2 more WD 320GB. All with only a 2 year warranty and all still working. I'm starting to replace them all with newer, faster drives now, but I'll keep them for extra storage etc.

Ironically, the only drives I've purchased (until recently) with a 5 year warranty were a pair of drives that needed a 5 year warranty. Damn Seagate 500GB Barracudas. Mind you I can't fault Seagate's customer service, just the need to keep returning them. I still only use them for external storage, and always with the same files copied to both drives, because even sitting idle they still make "odd" noises at times.

I have had plenty of WD drives fail inside the warranty. Our IT department has a shelf full of WD drives too. Nothing particularly special about WD vs Seagate, or anyone else for that matter. Every mfr experiences failures. It is as simple as that.

[hello_hello]

Sadly, I had another BSOD today, so the dying hard drive wasn't responsible for that as I thought. I now suspect one of the DVD burners.......

The good news is with 4 TB ssd's on the market now, spinning rust is going the way of the floppy drive.

Aren't hard drive platters made of glass or aluminium?
Mind you I'm looking forward to HDDs being completely replaced by SSDs, but I think we're still a ways off yet. It'll be nice to be able to drop a drive and not kill it.

[PuzZLeR]

For the longest time, since 2008-ish, I would buy those slick shiny black Neil Poulton design LaCie externals. Not only were they gorgeous, and look great in a computer room, also - I have never, ever, ever had a problem with these, even after much abuse. Even today, all 10 or so still work. (However, never had one DOA since they've always been available locally, so can't comment on this.)

I would honestly have them as my highest recomendation for this thread, but for some unholy reason they don't make them anymore, and I don't know why.

Since Neil Poulton started to design these ugly (IMO) orange "rugged" ones for LaCie, I've only been buying internals on a rack and dock since.

If these orange ones are anything like the LaCie's I've bought, and you can get over the appearance (of which someone may like) then they are my recommendation. (Not sure exactly what brand is under the hood.)

[SameSelf]

Aren't hard drive platters made of glass or aluminium?

You said "aluminium" (chuckles to himself).

[hello_hello]

Aren't hard drive platters made of glass or aluminium?

You said "aluminium" (chuckles to himself).

Yeah, I'm not American. I live in an English speaking country. One where words such as quantise are spelled correctly and where aluminium doesn't rust.

[PuzZLeR]

Both spellings, aluminum/aluminium, and even different pronunciations, are valid depending on where you live. But I would think the chemical properties would be the same.

Sometimes I don't know which spelling to use for the word color/colour here since this is an international forum.

[hello_hello]

Two dead hard drives in the space of a week. What are the odds? It's not like I own hundreds of them.

Today's dead drive was a 2TB WD Green drive. I have quite a few of them, manufactured from 2010 onwards. They're mainly used for external backup storage so they haven't had much use.

Computers are vicious bastards. Vindictive, horrible creatures. I copy everything to two external drives so I have a backup copy. A few days ago I bought four more 2TB WD Green drives as I was completely out of free space, and consolidated/organised files more logically while copying data to them. After I filled up the new drives, I reformatted the old ones and copied the data back from the new drives. Only one drive having the "re-organised data" copied to it wasn't new. I plugged it in today to copy the files to a second drive.... dead as a Dodo. No warning at all. I'd reformatted all the old drives and copied files back to them except one. Fortunately, due to some cosmic roll of the dice, the one I hadn't reformatted was the backup drive containing most of the data on the dead drive. That was sheer luck. I hadn't been reformatting the old drives in any particular order. If I'd reformatted a different drive last, I would have lost nearly 2TB worth of files.

Fortunately it's still under warranty, but a 1.5 year old drive picking the time I was data shuffling to die......
Not during the two years prior, not any time after, but during the small 24 hour window where all my files weren't residing on two hard drives because I was moving them around. Computers are such spiteful creatures.

[SameSelf]

I feel your pain, man. Welcome to the world of modern drives. "They just don't build them the way they used to." That is why I have punted on spinning rust. As far as I am concerned, if your data isn't stored on optical media and/or tape, your data is NOT BACKED UP. Exigency makes converts of us all. Having multiple copies of data across as many hard drives is just that, copies only and potentially corrupt copies. When you were filling up the new drives, did you bother to do a checksum on the data? How do you know no silent data corruption has crept into your data? Also, those four new drives need to be burned in for at least three months before you can safely say you are out of the infant mortality window. Spinning rust is just not worth the effort anymore when dealing with TB+ amounts of data.

I have all my data backed up to dual layer BD-R and tape. None of it resides on spinning rust anymore since DL BD-R became affordable. In fact, I suddenly have a surplus of hdd space I don't need anymore. It is kind of a strange but freeing feeling. Since BD-R and tape is WORM technology, I never have to worry about moving vols again. The data was checksummed, verified, and carefully cataloged when written out. In fact just last night, I was looking for a video that I am in the process of remastering. Found it quickly on the blu-ray my catalog told me it should be on. Then loaded the video off the tape. All the checksums verified. It was a flawless process. When I am done remastering the video and authoring the blu-ray, it will be deleted from the hard drive. My hard drives are nothing more than glorified OS drives.

I invite you into this stress and worry free world.

[ron spencer]

what tape drive? can you explain how it all works...I've never used one

[usually_quiet]

Yeah, I'm not American. I live in an English speaking country. One where words such as quantise are spelled correctly and where aluminium doesn't rust.

Aluminum doesn't rust, but it still oxidizes and corrodes. The thin transparent layer of aluminum oxide that forms naturally (and immediately on contact with oxygen) is somewhat protective, but not very helpful against salt water or dishwasher detergent. ( I have a lot of pitted and dull medium-gray aluminum items in my kitchen. ) Fortunately hard drives are rarely exposed to such caustic solutions.

[solarfox]

I see a lot of smug dismissals of "spinning rust drives" in this thread. Well, let me tell you... SSDs aren't exactly the greatest thing since sliced bread, either. I've had them die on me without warning in as little as 4 months -- and I mean, without any warning at all. One day, the PC ran fine; the next day, it was un-bootable and the SSD can't even be recognized by the motherboard. Say what you will about traditional hard drives, but I've never had one suddenly go from "fully-functional" to "stone-dead" literally overnight without at least some warning signs (sluggish access, unexplained read/write errors, increased noise, etc.) prior to failure. And these were supposedly good drives from outfits like Samsung and Intel, not bargain-basement drives from fly-by-night off-brands...

[usually_quiet]

Both spellings, aluminum/aluminium, and even different pronunciations, are valid depending on where you live. But I would think the chemical properties would be the same.

Sometimes I don't know which spelling to use for the word color/colour here since this is an international forum.

Most members here accept both British and American spelling/usage as correct. I say use whatever you were taught.

[hello_hello]

I feel your pain, man. Welcome to the world of modern drives. "They just don't build them the way they used to."

I'd still disagree with that. Sure there's been problem models, but as a general rule I think hard drives are more reliable now than they were 10-20 years ago.

That is why I have punted on spinning rust.

Yeah, but if you're going to describe something in a derogatory manner you should try to make it technically accurate otherwise it detracts from the point you're trying to make.

As far as I am concerned, if your data isn't stored on optical media and/or tape, your data is NOT BACKED UP. Exigency makes converts of us all.

We all have a line where we consider our data to be backup up, and it's probably different for everyone. For someone else, your data burned to optical media isn't backup up unless it's off-site and stored in a temperature controlled bunker capable of surviving a direct nuclear missile strike.

Having multiple copies of data across as many hard drives is just that, copies only and potentially corrupt copies. When you were filling up the new drives, did you bother to do a checksum on the data? How do you know no silent data corruption has crept into your data? Also, those four new drives need to be burned in for at least three months before you can safely say you are out of the infant mortality window. Spinning rust is just not worth the effort anymore when dealing with TB+ amounts of data.

No I've never done a checkum. There's not been a need before, so I really don't see a reason to do so. I've got 15-20 year old files, that have been copied and copied and copied, but they still open without errors. Some were probably originally created when the largest hard drive you could buy was something like 2GB and if you wanted to back them up you used floppies. I remember having to split large files so they'd span multiple floppy discs. I've also got plenty of old DVDs with burned data, so I might do a checksum on a few files at some stage, just out of curiosity. How often do your checksums fail because the data was on a hard drive? It sounds a bit like a storm in a teacup to me.

I'm aware new drives need to be "burned-in". Many of my backup drives have had so little use it'd probably take much longer than 3 months. When they're new and I've finished copying data I tend to leave them sitting in a dock and running for a few days to help speed up the process. The 1.5 year old drive that died yesterday could very well have suffered from infant mortality despite it's age, but that's why I tried to make sure all the data was saved to one of the new drives and then copied back to an older one.

I have all my data backed up to dual layer BD-R and tape. None of it resides on spinning rust anymore since DL BD-R became affordable. In fact, I suddenly have a surplus of hdd space I don't need anymore. It is kind of a strange but freeing feeling. Since BD-R and tape is WORM technology, I never have to worry about moving vols again. The data was checksummed, verified, and carefully cataloged when written out. In fact just last night, I was looking for a video that I am in the process of remastering. Found it quickly on the blu-ray my catalog told me it should be on. Then loaded the video off the tape. All the checksums verified. It was a flawless process. When I am done remastering the video and authoring the blu-ray, it will be deleted from the hard drive. My hard drives are nothing more than glorified OS drives.

That all sounds quite nice, and for years I burned everything to DVDs, then to Bluray, but after one backup session involving (literally) sitting at the computer for the best part of a day while burning discs and then checking burn quality, often two or three at a time, I decided I'd had enough of optical discs. So I bought enough hard drives to duplicate the existing ones, copied all the data to them and never looked back.
Sure, it's not as reliable as burning to disc. I proved that to myself yesterday, but from now on my files will always be on two drives even while I'm shuffling them around, and even if I've got to buy an extra drive or two in order to have enough space to ensure it.
Giving up burning discs was a liberating experience for me. I hate it. It's a slow and tedious process and I've run burn quality tests on more discs than I care to remember. No doubt about it though, the "permanency" of discs has it's advantages. I still dig out old discs to retrieve files now and then. Files I'd no longer have if I'd always just backed up to hard drives as they would have been deleted and replaced. Discs on the other hand.... I don't throw them away. They're like having a time capsule in that respect. Still.... the amount of time I don't have to spend burning discs is a trade-off I've not yet regretted, even though maybe I came close yesterday.

[SameSelf]

I see a lot of smug dismissals of "spinning rust drives" in this thread. Well, let me tell you... SSDs aren't exactly the greatest thing since sliced bread, either.

Agreed. That is why I have not recommended SSDs in this thread as a legitimate form of backup. They are great for performance but worse than spinning rust for data archiving/backup for numerous reasons I don't have the bandwidth to discuss. But, hard drives aren't even good for performance let alone data archiving/backup. That is why they will go the way of the floppy soon.

[SameSelf]

what tape drive? can you explain how it all works...I've never used one

It is an expensive option unfortunately. But a simple solution is an HP Ultrium External Tape Drive which run about $1,500 (your choice of SCSI/SAS which can up the cost more if you don't have an interface card). The LTO cartridges are, fortunately, fairly inexpensive. You can also minimize costs if you are willing to go with an earlier standard like LTO-3 or LTO-4. The current is LTO-6 while LTO-7 is slated for 2016. So maybe in 2016 the excess LTO-6 tape drive inventory will be going at a discount. However, if you are willing to spend hundreds on spinning rust, at some point, why not $1,500 for an external LTO drive?

Tape drives are serious stuff and there is a reason Google and IT managers who don't want to get fired use them. https://www.youtube.com/watch?v=74l96vMPI5c

[SameSelf]

No I've never done a checkum.

I recommend you start right away. I have checksum libraries of all my media files. I never copy/write/migrate data without checksum verify.

[hello_hello]

No I've never done a checkum.

I recommend you start right away. I have checksum libraries of all my media files. I never copy/write/migrate data without checksum verify.

As I've already asked, that'd be because the checksum has revealed files have become corrupted due to being stored on a hard drive, how often?

The problem there is, if a file on my hard drive has already become corrupted, how would a checksum while copying it now let me know? The other problem is, it's time consuming. Copying large files to a 2TB WD Green drive must take something like (I haven't timed it) six hours to fill a drive from empty, then it'd probably take a similar amount of time to verify them all. I don't mind as long as it's not looking for a problem that doesn't exist.

I'm open to trying things though. What do you use for checksum verification? I have TeraCopy installed and it's easy enough to tell it to verify after copying, but I don't think it'll save the checksum data for later use. I'm not sure. However, I still have a couple of drives containing duplicate files I haven't messed with recently. Some of those files would have been burned to DVD discs several years ago and some on the drives would have been updated over time, but the majority would have sat on the drives untouched for a couple of years. Let me know what you use, and if it's not too time consuming I'll compare every file on both drives. I'll also dig out several old DVDs and if I still have the same files, compare the ones burned to them. If I find any files that don't have matching checksums I'll give serious consideration to joining the church of checksum verification. If I don't.... I'll probably just add another layer of lining to my PC's tin foil hat.

[Nelson37]

Wow, this is amazing!

You mean that I can buy a $1,500 LTO tape drive, plus cartridges, to back up a couple $70 TB hard drives? Incredible! Even better, if I have all those cartridges, and the $1,500 LTO drive fails, which it will eventually, the ONLY thing I have to do is spend another $1,500 to get my data back? I'll get right on that. I left tape drives, including LTO, behind a long, long time ago.

Or, I could buy a BD burner, and 20 blanks, and do that same backup in what, a day or two? BURNED, optical media? Goodie!

Avoid expensive, proprietary hardware, and also avoid oddball, free (meaning likely to change or disappear) file storage systems. Buy a drive that can plug into any PC and be read by standard hardware, immediately. I use and recommend WD Caviar Blacks, SFAIK the only consumer drives with a 5-year warranty. $70 for a TB of reliable, high-speed, immediately accessible data in standard, usable format. That's 10 TB, with backup, for less than the cost of a tape drive with no cartridges.

Drives use many types of coating for the platters, SFAIK iron oxide is not that common any more, also SFAIK the coating is the same for most tape types. HD is on a much more durable media.

On hard drive cooling, i use either heat sinks for passive cooling or fans mounted at some distance and on a different power connector. IMO, fans mounted directly on the drive and/or sharing the electrical connector with the drive introduce unwanted electrical noise.

In my experience, corrupted files outside of a failing drive or storage system are extremely rare, not counting database corruption which is not the fault of the drive and would not be detected by any checksum procedure.

[SameSelf]

As I've already asked, that'd be because the checksum has revealed files have become corrupted due to being stored on a hard drive, how often?

The problem there is, if a file on my hard drive has already become corrupted, how would a checksum while copying it now let me know? The other problem is, it's time consuming. Copying large files to a 2TB WD Green drive must take something like (I haven't timed it) six hours to fill a drive from empty, then it'd probably take a similar amount of time to verify them all. I don't mind as long as it's not looking for a problem that doesn't exist.

I'm open to trying things though. What do you use for checksum verification? I have TeraCopy installed and it's easy enough to tell it to verify after copying, but I don't think it'll save the checksum data for later use. I'm not sure. However, I still have a couple of drives containing duplicate files I haven't messed with recently. Some of those files would have been burned to DVD discs several years ago and some on the drives would have been updated over time, but the majority would have sat on the drives untouched for a couple of years. Let me know what you use, and if it's not too time consuming I'll compare every file on both drives. I'll also dig out several old DVDs and if I still have the same files, compare the ones burned to them. If I find any files that don't have matching checksums I'll give serious consideration to joining the church of checksum verification. If I don't.... I'll probably just add another layer of lining to my PC's tin foil hat.

I missed your reply so sorry for the late response. Try this tool for checksum verification:

fciv

It is a cli, but it's free and easy to use. And unfortunately you are correct. Doing a checksum on 2 TB of data would probably take all day. Checksums are a pain on any non-ZFS volume.

My workflow basically goes like this. Capture footage on an SSD in my Atomos Ninja. Before transferring to my pc hard drive for editing, I run a checksum on the ssd file. Copy the file over, then run a checksum to verify the copy was exact. When I accumulate a large enough chunk of files (doesn't take long), write out to WORM backups. Run a checksum verify on the written out files. Then never worry about running checksums again unless I copy data off the WORM backups to verify the copy was exact. As for moving vols around from hard drive to hard drive, I don't do that anymore. Incremental backup software? Garbage. External spinning rust backups? Garbage. I don't do it anymore.

[SameSelf]

I hate to dig up an old thread. But this thread perfectly illustrates why I hate hard drives now.

https://community.spiceworks.com/topic/424891-best-long-term-backup-dvd-bluray-hdd