I've been mulling this over for a while and the inner nerd in me finally got around to posting it.
I often wonder when posters moan about the high cost of hard drive storage or try to compress their video collection into the smallest (and lowest quality) file sizes.
I understand spending a relatively large ($100+) amount of money at one go is difficult for some, but compared to the thousands (possibly tens of thousands) of dollars spent on their disc purchases, a few hundred $$ for convenience is considered outrageous.
I'm sure there are many who will correct (hopefully politely ) any errors I've made in my calculations, but I think it's pretty clear that hard drive storage isn't as outrageous as many make it out to be.
A few points about my spreadsheet:
All prices are in US dollars based on known data as of 04/26/16.
The $35/TB cost for hard drives is actually on the high side. 2TB drives are currently available on Newegg and Amazon for as low as $25/TB. 4 & 8TB drives are averaging less than $30/TB
I didn't include DL Writable DVDs because I'm not familiar at all with their pricing (having never bought any) and AFAIK, they're premium priced for double the storage. In addition, according to Amazon, Verbatim DataLife SL DVDs can be a few dollars less than $25, so I think that doubling the slightly high SL price for the DL comparison is a fair compromise.
Also, I've never bought a writable BD-R either, but a recent post mentioned $0.75 for 25GB and $1.80 for 50GB and a quick check on Amazon confirms that this seems in line with the going price for these media.
The Non-usable space column represents space that typically wouldn't be used on that particular media. For the hard drive, I've deducted 2% allowing for a file that just won't fit on drive or for file organization (my WDTV media player requires at least 4-5GB for indexing).
For optical discs, I've deducted 10% to account for failed burns, files that don't completely fill the full capacity of disc and recommended no burning to the extreme outer edge of the disc.
Yes, my personal personal preference is for hard drives, especially as the bulk of my collection is leaning towards Blu-Rays vs DVDs. And yes, hard drives WILL FAIL. But that's why I only buy drives with two or three year warranties and swap them out just before or slightly after the warranty ends, either using them for non-critical data or selling for half of the going rate of a new drive (one of the advantages of buying larger, 4TB+ drives is they tend to hold their value better than smaller drives), reducing my replacement purchase cost.
+ Reply to Thread
Results 1 to 20 of 20
Last edited by lingyi; 28th Apr 2016 at 03:57. Reason: Just realized I made some major errors on my spreadsheet, recalculating. Upladed correct calculations.
Here the cheapest would be around $66.00AU for a bare 1 terabyte drive returning 37c per sl dvd stored ... allowing for 70gb unuseable and a further 10% for system.
Still with compression that's over 320+ sl dvds stored away from little fingers ... and without all those cases we get a bigger lcd tv to fit, lol
You seem to mixing up Decimal and Binary data units. Hard drives are sold/advertised with using the decimal system but Windows uses the binary system while still using the decimal units. Decimal units would be KB/MB/GB/TB and Binary units are KiB/MiB/GiB/TiB/so on.
I recently bought a 4TB HDD, and I got exactly 4TB of HDD space. When measured in the binary system it comes out to 3.63TiB. Windows refuses to change to binary units causing all sorts of confusion, and leading people to believe that the HDD companies rip people off.
DVD5s have 4.7GB of space (4.38GiB)
Helpful calculator http://www.gordonengland.co.uk/conversion/binary.htm
Last edited by KarMa; 28th Apr 2016 at 06:15.
Your figures are correct, the only thing you didn't add in with regards to HD costs (vr. Optical storage) is the cost of the raid enclosure and additional space required depending on what level (raid 5/6) one wants.
Current large scale raid systems (>80TB) are in the $100/TB range with raid 6/60, using 6TB drives in a 16 drive enclosure (thecus for example). Cost slopes downward as capacity goes up due to daisy chain system they have.
Smaller (7 bay) units are somewhat less, due to recent cost reductions, in cost per TB. Units are at $2500, figuring raid 6 = 30TB space so abour $85/TB. Simply divide that space by ~size per movie (25G or 50G) and you have some base figures. Smaller systems you'd be squabling over the slight differences, even in my calc here I'm rounding the space, but the difference between 30TB and the reality of 28TB is a bit of a wash. Both will hold some 1200 movies (at 25G/movie) so we're talking +/- a hundred.
I currently have 4 7bay units, so some 80TB using 4TB drives (largest avail at the time). Lust after 16 bay unit, may swap out if I can justify at some point.
Figures are appox. but close enough, before we start talking about Netflix levels of storage. But realize it took me close to 10 years to get to this point, running recode opns 24/7/365.
About the only figure that's off by a bit is that BD50s are continuing to slide, at bulk (>200+) are down to $1.60/ea at newegg. But that's in the 'wobble' range, really.
Last edited by Beck38; 28th Apr 2016 at 09:10.
I should have noted that I purposely converted to binary (vs decimal) in the 'Formatted size per disc' column forward to allow better comparison between the large per disc size differences between hard drive and DVD 5. In addition, this hopefully avoids confusion about why a 4.7GB (binary) file won't fit on DVD 5.
As stated in in my OP:
"The Non-usable space column represents space that typically wouldn't be used on that particular media. For the hard drive, I've deducted 2% allowing for a file that just won't fit on drive or for file organization (my WDTV media player requires at least 4-5GB for indexing).
For optical discs, I've deducted 10% to account for failed burns, files that don't completely fill the full capacity of disc and recommended no burning to the extreme outer edge of the disc."
I didn't include RAID (or any other type of backup) because my assumption is that the original disc is the backup.
Kinda like I said, in the whole gist of things, it's a bit of a wash. I do both the nas and optical so the 'backip is right in front of me, but restoring 25TB+ on one machine is a week worth if task (I had to do it one one machine several years ago when the raid5 'rebuild' function failed after replacing a bad drive). Tons of fun (not). Redid machine as raid6.
But I also did the usenet cloud since I had the bandwidth and machine to do it. Very low cost, and it's a distributed system with peering from multiple companies, so in my mind it's better than oaying for a 'cloud' storage company, I pay $20/month for unlimited and can stop that at any time and come back years later and it's still all there; so if house burns down/floods/tornado etc. I can rebuild, all I really need is my usb flash drive and those years of work are recoverable.
It's all what one wants to do, how much they're willing to spend. My biggest worry is that over the long term that the hardware piece of things will remain available and the landscape of s/w, h/w, and networks (local and internet) continues into the future.
My experience (and based on other reports) with downloading files (especially large files) from Usenet requires multiple servers (i.e. multiple accounts) as parts of the files are either corrupted or missing. Yes, PAR files can go a long way towards repairs, but you still need a large percentage of the numerous parts to successfully rebuild the original first. This is assuming the original uploading (posting) was complete in the first place. A fact that generally is unknown unless someone in that particular group has posted a message about it.
In addition, despite there being dozens of Usenet providers, there are only a much smaller number of server farms with most providers either a subsidiary or lessor of server space. When you post to one server, that data (possibly corrupted) is shared with the other servers.
Then there's retention. The longest binary (not text) retention that I know of is 6.5 years (~2380 days) and if you read the fine print on any provider you see that they don't guarantee 100% binary post retention (usually 97-98% for the top providers). When the year passes, the oldest binary posts are deleted and retention remains at 6.5 years.
Your 'data' regarding usenet is seriosly out of date. Most if not all premium providers have 8+ years retention, and haven't rolled off ANY data since ~aug2008. I pull out the first things I ever posted for long term storage 2-3 times a year, and it's all still there. You may be talking about 'non-premium' providers, not anywhere near the good ones; I also retest my uploads a day after in both the plant I post to and another on the other side of the planet. So, excepting a planet-busting asteroid hitting, I think I'm okay, and maybe even then.
That's the reality today. There is no data 'roll-off', or any other macinations going on, since Aug08, simply because hd space has gotten so cheap they simply slot in several new 8TB drives every day. Again, that's reality in 2016.
Of course, all the id's/passwords are stored on multiple raid nas bixes and non-volital memory devices, an the data is all encrypted with aes256 so that may be good enough (!).
One could believe, I guess, the occasional 'sky is falling on usenet' folks who come out of the woodwork every few months. HaHa. Not yet, not even close.
Last edited by Beck38; 28th Apr 2016 at 18:04.
Frankly, I have a hard time understanding your table, and simple arithmetic hardly qualifies as discovering one's inner nerd. But I digress.
If you want your "cost of storage" to qualify as "real", it is missing the cost of replacement when hdd's fail. Over the life of BD50, which costs about ~$0.05/GB here, I think a reasonable assumption would be to replace half of the drives once. So if I understand your table correctly, your cheapest hdd solution of $0.036/GB is really closer to $0.054/GB which is more costly than BD50.
I own that Verbatim spindle and have burned through about half of it, not a single coaster.
Several years ago the hard drive in my PC was beginning to run out of free space because I'd not been regularly burning discs. I worked out how many BD25's it'd require (I'd never use dual layer discs) and realised that was almost the price of a hard drive. So I bought three hard drives (in order to also have a duplicate of an existing hard drive) and I've never looked back. My hard drive collection has brown a bit since then, but of the ten drives I now use for backup (five 2TB drives plus another five 2TB drives with exactly the same data) the oldest was manufactured 2010 and I've only had one die and that was pretty early in it's life. The drives aren't used all that much, mainly just for backing up and playback now and then, so I'd be surprised if it's not a very long time until the next failure.
I just checked a drive manufactured in 2010 and it's total power on time was higher than I expected. I must have used them for playback more in the early days. 240.7 days total power on, but I'd be willing to bet the total power-on time for most drives wouldn't get close to averaging 6 months every five years, but at 6 months per five years it means if a drive has a three year power-on life expectancy.... well.... it'll take 30 years to achieve that. I've got drives used more regularly that've already exceeded 4 years power on time, so I don't think that's unrealistic (there's a 2TB drive in this PC with a total power on time of 5.3 years). Chances are if I'm still alive when the backup drives start dying I'll be happy for an excuse to replace them with whatever's the latest and greatest by then.
One thing's for sure, I don't miss burning discs. Especially when you're a bit OCD like me and prefer to check each one for burn quality. It's a very slow process.
There's definitely one advantage to burning discs. Even when I update existing video with newer, higher definition or better quality versions, I'd never throw the old discs away. So my huge collection of obsolete discs that'd otherwise be a complete waste of time and money are at least a bit like having a time machine in that respect, whereas for hard drives I delete the old and replace with the new. Not that I've had a need to go back in time all that often, but still, that's one advantage of burning to discs.
For many people having a copy on a hard drive and burning a copy to disc might be a reasonable compromise. I used that method of creating two backup copies for quite a while, but I sure don't miss burning discs. Even if having to store everything on two hard drives makes the hard drive option more expensive., it's still much faster.
Last edited by hello_hello; 29th Apr 2016 at 20:10.
One was a 20GB Seagate drive that I found in the trash a decade ago, made in 2001. This drive never acted funny and sat in my closet for many years without being used more than once a year or so. A few months before the failure I decided to use it to back up family photos, and so filled it up with that. Just to give the drive purpose. On the day of the failure, one of circuit chips started to smoke and then a small area of the chip turned red hot, on the bottom of the HDD. Turned out a bug had died on that chip awhile ago and so his body managed to short circuit the HDD board. If I cared to, I could probably buy a new board and get it working again. But it was a back up so I don't care.
The second drive that died that faithful day was a 120GB Western Digital, made in 2002. It was the boot drive to a computer I got in 2003 and it served for 5 years doing just that until I moved on to another computer. It was mostly full of digital junk, stuff I don't want to delete but also don't care to move on to a new drive. Anything of importance had long be moved out. A few months before the failure it worked fine and was giving me impressive speeds for such an ancient drive. On the day of the failure I plugged in the 12V and nothing happened, it just gives off a quiet high pitched sound. It no longer spins up. I can hear the platter move when I physically rotate the HDD.
These are the only drives I have ever had completely fail on me. Before this I only had minor problems, which could usually be fixed.
I had powered both of these drives with a old PSU, that had been sitting out after it started failing in my computer and was replaced. Did this failing PSU cause one or both of these drives to fail, idk maybe. Certainly did not help.
Anyway your 50% stat is crap.
http://www.newsgroupreviews.com/best-usenet.html Best Usenet Providers of 2016, five of the top seven* recommended (and to my knowledge top Usenet Providers) offer 2800 (7.8+ years) retention (based on the actual website info).
*Easynews is a sister site of Giganews and shares servers.
Newshosting – 2814 days
UsenetServer – 2124 days
Supernews - - 2357 days
EasyNews – 2815 days*
Eweka – 2811 days
Giganews – 2367 days*
Astraweb – 2813 days
Thundernews – 2815 days
However I stand by my statement that just having X days Binary Retention does not guarantee availability of files as having used all the providers listed below except Eweka, plus several others, I have never been able to download certain files without having to utilize a second provider for post fills and have never been to able retrieve certain other files regardless of providers.
I'm glad that your experience with Usenet has been positive and would love to know what provider you're getting such great service.
As we've already strayed off the original topic enough, I'll leave the last word to you if you wish.
Last edited by lingyi; 30th Apr 2016 at 00:27. Reason: grammar
As far as hard drive failure, I agree there's a certain probability factor that I addressed in my original post by acknowledging it and my PERSONAL strategy for cost reduction.
However, not to be dismissed is the certain probability of optical disc failures due to bad burns (known or not), scratches, breaks or dye breakdown among others (insert fungus joke <here> ).
As demonstrated by your post and those just below, there are a multitude of "what if" factors (e.g. electricity, storage space, backups of backups, convenience, etc) that can skew the numbers up or down. Whether these factors affect the individual's choice of backup media can only be answered by that individual.
It's all up to what one's pain level is; I use Antec enclosures exclusively and easily 'swap out' 2TB drive caddies when I'm 'in production' on things, which is basically all the time.
But the once in 15yrs that I had a failure of a raid nas, it took about two weeks to recover (and that included the shipping time for the replacement drive, I now have two spares sitting on the shelf backing up 28 drives that are running) so the actual repair time is more like 3days now. Minimal pain. The use of usenet is simply the cheap 3rd tier if the optical backup fails (I pull discs at random 2-3 times a week and watch movies 'just fer fun').
Really minimal hassel, but gives me just enough warm fuzzies knowing that the last 15 years of work has not gone to waste, and I don't have to rely on commercial internet streaming to show me any of my 'favs' any time I want.
I keep hard drive docks on the floor now because dropping a drive will likely kill it, and a dock or USB hard drive on a desk is just asking for trouble if you own a cat. The one she knocked off my desk died, so I learned my lesson.
Last edited by hello_hello; 1st May 2016 at 00:24.
At any rate there is a fair amount of hedging in your response. Which one do you recommend? It sounds like it is not the cheapest $/GB, to which I say, exactly. If you care about your data, you should NEVER choose the rock bottom cheapest option.
As for hdds and in order to appease the angry hordes, let me just say, they are good for temporary storage if you have insufficient ssd capacity.
That said, to the extent that my recommendation carries any weight (which I acknowledge is worth no more than one anyone else's opinion) is one set of HDDs in external cases, a backup of that set in my PC and replacement / upgrade every 2-3 years.
Note: Yes, this does double my HDD cost, but for ME, that is a monetary cost offset by the cost of time/inconvenience of having to re-RIP the discs in the possible event of double HDD (main and backup) failure.
Last edited by lingyi; 1st May 2016 at 12:06.
SSDs are fairly irrelevant to this discussion. Unless you're using SSDs for permanent file backups, which I kind of doubt, given they still don't compete with hard drives for price per GB.
Last edited by hello_hello; 1st May 2016 at 13:41.