External hard drives for archiving captured avi?

[Leanoric]

Apologies if this should have been posted in the computer section, but I wanted to ask you capture guys what external hard drives you use for archiving video files.

My internal hard drive on my pc is quickly filling up and its time for some additional storage. I’m thinking a 4-8 tb external, but not sure what to go with. I know that hdd is recommended over sdd, but I’ve read about wd and seagate, the most common ones being unreliable and hard to retrieve data from if they go bad, apparently sata drives are preferable?

I’ve also seen docking stations that hard drives are inserted into, being recommended over externals.

What do you fellas use?

[dellsam34]

I don't think brand matters but I personally use NAS internal 3.5" drives, They are built better to withstand heat, vibration and long hours of run, So for your purpose of just backing up they will not see a lot of use so they should last for a long time, Get a 3.5" docking station like this one that I just ordered few days ago along with a 18TB Seagate HDD, What a coincidence. Format the whole volume in NTFS with GPT not MBR and name your volume to your liking.

[Leanoric]

Ok thanks, I’ll go with the external and docking station as well then. I see that they sell the docking stations with and without fans, do you think a fan is needed?

[dellsam34]

No, Not when you buy a NAS internal drive, I had mine run for several hours in a large backup and it was barely warm to the touch, If you buy a regular internal HDD you may need a docking station with a fan or an aluminum case for better heat dissipation. You can also get an external hard drive with an enclosure but expect to pay more than what an enclosure cost when you can do that yourself.

[Sharc]

@ the OP: What exactly do you mean by archiving? Just more storage space? What are your expectations regarding data safety and availability? How important are your video files? Keep in mind that even an ultra reliable device with impressive reliability figures may fail within the next minute (random failures). Means you should also think about backup solutions with adequate redundancy and diversity. An external HDD or NAS etc. may be stolen (burglary) or damaged by fire etc. if located at the same location, e.g. in your home. You may therefore also consider an external cloud service or equivalent as a backup (archive) for your precious data. It all depends on your expectations, just think about it.

[Leanoric]

@ the OP: What exactly do you mean by archiving? Just more storage space?

Yes I suppose I mean just more storage space. I’ve quickly filled up my pc hard drive and have many more tapes that I’m going to work my way through.

How important are your video files? Keep in mind that even an ultra reliable device with impressive reliability figures may fail within the next minute (random failures). Means you should also think about backup solutions with adequate redundancy and diversity. An external HDD or NAS etc. may be stolen (burglary) or damaged by fire etc. if located at the same location, e.g. in your home. You may therefore also consider an external cloud service or equivalent as a backup (archive) for your precious data. It all depends on your expectations, just think about it.

Yes thats a good point, many of the tapes arent massively important, but some of them are, old family footage, weddings etc. Hard drive failures are what worries me, especially when reading some of the reviews on the externals. I hadn't really thought about cloud but I’ll look into it. It would be worth having just for the more important stuff.

[Sharc]

Yes I suppose I mean just more storage space. I’ve quickly filled up my pc hard drive and have many more tapes that I’m going to work my way through.

Also, if you don't plan for redundancy in some form, you take the risk of a 'single point of failure'. If so, think about the capacity of the storage device: For a 1TB device you will loose at maximum 1TB of data when it fails, for a 10TB device you may loose up tp 10TB of data in case of a device failure.

[Alwyn]

For a 1TB device you will loose at maximum 1TB of data when it fails, for a 10TB device you may loose up to 10TB of data in case of a device failure.

The problem with that is that you end up with many HDDs to cover what could be 8tb of data or more. Better to have two big backup drives, in addition to the drives with the working files in your computer/at your workstation.

@Leanoric, can you put more HDDs inside your box? I have 3 big SATA HDDs for my videos in mine, in addition to my normal data drive and my system C drive. I have two backup drives to give me two copies of those three video drives. A RAID system would be the best, but that's too brain-intensive for me.

[Leanoric]

@Leanoric, can you put more HDDs inside your box? I have 3 big SATA HDDs for my videos in mine, in addition to my normal data drive and my system C drive. I have two backup drives to give me two copies of those three video drives. A RAID system would be the best, but that's too brain-intensive for me.

I dont think that I can. To be honest, I had not owned a pc for 25 years since leaving school, so im not too computer savvy. When I decided to start capturing my old tapes, I just went out and bought the cheapest mini pc I could get. With hindsight, I should of done more research and bought a better pc, one thats more powerful and can be upgraded, although to be fair, it seems to be coping quite well with capturing so far.

I’ve ordered a dual bay docking station and 2 internal sata 4tb drives for now. I’ll keep a copy of each video on both drives, and then maybe get a couple of bigger ones when they start to fill up.

I just had to google raid drive, it would be nice, but way above my capability lol.

[Sharc]

I’ve ordered a dual bay docking station and 2 internal sata 4tb drives for now. I’ll keep a copy of each video on both drives, and then maybe get a couple of bigger ones when they start to fill up.

That's reasonable to begin with. The single point of failure will be the common power supply though, which hopefully will not corrupt the data on both drives simultaneously when it fails.

[Leanoric]

I’ve ordered a dual bay docking station and 2 internal sata 4tb drives for now. I’ll keep a copy of each video on both drives, and then maybe get a couple of bigger ones when they start to fill up.

That's reasonable to begin with. The single point of failure will be the common power supply though, which hopefully will not corrupt the data on both drives simultaneously when it fails.

Hopefully not. Your making me paranoid about having them both plugged in at the same time now lol. I wont be leaving them plugged into the computer though, just when I need to save or retrieve a file.

[dellsam34]

Yes, Always have each hard drive powered individually, My hard drives have separate docking stations, I've had to replace a docking station because the HDD would disconnect during a file transfer, I quickly decommissioned it before a serious problem occur. Ideally you would have to have a UPS or power backup too if you live in an area where power surges happen frequently. Again just like capturing it's a giant rabbit hole, Your question was specific that's why I didn't dive into all this. It is a slim chance that you will have a fail if you just occasionally power up the drive compared to a computer HDD that is always running but it doesn't mean that you're safe.

[Leanoric]

My two drives turned up today. I’ve read conflicting info though, on what unit allocation size to go with when formatting.

Its going to be mainly large avi files stored on them, I went with 64kb, will this be ok?

[Cornucopia]

Choice of sector size is a matter of efficiency. IOW, if you choose wrong, nothing bad happens, it is just less efficient. If the drive is intended to primarily store large video files, then the sector size should be toward the large end in order to make more efficient use of the space.


Scott

[Leanoric]

Ok thanks

[dellsam34]

I always leave at default.

[jagabo]

Drive usage is based on "allocation units". The OS maintains a map of which allocation units are used by files and which are free. The smaller the allocation units are the more of them it takes to represent the entire drive, so the bigger the map is. But each allocation unit can be used by only one file. If a program writes one byte to a file on a drive with 4KiB allocation units, 4KiB of drive space is consumed. The rest of that allocation unit is available to that file but not to any other file. If more data is written to the file it is written to that allocation unit -- until it grows larger than 4KiB, at which point another allocation unit is given to the file. When the file size crosses 8KB another is consumed. Etc. On average, with a random assortment of file sizes, half an allocation is wasted by each file.

So the optimal allocation unit size is a balance between the drive size and the file sizes. For example, if you know in advance you are going to write millions of 4096 byte files then 4096 byte allocation units is optimal. But since one generally doesn't know in advance what all the file sizes will be it's best to just let the OS decide on the allocation unit size.

[lordsmurf]

Drive usage is based on "allocation units".

In Linux, inodes.

[Leanoric]

How much free space do you guys recommend leaving on hard drives that are only used for storing video files? I've read that 15-20% should be left free so that windows can operate properly, but as I dont have windows on these drives, just wondering if its ok to go higher than that.

Also, they are 4tb NAS internals, but even when they were empty, they only had 3.6tb of free space, should I aim to leave say 15-20% of 3.6tb free or 15-20% of 4tb free?

[jagabo]

If it's not your boot drive there's no need to leave space for Windows.

Drive manufactures and Microsoft just measure drive size in different ways. To a drive manufacturer 4 TB is 4 * 1000 * 1000 *1000 * 1000 bytes, or 4,000,000,000,000 bytes. To Microsoft 4 TB is 4 * 1024 * 1024 * 1024 * 1024 bytes, or 1,099,511,627,776 bytes, in more modern nomenclature 4 TiB. So to Microsoft your 4,000,000,000,000 byte drive is about 3.638 TiB.

[Leanoric]

Ok thanks for the explanation Jagabo.