What's the best storage option for a large VHS-to-digital project? I initially got an 8TB external, thinking that would last at least a good chunk of it, but I've run out of space almost immediately.
I also currently have no backups, which isn't ideal.
EDIT: It's important to pay attention when using multiple tabs for different subforums. Oops. Meant to put this in Newbie/General.
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 1 to 30 of 124
Thread
-
Last edited by koberulz; 10th Apr 2018 at 11:43.
-
Maybe a external box with several internal HDDs. Use USB 3 or eSATA connections.
You could also add RAID, but that would eat up a lot of drive space.
Just an example, not a suggestion:
https://www.amazon.com.au/TerraMaster-D5-300-External-Enclosure-Diskless/dp/B01DW1X348...ive+enclosures
I'll move this thread to Newbie/General Forum. -
+1 to redwudz suggestion of an external box.
I've been using the non-RAID version Mediasonic Probox(es) for years http://="https://www.amazon.com/Mediasonic-ProBox-HF2-SU3S2-SATA-Enclosure/dp/B003X26V...dp/B003X26VV4/ and highly recommend them. There's also RAID versions, though I haven't used them.
At least in the U.S., 8tb HDDs are the sweet spot for price (<$20/tb). As discussed in numerous threads, at the risk of voiding the warranty, it's highly recommended to remove the drives from the case due to issues with overheating and USB interface failure.*
*The WD Easystore drives are really easy to open without breaking any internal tabs and at least through the end of 2017 contained WD NAS drives.
Currently the only other viable options are optical discs (Blu-Ray) and cloud. Neither as convenient or cost effective and HDD storage.
Also, BACKUP, BACKUP, BACKUP! I've had multiple gigs of data disappear in an instant many times and while it's no fun restoring a backup, it's better than having to start from scratch. -
What video format and what resolution are you using? I use DV to transfer and store VHS video. It is a constant 13 GB/hour. Basic math says that your 8 TB drive will store 615 hours of video. That is over 300 2-hour cassettes, each filled all the way up. Perhaps your project is bigger than that.
Many people now archive using long-GOP delivery formats, like h.264, and you can actually get very good quality at less than half this size, which means you would get well over 1,000 hours onto 8 TB.
So, it would be interesting to know how much space one hour of your video is consuming, and if it is more than these numbers, perhaps you should reconsider your workflow.
And yes, without backups, you have nothing, because eventually the primary will fail.
Without knowing how many hours or how many VHS cassettes are involved, my final recommendations may not make sense, but for me, I always make a DVD of everything, because that is the closest thing to true archival digital media that we have. It could last 100+ years. For VHS, you can easily get 1.5 hours on a single layer DVD, and with low-motion content can stretch that to two hours, if you have a good multi-pass MPEG-2 encoder. -
I used DVD and DVD DL, then BD and DL BDs discs for quite a while for backups. But then I had a lot of discs to store.
Lately, I've been backing up to external USB 3 drives, usually the WD 3.5" externals. Not the best for security,
but most of my backups are from my DVD/BD collections, so I could just convert them again.
For more important video/data, I still use the DL BDs as they are seem most effective per GB of storage, more so than DVDs, IMO. -
lingyi, that link doesn't work.
"BACKUP, BACKUP, BACKUP!" is obvious, but I'm not sure the best way to go about it. Making multiple copies of dozens of terabytes on a daily basis seems suboptimal, for example.
john, I have a too-large-to-count number of tapes (definitely well over 300), at between 1.5 and 3 hours. Capturing Lagarith, and also doing restoration work. Some tapes require multiple captures. I intend to retain the original capture in Lagarith for flexibility and the uncompressed result of my AviSynth and VDub processes for cross-platform compatibility of the Premiere files that will create the final result.
They're all basketball games, so no low-motion content here. -
Capture and short term storage: a lossless format like UTVideo
~4 Bits/(Pixel*Frame)
Long term storage: a "near-lossless" format like H.264 CRF 14, preset 'medium'
~0.3 Bits/(Pixel*Frame) (less than 1/10 the size of lossless)
If you need to go back and edit the footage, re-convert to lossless. -
[QUOTE=lingyi;2516962As discussed in numerous threads, at the risk of voiding the warranty, it's highly recommended to remove the drives from the case due to issues with overheating and USB interface failure.*[/QUOTE]
Here's a timely note from the Federal Trade Commission. Opening cases, replacing parts, breaking "Warranty void if broken" seals, etc. do not void warranties.
FTC Staff Warns Companies that it is Illegal to Condition Warranty Coverage on the Use of Specified Parts or Services
In fact, those warranty void stickers may actually be against the law:
FTC Warns Manufacturers That 'Warranty Void If Removed' Stickers Break the LawThey that give up essential liberty to obtain a little temporary safety deserve neither liberty or safety.
--Benjamin Franklin -
What would you do with it after removing it from there anyway? Use it as an internal? I don't have that many bays and portability is important.
-
Get a drive dock, something like this https://www.newegg.com/Product/Product.aspx?Item=9SIA1DS28Y2007 and use regular internal drives. If it is only for storage and not while editing then you can user slower and cheaper drives.
-
Here's a corrected link to the Probox: https://www.amazon.com/Mediasonic-ProBox-HF2-SU3S2-SATA-Enclosure/dp/B003X26VV4/ref=sr...rds=mediasonic One of the nice things about his line is that they can be used as USB 3.0 or eSata.
For portability, there are external cases with fans for better cooling or SATA to USB adapters and HDD carry cases (individual and multiple). -
Hi,
the most convient price/performance solution is to buy external hard disc (2.5 inches) with 4 TB disc to be connected by usb. Keep them twice: every file should be save twice on different device.
Usb is quite universal and I suppose could be last a couple of decades. So I think you'llread them within 20 years. -
I've got through about 5% of the project on this 8TB drive, I doubt a 4TB drive is the solution.
-
Well at that rate you can either get a total of 160TB of HDD space, or you can go for LTO-6 tape drives. LTO-6 is usually around half the price of HDDs but the investment in the tape drive is the most expensive part usually, I've seen deals on ebay for LTO6 and LTO7 drives for below $1000. You would also need a proper SAS card connected to the motherboard via PCI-E, and probably 3 HDDs in RAID0 to be able to feed the tape drives a constant ~300MB per second.
Personally I don't have this level of data needs and so just use single layer BD-R for cold storage, with added par2 data. -
It takes ~14-18 hours to backup 6TB of data from from drive to another, ~100-130MB/s in my USB 3.0 enclosure, slightly quicker with both drives on internal SATA. This is in line with others report online. The bottleneck being the mechanical nature of regular HDDs. Seagate Archive drives take about 50% longer because of their slow write speeds.
Assuming you're generating several terabytes of data a day (given capture and editing time), you could set backup the drive overnight or upon completion of a project, set up that drive for backup while using a third drive for the next project. Rinse and repeat.
The only alternative I can think of is RAID (1 or 10) which will give you an "one the fly" copy, but is no better than a single drive if data is corrupted. You'll still need a backup solution (ideally at least two, one local and one offsite), so this is a costlier solution than the one above.
Unless something drastically changes in the very near future, your choices are HDDs for capture and HDDs, Blu-Ray or LTO for backup. I put cloud storage as secondary backup because of upload transfer speed. -
-
It's not about being a virgin again, it's about having "good enough" quality for long term storage -- this is VHS footage we're talking about, and H.264/CRF 14 is plenty good enough.
Converting back to lossless is done simply because intra-only lossless formats work a lot better in an NLE -- they seek better and decode faster than long-GOP video does. (to state what should be very very obvious) -
I still think, with this much material, and with the original quality being VHS, that the OP really ought to re-think using lossless (Lagarith) compression and, if I read his posts correctly, also rethink storing uncompressed. I would use neither of these things. If I wanted to store something that is editable, and can be edited several generations, then Cineform is clearly the best option. I say "clearly," because this format was developed specifically for having great editing speed on an NLE timeline; for have extremely small initial degradation; and for having extremely low generational loss when saved (re-compressed) multiple times.
What's more, it used to cost several thousand dollars, but is now free, via the GoPro site (they purchased the Cineform company).
I would strongly recommend taking 1-2 minutes of sample footage and then save it using uncompressed, Lagarith, Cineform, and h.264. Make sure to save using the same color space so there is no detectable color shift. Then, find at least three willing souls to look at thirty seconds from each saved video, preferably on a big screen, and have them vote which is the best. I expect that no one will be able to tell the difference between any of them even, possibly, the very lossy h.264.
The amount of storage you are talking about is absolutely massive and, having archived a LOT of sports footage (1,000+ hours), I can tell you that NO ONE will ever watch it all, so it is crazy to spend this much money, and take up considerable storage space, for something where only small bits will ever be looked at closely.
I am not arguing to not do the project, but instead try to come up with a solution that is appropriate for what you are doing. VHS just doesn't call for this workflow. -
I don't think I could find three willing souls who can tell the difference between SD and HD, comparing lossless and near-lossless video would be a waste of time.
Here's my workflow:
Capture to Lagarith
Restore using AviSynth and VDub, with Lagarith intermediate files where necessary
Export to uncompressed to bring into Premiere (needs to be compatible with both Mac and Windows, and properly treated as YUV by Premiere CS6)
Color-correction in Premiere, along with overlaying graphics, editing out commercials, etc
Export to Lagarith AVI, 576p50 (QTGMC used to double-rate deinterlace during restoration step)
Use MeGUI to generate x264 files in 576p25 (square-pixel), 576i25 (anamorphic), 720p50 (square-pixel)
Can also go to DVD or Blu-ray via the latter two AVS files in either MeGUI or HCEnc
Delete any intermediate Lagarith files -
Well if you are keeping the x264 versions as your final masters then there really is nothing more you can do besides use a higher CRF (lower bitrate) to save space. Not sure if you consider your final Lagarith copy to be an intermediate and if you are keeping those too, if so then you might want to delete those if you are creating duplicate lossy x264 copies. Otherwise you simply need more storage, whether it be HDD/LTO/BD-R.
-
-
I feel like half the time I've spent watching TV with others has been spent trying to steal the remote and change to the HD channel.
Yes, the Lagarith out of Premiere is deleted.
I've also received the suggestion of MagicYUV for VDub-to-Premiere for multi-OS compatibility.
But yes, I'm not here to find out how to fit this whole thing into 8TB. I'm asking what the storage options are well above that, and backups as daily backups would mean I spend more time backing up than working. Creating a new backup each time is incredibly space-expensive, but overwriting the one backup repeatedly doesn't seem ideal either as you're basically deleting your backup every day and if a file gets corrupted without you noticing or something... -
>I'm asking what the storage options are well above that,
Which people have been giving you
>and backups as daily backups would mean I spend more time backing up than working.
Overnight backups are a thing.
overwriting the one backup repeatedly doesn't seem ideal either
>as you're basically deleting your backup every day and if a file gets
>corrupted without you noticing or something...
That's why you need at least 2 backups in rotation (plus the original data)
There is a saying:
If you can't find the time to do it right, when will you find the time to do it over?
(because you will lose at least part of your data at some point in this project) -
You can't find a lossless codec that Premiere on both Windows and Mac can use properly? How about editing out the commercials in the second stage of your workflow in Vdub? (Could you not do the colour correction at the same time?) Seems like an awful lot of work for basketball games on VHS, but each to their own.
As to your assertion that you don't think you could find three willing souls who can tell the difference between SD and HD, well, either stop tormenting your blind friends with stupid questions, or get a decent screen! Seriously, I'd be interested in seeing a small sample of your finished work. -
Someone mentioned 14-18 hours per backup earlier.
overwriting the one backup repeatedly doesn't seem ideal either
>as you're basically deleting your backup every day and if a file gets
>corrupted without you noticing or something...
That's why you need at least 2 backups in rotation (plus the original data)
You can't find a lossless codec that Premiere on both Windows and Mac can use properly?
How about editing out the commercials in the second stage of your workflow in Vdub? (Could you not do the colour correction at the same time?) -
How were you cleaning up the VHS drop outs/errors in the 1982_GEE file?
-
Similar Threads
-
The Complete Guide To Making Blu-Ray Backups (1:1 Backups + BD5 Backups)
By milOtis in forum User guidesReplies: 222Last Post: 21st Aug 2021, 12:09 -
What format for digital storage?
By skwnmkr in forum Video ConversionReplies: 2Last Post: 19th Feb 2018, 13:21 -
My opinion of free best cloud storage and large file transfer
By jimdagys in forum ComputerReplies: 1Last Post: 25th May 2017, 13:19 -
Video Converting and Digital Storage
By FastMotion in forum Newbie / General discussionsReplies: 13Last Post: 2nd Nov 2015, 11:34 -
VHS to DVD project. Looking for advice on conversion workflow and storage
By zinkatres in forum Video ConversionReplies: 14Last Post: 9th Jan 2015, 16:17