Storage/Backups for Large VHS-to-Digital Project

[poisondeathray]

Was MPCHC x64 or x86 ? Try the other

Are you sure the GoPro install was x64 ? IIRC if there were 2 versions for the old installer, one for x64, one for x86. (PP CS6 was x64 only)

[koberulz]

Get a Windows Tbolt or esata card and an enclosure that allows for USB2 as fallback + USB3/USB-C/eSata/Tbolt (or whatever combination you expect to need).

This is about where you lost me.

Being able to easily backup and swap out drives is a huge appeal of the hot-swap tray. That'd be less of an issue at the Mac end. And is it worth sending enclosures back and forth, rather than each end having one and just swapping drives in and out as required?

Are you sure the GoPro install was x64 ?

It seems it was not. MPCHC x86 opens the file. It says it is though.

[koberulz]

The GoPro software has folders in both Program Files and Program Files (x86).

[Cornucopia]

How could I lose you?

You have a windows desktop or laptop. It has either a stock USB3 port, or you've added a Thunderbolt port (via card, etc) or USB-C (similar capability), or eSata. One or more of those 3. Of course, you already have USB2 as a fallback (but it will be SLOOOWWW).
You have a Mac. It likely has besides USB2, a Thunderbolt, or if very new, USB-C. USB3 is a little rarer.

Buy a raw internal drive (e.g. Seagate 2TB 7200rpm, etc).

Buy a sata drive external enclosure. Many have USB2 + one of those other faster options. Choose your favorite.

Put the drive in the enclosure.

Connect the drive to the computer that it will reside the most in and format it for the native filesystem (NTFS for Win, HFS+ or AFS for Mac).
On the opposite platform, install a filesystem compatiblity driver/app (as mentioned).

Move back and forth between platforms. All those port types are hot-swappable (though you should ALWAYS do a proper dismount+eject). This isn't any more difficult than a swappable raw drive (in caddy), except you have a few extra wires.

What I was saying was IT IS AN ISSUE AT THE MAC END. Which Mac do you have? Unless you have a MacPro tower desktop of 2012 era, there are NO Mac machines with user-configurable 3.5" bay slots.

If you were thinking this was a pair of attachable dock enclosures (one permanently attached to the PC and one permanently attached to the Mac) that ran their own embedded OS and had their own bays, that is not how it works. The one listed earlier is just a plain 3.5 shuttle & caddy, which requires a 3.5" open bay in the host machine.

Now, if you want, there are some "drive duplicator" external enclosures that really aren't enclosures but drive docking stations (yet are still often called enclosures): https://www.bhphotovideo.com/c/product/1071777-REG/startech_satdock2reu3_hdd_duplicato..._1_eraser.html is an example of a modestly priced one, though you get get upwards of $250-350USD for a Drobo or OYEN(?) that supports multiple pathways & RAID setups. You can find even cheaper options than the startech, but I probably wouldn't recommend them.
Note: in this setup, you would either need to move the enclosure/dock back and forth, or get 2 enclosures/docks and move the raw drives.

Plus, you will still need to get the opposite platform's filesystem driver app*. You will also need to make sure both have the appropriate fast transfer port option compatible with that platform.

Scott

*what I mean by this is that you say work mainly in Win, so you format for NTFS. Then get Tuxera for Mac so it can also read/write NTFS.
Or you work mainly in Mac, so you format for HFS+. Then get MacDrive for win so it can also read/write HFS+.

[koberulz]

What I was thinking was getting the three-drive 5.25" thing, which allows easy swapping out of drives, attaching multiple drives, etc, then making a copy of each drive as I finish with it. Pack one copy away at this end and send the other copy to the Mac end. They can get their own enclosure, and then it's just down to whether the drive formats are compatible (they have previously given me drives I've not been able to use, although we worked around that with exFAT, but I'm not sure if that's a thing that would still work here).

You lost me because I don't know what USB-C or Tbolt are, I'm only vaguely familiar with SATA, and I don't know what the differences and such are between them.

[Cornucopia]

If you're working in this field, you should learn what those are. And what a "thing" is.

exFAT is not a good/optimal choice for either system.

Scott

[lingyi]

@Cornucopia

According to koberulz, he has an Z170A GAMING M3 motherboard and it has USB 3.1 (Gen 1 and Gen 2) and USB 2.0 ports only.

Some drive cages are trayless and the HDD just pops in and out. On my old PC, I installed one upside down and the drives would drop in my hand when the door was opened.

AFAIK, most multi-drive enclosures are trayless. My personal favorite is the Mediasonic Probox I linked to earlier. It's trayless and has both USB 3.0 and eSATA connections (both connections can be setup and switched from the front). I also have a 4 bay dock from the same company that has the same capabilities, but mainly use the full enclosures because they have an internal fan. The only quirk on the full enclosure is that if you don't use the included handle, the drives are a bit difficult to pull out because they're about 1/4" apart. On the one I use for quick drive swaps, I don't put a drive on the top bay leaving me enough room to pull the drives from top to bottom.

I agree that two enclosures or docks (one for each PC) is the way to go. I currently have 7 multi-bay enclosures and turn them on only when I need to access the drives in them. This reduces the heat and noise lordsmurf mentioned at digitalFAQ.com (koberulz has a duplicate thread on this topic there). Also they serve as HDD storage when not in use.

[TimA-C]

I don't think this guy even has a Mac.

Do you really need to be able to work on these files in Premiere on both Mac & Windows?

Me, personally? No, I'm quite happy working in Windows. I'm not everyone though.

Only you can decide how valuable these recordings are to you

I'm still not everyone though.

I think he just wants to make sure that his efforts are fully available for anyone down the road to work on them from whatever system. If that's the case then I would suggest that he sticks to current video formats and codecs (preferably those still being developed) and let whoever, down the road, sort out getting the videos into their system. Get an external USB3/E-sata bay - maybe one with two slots and the option to clone from slot 'a' to slot 'b' without needing a PC - work on a couple of tapes at a time, back them up to the external drive. When drive is full, make clone, put drives back in their non-conductive plastic bags and store in a watertight container somewhere with fairly even temperature and not too much humidity. Keep one of the clones at a different location in case of fire or theft etc.

If I'm right, then I applaud his altruistic ambitions but I think he's making his life unnecessarily difficult (and costly) without actually benefiting any future users all that much.

[poisondeathray]

Are you sure the GoPro install was x64 ?

It seems it was not. MPCHC x86 opens the file. It says it is though.

Check in the Windows/System32 directory if CFHD.dll exists - this is the x64 version

The x86 version of CFHD.dll should be in the Windows/SysWOW64 directory ; but if MPCHC x86 plays it, then we already know you have this one

[koberulz]

CFHD.dll is in both locations.

[lingyi]

If a dock with disk copy ability is used (I have one and never used the copy function), I highly recommend (would say it's actually required) using a file compare program to ensure all the files are identical. I've been using ViceVersa for years and highly recommend it. Here's a link to the free version: http://www.tgrmn.com/free/

[poisondeathray]

CFHD.dll is in both locations.

Not sure what is going on, maybe try downloading from somewhere else

Also check C:\Program Files (x86)\Common Files\CineForm directory

CFDecode64.ax is the directshow x64 decoder component . I think this is the one MPCHC x64 should be using

Vdub x86 and x64 will be using the appropriate CFHD.dll versions

But I think older PP versions actually use the VFW component CFHD.dll , not the directshow version

[Cornucopia]

If a dock with disk copy ability is used (I have one and never used the copy function), I highly recommend (would say it's actually required) using a file compare program to ensure all the files are identical. I've been using ViceVersa for years and highly recommend it. Here's a link to the free version: http://www.tgrmn.com/free/

Another recommend: Teracopy. Have been using it with satisfying results for the last 4 years.

Scott

[lingyi]

If a dock with disk copy ability is used (I have one and never used the copy function), I highly recommend (would say it's actually required) using a file compare program to ensure all the files are identical. I've been using ViceVersa for years and highly recommend it. Here's a link to the free version: http://www.tgrmn.com/free/

Another recommend: Teracopy. Have been using it with satisfying results for the last 4 years.

Scott

+1 one on Teracopy, I've been using it for years also. Much faster and reliable than the regular Windows (especially Win10) copy or move.

[koberulz]

@pdr: Neither Program Files nor Program Files (x86) have a CineForm folder in Common Files.

[koberulz]

AFAIK, most multi-drive enclosures are trayless. My personal favorite is the Mediasonic Probox I linked to earlier.

Is there any reason to go for that over the internal 5.25" bay cage? The 5.25" bay one has the obvious advantages of no USB cable, no external power source, and no desk space.

[lingyi]

For me, the main reason is being able to to turn on the Probox only when I to access the drives. I have my PC on 24/7 and save on the electricity and wear on the drives. I currently have three 4 bay boxes connected to my PC and rarely have to access all the drives at once.

If you have a PC or Mac with an eSATA port, you could have both USB and eSATA connected (i.e. USB cable on the PC, eSATA on the other PC/Mac) and switch between the connections from front (best to turn off the box before switching).

[koberulz]

Browsing around I found Program Files/Adobe/Common/Plug-ins/CS6/MediaCore/CineForm, which contains CFHD_AVI_Importer.prm, CFH_File_Exporter.prm, CFHD_MOV_Importer.prm and CFRenderProc.exe. Tried copying them to Premiere's plugins folder, but it still won't import the CineForm file.

The two export options are available, though.

EDIT: I tried exporting as CineForm from Premiere, and then importing the result. Same error.

[koberulz]

Working on an unrelated DVD, and when I hit 'queue' on my M2V/AC3 exports from Premiere, Media Encoder spat out this error:

---------------------------
Monitor: PProHeadless.exe - System Error
---------------------------
The program can't start because CFHDDecoder64.dll is missing from your computer. Try reinstalling the program to fix this problem. 
---------------------------
OK   
---------------------------

[koberulz]

The AME has now disappeared. Not sure if it was rebooting or deleting the extra copies of the plugins, or just random chance.

I get this when I open AE though:

---------------------------
After Effects
---------------------------
After Effects warning: The following plugins have failed to load.  Please reinstall these plugins:
C:\Program Files\Adobe\Common\Plug-ins\CS6\MediaCore\CineForm\CFHD_AVI_Importer.prm
C:\Program Files\Adobe\Common\Plug-ins\CS6\MediaCore\CineForm\CFHD_MOV_Importer.prm



---------------------------
OK   
---------------------------

Interestingly only the importers fail to load. Which lines up with being unable to import the CineForm file created in VDub, but being able to export one just fine.

[koberulz]

Just browsing around C:/, and stumbled across CFHDDecoder.dll and CFHDDecoder64.dll in C:/Program Files (x86)/CineForm/Tools. Copied them to C:/Program Files/Adobe/Adobe Premiere Pro CS6, and the file now imports successfully.Just browsing around C:/, and stumbled across CFHDDecoder.dll and CFHDDecoder64.dll in C:/Program Files (x86)/CineForm/Tools. Copied them to C:/Program Files/Adobe/Adobe Premiere Pro CS6, and the file now imports successfully.

[koberulz]

Giving Cineform a whirl for my VDub-to-Premiere step. What's the best quality to go with?

"filmscan2" for the regular version

It's actually a native format in newer PP versions - they implemented the gopro sdk

You sure? It won't import at the Mac end with CC 2018, gives an error about the codec being missing or unavailable.

[pandy]

The OP is struggling with the concept and cost of of 1-3 bay drive cages and 4-5 bay external enclosures and never mentioned needing a NAS, especially large 16-24 bay enclosures you recommended.

Well... it was not clear how commercial is work is and how OP is eager to make some DIY approach (i.e. time vs money vs own work) - IMHO you can buy surplus older NAS for around 100 - 200$, you can build own NAS and IMHO for large projects where backup is more than desired NAS provide healthy separation of storage space from workstation.
Once again i would try to not use workstation to work store and backup... that's all. You can buy used motherboard with CPU for 30$, some used enclosure for perhaps 15$ side to this some PSU and HDD's and build own NAS - my impression after OP description was that he need hundreds of TiB storage i.e. more than regular mobo has SATA ports - to avoid problems discussed for half of topic (like external enclosure for HDD or PSU) it better to use dedicated solutions.

[poisondeathray]

Giving Cineform a whirl for my VDub-to-Premiere step. What's the best quality to go with?

"filmscan2" for the regular version

It's actually a native format in newer PP versions - they implemented the gopro sdk

You sure? It won't import at the Mac end with CC 2018, gives an error about the codec being missing or unavailable.

Yes I'm sure. The newer cineform is MOV wrapped now (kind of ironic since it began life as windows only AVI)

https://helpx.adobe.com/premiere-pro/using/gopro-cineform-codec.html

Adobe Premiere Pro CC, Adobe After Effects CC, and Adobe Media Encoder CC can natively decode and encode QuickTime files using the GoPro CineForm codec on both Mac OS X and Windows systems.

In fact, it smart renders on a Mac, but has limitations on a PC . (very ironic given it's windows roots)

[poisondeathray]

Another option might be prores . This is industry standard . It's much slower on a PC , but flies like you were cutting SD DV on a Mac. Even UHD is incredibly smooth

If you need vdub / avi / avisynth compatibility, use vdub2 (virtualdubfiltermod) . In case you missed it, I mentioned it already earlier - it imports / exports other formats too now . You can import/export, say MOV, or MP4 . Directly. No longer limited to AVI. Prores (ffmpeg variant) is included in vdub2

[koberulz]

I exported a CineForm AVI from VDub, so is that why it won't import at the other end?

EDIT: Yes, I remember VDub2 coming up but a lot came up and ended up forgetting about it before doing anything about it. I doubt ProRes is a good idea as 90% of this is PC work.

[poisondeathray]

I exported a CineForm AVI from VDub, so is that why it won't import at the other end?

EDIT: Yes, I remember VDub2 coming up but a lot came up and ended up forgetting about it before doing anything about it. I doubt ProRes is a good idea as 90% of this is PC work.

Cineform is MOV now. Export a Cineform MOV.

Mac's don't like "AVI" container and that has never changed, and never will change.

Prores runs on PC perfectly fine, it's just slower than on a Mac

x264/mp4 is another option, especially for lossless and compressed. But it is sloooooowwww compared to cineform decoding (even when using --tune fastdecode and Iframe) . It feels like molasses compared to cineform or prores on a mac. But it has many options, and quality is much higher at a given filesize, even in I-frame mode

[koberulz]

"Perfectly fine" and "slower" are somewhat contradictory.

Used VDub2 to create a CineForm MOV and sent it through. Probably won't hear back until Monday though.

[koberulz]

The CineForm MOV doesn't work at my end. Video is fine but there's no audio. The icon in Premiere indicates both video and audio but there's just silence, no waveform, etc.

Also I can't seem to add VDub2 to the right-click->"Open with..." menu. Windows just assumes I mean standard VDub.

[poisondeathray]

The CineForm MOV doesn't work at my end. Video is fine but there's no audio. The icon in Premiere indicates both video and audio but there's just silence, no waveform, etc.

Are you sure you exported audio ?

You might need to upgrade , or perhaps there is an issue with the MOV wrapper from vdub2 ?

They work when exported from Adobe (at least in CC , older versions might have problems)

Also I can't seem to add VDub2 to the right-click->"Open with..." menu. Windows just assumes I mean standard VDub.

It's been reported, on the to-do list
https://forum.doom9.org/showthread.php?p=1840682#post1840682



There are other near-lossless production options, such as avid dnxhd /dnxhr , grass valley hqx , but everything has pros/cons potential compatibility issues. Prores is the standard right now and probably going to give you the least headaches, especially if you need to satisfy your mac friends