Compressing raw AVI captures for long term archive

[jeby1980]

I have kept all of my raw lossless vhs capture avi's, around 2.0TB, who knows if I'll ever use them again, but in the future world of AI who knows.

7zip has a 98% compression ratio, which really isn't worth the time. I haven't seemed to find any other options but thought I'd ask as I havent found any recent threads on the subject.

[dellsam34]

Are they uncompressed AVI or losslessly compressed, Because lossless compression can save you up to 40%, albeit it's a tedious work. Though a 2TB is around $60, so I would just put them in a hard drive and forget about them.

[rgr]

I have kept all of my raw lossless vhs capture avi's, around 2.0TB, who knows if I'll ever use them again, but in the future world of AI who knows.

7zip has a 98% compression ratio, which really isn't worth the time. I haven't seemed to find any other options but thought I'd ask as I havent found any recent threads on the subject.

Lossless won't be smaller.
You can compress to H264/H265 with low CRF, but it will be (slightly, imperceptibly) lossy compression.

[pandy]

FFV1 should deliver higher lossless compression than 7z - it is tailored for video and 7z i general purpose.
I also agree that lossy (but imperceptible) compression with modern video codec can be acceptable (depends on your criteria).
Give a chance to FFV1 - if not try lossless H.264/265 but also test minimal lossy way (like qp=4).

[lollo]

As dellsam34 says, keep your raw lossless capture as they are. They are your masters for present and future enhacement, if any.

More advanced techniques can be used for extremely important material (error detection and correction, storage of 3x copies in 3 different places, etc) but that’s too much for you I suppose.

[jeby1980]

Thank you for the replies. They're the AVI's created during capture with lagarith. Yes, I just didn't know if there was anything that reduced them even further like zip or 7z. Just wanted to make sure that there wasn't something obvious before I spend the money on hard drives for only something that I "might" use in the future (HDD prices havent come down in price for quite some time now).

[jagabo]

Lagarith (and all the other lossless video codecs) use algorithms very similar to the archivers like 7zip, WinRar, etc. So those programs won't get you much more compression unless there's some other uncompressed data within the video file (uncompressed audio, subtitles, padding, etc.).

[nicholasserra]

Some discussion on using x264/x265 with crf 0 (lossless) for compressing these raw captures. Considering doing some testing on my own, as I have about 60TB of UTVideo and Huffyuv captures for long term archival.

Seems in testing this person was able to compress and uncompress and get an exact result back, while also cutting the size down drastically.

https://www.reddit.com/r/DataHoarder/comments/1g0g1pf/should_i_winrar_archive_raw_vhs_records_or/

[nicholasserra]

Did end up doing my own testing here finally. Trying to compress UTVideo AVIs down to another lossless codec to save some space. Results were not good.

x264 lossless only gave me a 10% reduction, a little more if I also encode the audio to flac. x265 file sizes remained the same.

x264:
ffmpeg -i UTTEST.avi -c:v libx264 -crf 0 -profile:v high444 -c:a copy x264out.mov

x265:
ffmpeg -i UTTEST.avi -c:v libx265 -pix_fmt yuv422p -x265-params lossless=1 -c:a copy x265out.mov

[pchan]

2TB needs about 40 bluray discs. Though its future is a bit uncertain but it's the most reliable long term storage. Verbatim still produces the burner and media.

[VWestlife]

M-DISC, available in capacities up to 100 GB, claimed to last 1000 years:
https://en.wikipedia.org/wiki/M-DISC

[dannyboy48888]

If they already lagarath you won't gain much if anything with another codec current gen. FFV1 is a fantastic codec, bit it fails miserably on animation. For that I use x264 qp 0. For fun I tried pumping yuv4mpeg multiple episodes into zpaq, borg, etc to exploit redudancy over long datasets and it was bigger or the same size but at a huge time cost. Keep your lagarath and pick ffv1 or x264 for future use.