Split 13GB DV AVIs to archive on 4.7GBDVDs losslessly?

[Razz]

Is it possible to LOSSLESSLY split MiniDV-based Type-1 AVI files (into smaller AVI files) with NO conversion of the underlying audio and video? Or do AVI editing tools have to decode and then re-encode the underlying audio and/or video? (I don't care about the wrappers being different or perhaps the start & end frames being slightly different.)

If you want details on why I'm splitting the AVI files and why I'm suspicious of the edited results from NVE3, WMM2, and VirtualDub1.6.4, please feel free to read on.

I transferred my 1-hour miniDV tapes to my PC with Windows Movie Maker 2, resulting in roughly 13GB files. Now I want to losslessly split and archive each of my 13GB AVI source files to sets of four 4.7GB DVD-Rs. I want the resulting (sub-4GB) AVI files to be exactly the same quality as the original 13GB AVI file. That is, I don't want them to put them through ANY conversion. I just want 4 smaller files that I can archive and use in the future.

I tried splitting the 13GB source file with Nerovision Express 3, but I couldn't figure out any way to keep Nero from grinding through a long transcode (not even with the "Enable Smart Encoding" box checked).

I tried splitting a 13GB AVI file Windows Movie Maker 2. But the total size of the resulting 4 smaller files was slightly smaller than the original 13GB source file. That scared me, as I expected the total to be slightly larger because there would be four headers instead of one.

I also tried splitting the same 13GB AVI with VirtualDub (using "Direct Stream Copy" audio & video settings). The total size of the resulting 4 AVI files was, slightly larger than the original source file (as I expected). But the size increase was more than I expected, which makes me slightly suspicious that some re-encoding was done. Maybe not. I'd just like to know for sure.

Here's why I want to archive the data to DVDs... I'll be making DVD-videos from my 13GB AVI files over the coming months. But in the meantime, I would like to archive the raw AVI video onto DVD-Rs:
1) For off site safekeeping (in case my house burns down).
2) So that I can free up some hard disc space, yet preserve my ability to easily reuse the AVI files at some point several years from now, to create HD-DVDs (or some other future media format).

I don't want to archive the 13GB AVI files back to tape because it takes a lot of time, and I'm not sure that I'll own a MiniDV camcorder (or that mine will still work) several years from now. (I recently had this problem with my 8mm tapes, and had to buy a new camcorder simply to view and archive my old tapes.)

I don't want to split the file using something like Nero BackItUp because several years from now, when I try to retrieve the archived files, I'll probably have a new PC that doesn't have (or can't run) Nero BackItUp.

If there is no way to losslessly split the AVI files, I'll just transfer them from my camcorder to my PC again in smaller pieces.

Thanks in advance!

[BJ_M]

virtualdubmod will split the files without re-compression ..

do it all the time ...

[gadgetguy]

Virtualdub (or any of it's versions) Set both video and audio to Direct Stream Copy

I believe the difference in the filesizes comes from having to create new headers for each section. There should be no loss of quality though.

[ZippyP.]

virtualdubmod will split the files without re-compression ..

Set the audio and video to "direct stream copy", set your in and out points, save segmented avi.

[BJ_M]

virtualdubmod will split the files without re-compression ..

Set the audio and video to "direct stream copy", set your in and out points, save segmented avi.

you dont have to set in / out points in V-dubmod, you can just set size of segment you want and it will split them for you and name them ..

why i suggested the v-dubmod version ...

[gadgetguy]

I thought VDubmod didn't accept Type 1 DV

[BJ_M]

no problems -

install mainconcept or panasonic DV codec ... but it will not recompress in stream mode

[BJ_M]

ive done this on many projects , a couple were for the discovery channel in fact, so one of the producers could edit on a laptop using FCP and the disks to make a EDL


dvd data disks were d-beta captured as DV as a proxy format

[Razz]

Thanks everyone!

In VirtualDub1.6.4, I used "Direct Stream Copy" settings and tried saving the clips with "Save as AVI...", "Save Segmented AVI", and "Save Old Format AVI". Anybody know what the difference is between the three save modes?

[Wile_E]

Well Nero BackItup is my choice. I've tried VirtualDubMod stream copy, but after rejoining the files, the bytes are different. File usually has some bytes missing. Not sure why.

[BJ_M]

You can also use winrar to do this and set it to no compression ...


anyway -- never a problem with virtualdub, if you have bytes missing you are doing something wrong or not reading the data correctly .. 1000s of people use it for this and ive have not seen any reported problems..

[Razz]

Thanks. I' prefer not to use a backup program like Nero BackItUp or WinRAR because I'm not sure that I'll be able to run those programs on my computer, say, 5 years from now when I might want to access the archived video to make an HD-DVD or something. By that time, I may not own that same backup program, or I may own a 64-bit computer which potentially isn't compatible with older programs. So I'd just as soon break up the 13GB AVI into smaller AVIs, which I'm sure I'll be able to access a decade from now.

[BJ_M]

i am using a 64bit computer and ther eis also a 64bit version of winrar ...

but you have a good point

[Razz]

A BIG THANKS to all of you! I'm rockin' through the project.

[Wile_E]

Well I tried again splitting files with VirtualDubMod. I used Direct Stream copy to split. Then I rejoined it. There is about 1MB missing after rejoining the file. This happens only on the first rejoin. Multiple splitting/rejoining doesn't lose any more data.

I loaded the files in Gspot. As you can see, Gspot says the bitrate has changed. Bitrate was reduced from 28868k to 28775k. This is odd, since I did a Direct Stream copy(no re-compression). What is going on? Does Virtualdub change the DV Type? Gspot doesn't show DV Type I or II.

[BJ_M]

might have just stripped off some metadata strings

[Wile_E]

I suppose the OP has a good suggestion. Nero BackItUp is probably a bad choice for back up. I may not have a copy of it, 10 years from now. So I will take everyone's advice and use Virtualdub to split files. Easy enough.

How do I find the exact file size to split to, for DVD-R? I want to use the full disc capacity. Virtualdubmod only supports 2048MB segments?

[gadgetguy]

I wouldn't use 'Save to Segment' option. I would set my start/stop points and use 'Save to AVI' in batch mode.

[BJ_M]

I wouldn't use 'Save to Segment' option. I would set my start/stop points and use 'Save to AVI' in batch mode.

why not ? It is DV so which frame it splits on makes no diff...

[BJ_M]

I suppose the OP has a good suggestion. Nero BackItUp is probably a bad choice for back up. I may not have a copy of it, 10 years from now. So I will take everyone's advice and use Virtualdub to split files. Easy enough.

How do I find the exact file size to split to, for DVD-R? I want to use the full disc capacity. Virtualdubmod only supports 2048MB segments?

it will support any size up to 2048 , set it to split 1.44666GB chunks and you will find 3 of those fits perfect ..

[gadgetguy]

why not ?

...because as stated earlier, the largest segment allowed is 2048MB.

Edit: Just saw your other post. That would work too.

[Razz]

My 1-hour MiniDV family video tapes resulted in files of about 13.2GB - 13.4GB. I realize that I could have split my 13GB AVIs into THREE even pieces using VirtualDub and archived them onto 3 DVD+Rs, which hold about 4.5GB of data. (13.4GB/3 = 4.46GB which just barely fits onto a DVD-R.) HOWEVER, I was uneasy about splitting my AVI's randomly in mid-scene. So I split each of my 13GB AVIs into FOUR smaller AVIs AT SCENE CHANGES using VirtualDub.

I realize this was probably overly cautious because I'll PROBABLY be able to seamlessly splice AVIs back together in the future. But then again, maybe I won't be able to flawlessly splice them back together. Or maybe a decade from now when I go to access my archived videos to make an HD-DVDs (or some other future format) I'll have been using some new format camcorder and editing tools for so long that I won't remember how to flawlessly splice AVI's back together. Or maybe video editing software a decade from now will only pay a token effort to splicing AVI files together or something. I don't know. But DVD+Rs are only about 20 cents in bulk on sale. So for a neglible cost and a bit more time, I eliminated that risk. A decade from now, I don't want to have worry about having to seamlessly splice together my wedding vows in the middle of "I do" or something.

For me, basically each set of scenes had to be 20 minutes or less in order to fit onto a DVD-R. So for example, depending on the scene breaks in my videos, I might have broke them up into 4 files like:
4 consecutive scenes for a total of 15 minutes (3.4GB) +
2 consecutive scenes for a total of 20 minutes (4.4GB) +
3 consecutive scenes for a total of 15 minutes (3.4GB) +
2 consecutive scenes for a total of 10 minutes (2.2GB) =
Total = 60 minutes (13.4GB).

I realize that I could have saved a few DVD-Rs by putting, say, the last 10 minutes of one tape on the same DVD with the first 10 minutes of the next tape. But since I've already got my tape cases labeled with descriptions, dates, and time-stamps for each scene, I decided that it was easier to just split each tape into 4 DVDs and simply write a label on the DVD that associates it to tape case. (e.g. "2001 Ski Video Part 1", "...Part 2", "...Part 3", & "...Part 4".)

I realize that I'm probably just being over cautious. That's just me. I would totally understand anyone making the decision to just go ahead and split mid-scene. And in one case (where I had an Easter Egg Hunt scene that was about 25 minutes) I HAD to split the scene. But I tried to do it at a break in the action where it wouldn't matter if I introduced a glitch.

[vhelp]

Instead of all this clobering of missing pieces or strange sizes,
why not then, just use the DV software you used to retreave in the
first place, to only record in the amount of SIZE or MINUTES you
want to fit onto a dvd disk ??

As far as scene change goes. I dont' think you need to worry about
that when it comes to DV, because it is interlace, every frame.
You might be mixing MPEG; GOP 's; Scene-Change; and confusing it with
DV. Don't worry about this, as it does not participate in any way
with DV.

As far as your odd numbers are being revealed screwy, is probably
on account of the type of AVI format being used. And, during the
re-creation of the DV AVI (if using vdub) will use a different
AVI format, and this is what is causing your odd filesize reporting
in GSpot. I know that vdub uses a different AVI format than my DV
capture app uses. I too fumbled with this one, till I realized what
was going on. Not everyone stick to the script. (have you looked
at the AVI reference lately ? There is lots of room for variation,
but the job still gets done properly. Only a few odd bytes gets
tossed or added) For example, you have AVI 1.0x and OpenDML formats. Both
do the same things, but in different ways. Although the filesize
might change (slightly) the frames will all still be there, and in
there rightful place

-vhelp 3433

[Razz]

The DV "capture" software that I used (WMM2) automatically rewinds the tape, transfers the data, and stops at the end. So it takes about 30 seconds of my time to capture a 1-hour tape-- and I can go get other stuff done during the 1-hour "capture" process. Then it only takes a few minutes in VirtualDub to split the AVI file. If I were to transfer the data to my PC in smaller segments, I'd have to pretty much hang out by the PC so that I could start and stop the transfer process every 15 minutes or so.

I'd do that if that's what it took to "capture" and archive LOSSLESSLY. That's the whole reason I started this thread. I wanted to know if I had to spend many hours doing that or if I could just use VirtualDub to split losslessly.

I know that I can seamlessly splice mid-scene AVIs back together NOW, and that I SHOULD be able to do so a decade from now. I just don't want to take the risk that the next-generation-video-format editing software that I own a decade from now may not be able to splice (then ancient) AVIs or that it will be time consuming to figure out (or remember) how to seamlessly splice AVIs. It's a small risk. But it's not worth it to me.

[rkr1958]

no problems -

install mainconcept or panasonic DV codec ... but it will not recompress in stream mode

BJ_M, thanks for this tip. Worked like a charmed. I've been doing several VHS tapes (mainly Disney Movies) to DVD conversions. The program I used to capture the tapes produced Type 1 DV AVI files, which resulted in no sound in VirtualDub. After installing the panasonic DV codec I kow get sound and can extract the audio using VirtualDub.

[vhelp]

Well, at the present time and given your setup, I would just do
it the way it was suggested way in the beginning. Use vdub, and
dont' worry about a couple of odd bytes. Remeber what I noted
about this, in my previous post. So, stick with vdub, and just
make sure your setting is correct..

If capture DV AVI file is of Type 2:

* Video\Direct Stream Copy..
* File\Save segmented AVI...
* Filesegment size limit in MB (50-2048) [ 2048 ]

And, save your segmented AVI files in this way, and copy two
2048g size files to the dvd-r/rw disk.

Or, (untested math->dvddisksize here) use this routing..

If capture DV AVI file is of Type 2:

* Video\Direct Stream Copy..
* File\Save segmented AVI...
* [x] Limit number of video frames per segment: [ 32367.6 ]
* File segment size limit in MB (50-2048) [ 2048 ]


1sec = 29.970F * (60sec = 1798.2F) * (18min = 32,367.6F)

Where F=Frames

From a decimal perspective, because every 1 minute equals 1798.2 frames,
when we multiply that by how many minutes will actually (approx) fit
on a dvd disk, we come up with 32367.6 frames. This is the number you
record in the above box.

When you begin saving the segmented AVI, it will save in chunks of AVI's.
Meaning, chunks of 32367.6 frames per AVI file, every single file.. with
the exception of the last file, because they usually round up less.

When you open each file, the file will have exactly 32367.6 frames, minus
1, because vdub starts a frame at zero.. 0; 1; 2; 3; 4 ...

To bring in all these files back into one complete (and massive) AVI
file, you simply open the first AVI file starting with filename.00
using the following:

* Video\Append AVI segment...
* [x] Autodetect additional segments by filename.

If you want, I'll chaulk up a tiny calculator tool to do all this
behind the seems, and for different media sizes, based on frames

Using the above steps, I did a quick test run with a DV file with
the following specs:

Filesize: 609MB
Frames: 5,183

* [x] Limit number of video frames per segment: [ 128 ]
* File segment size limit in MB (50-2048) [ 2048 ]

..and the above step saved (41) AVI files, with a size of 15,130KB.

Note 1:
(2) files were slightly less, 1st file (15,012KB) and last file (2,822KB) )

Note 2:
each file segment had 128 frames, w/ exception of last file (was smaller)

You could look at this as a "frame-accurate" method

-vhelp 3434

[Westredd]

Due to my experince with archiving to DVD. I think you would be better keeping the miniDV tape as your archived copy. If you transfer an AVI file over 3x 4.7 GB DVDs and you pick up a tiny scratch on the surface of any of the dvds, then the whole 4.7 gb AVI will be ruined due to the nature of AVI files and how sensitive they are to corruption. It so easy to pick up glitches on a DVD-R surfacae, even if you never use it. Its happened to me! What I don't like about DVDs is the entire media surface is exposed to damage at once. whereas tapes are wound , any damage to a tape would only create a problem on the footage where the tape is damaged.

Just thought it was a point worh bringing up.

ta

West

[BJ_M]

but the tapes dont last that long -- sometimes only 5 - 7 years just sitting there as they pick up humidity

longer if properly stored ..

[Westredd]

5-7 years is pretty good!

at least you can give tape a time period, DVDs can pick up problems after a few months, and because they will have huge AVI files , the whole AVI on that DVD will be history because you wont be able to copy it off. DVD-R media to jsut far too sentive, and that combined with the sensitvity of big AVIs is a really bad combination.

[MpegEncoder]

It's easy to split ANY file. Just use the split utility.

In MS Windows, you can install it as part of the cygwin utilities (www.cygwin.com).

Usage: split [OPTION] [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is `x'. With no INPUT, or when INPUT
is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
-a, --suffix-length=N use suffixes of length N (default 2)
-b, --bytes=SIZE put SIZE bytes per output file
-C, --line-bytes=SIZE put at most SIZE bytes of lines per output file
-d, --numeric-suffixes use numeric suffixes instead of alphabetic
-l, --lines=NUMBER put NUMBER lines per output file
--verbose print a diagnostic to standard error just
before each output file is opened
--help display this help and exit
--version output version information and exit

SIZE may have a multiplier suffix: b for 512, k for 1K, m for 1 Meg.

Report bugs to <bug-coreutils@gnu.org>.