Virtualdub crashing when outputting very large files

[nick1977]

Hi all

I've hit a weird problem in Virtualdub when dealing with very large files. The input files aren't particularly big - it's the process of outputting that seems to go wrong. I originally had this problem when trying to output a 1.5hr HD file using HuffyUV. It seems that somewhere between 300-400GB it can't handle any more, and crashes. I have included a screengrab of the image below.
I seemed to get success by using the frameserver and AVFS (as my eventual target was to convert to mp4 in handbrake) and that seemed to work for that file.

However my latest project is having the same crashing issue, even though I'm serving using AVFS. The virtual file size in the "Volumes" folder I around 742gb. Virtualdub crashes at around 47% conversion complete, which again is in the 300-400gb range.

Is there a known issue with virtualdub in these situations, or is it something with my set up?



Thanks

[hech54]

If your computer details are correct....I think that is your answer.

[manono]

Choose a lossless video codec (Lagarith, UT Video Codec, HuffYUV, etc.) and your files will be much smaller. You're saving as uncompressed (meaning, you didn't choose a codec), right?

[nick1977]

If your computer details are correct....I think that is your answer.

Oops lol
Yes that hadn't been updated in a long time!
I've updated it now to more accurately reflect my current setup

[nick1977]

Choose a lossless video codec (Lagarith, UT Video Codec, HuffYUV, etc.) and your files will be much smaller. You're saving as uncompressed (meaning, you didn't choose a codec), right?

It is set as huffyUV for output, but if I'm frameserving does that even make a difference?

[poisondeathray]

If you're using avfs, what are you using vdub for ?

what sort of operations are you doing in vdub ?

[nick1977]

If you're using avfs, what are you using vdub for ?

what sort of operations are you doing in vdub ?

Levels adjustments, color balance and HSV.

[poisondeathray]

If you're using avfs, what are you using vdub for ?

what sort of operations are you doing in vdub ?

Levels adjustments, color balance and HSV.

Those ones are unlikely to "cause" the crash

I've used larger files in vdub before - I don't know the reason "why" it's occurring for you

1) As temporary workaround you can split segments, say 200GB segments, then join them in an avs script, then use avfs

2) Or do the levels/color/hsv in avisynth

3) Do you need to use handbrake ? did you look at the vdub external encoder , or virtualdubfiltermod - you can output different types of formats directly from vdub (but this won't help you if it crashes from other reasons in vdub halfway)

[nick1977]

Thanks for the suggestions. Ideally I would like to "fix" what is going wrong, as I am happy with the workflow. However, I am happy to look at alternatives if necessary.

To deal with them in order:

1) I'm assuming this would involve actually saving the output file? One benefit from having the avfs server was to avoid taxing the HDD too much. I'd prefer to avoid creating temporary files 100's of GB in size. If I've understood you wrongly, and I don't have to actually create large temporary files please let me know

2) I would be happy to do this, but I'm a bit of a novice with avisynth. How easy is it to replicate the settings I have created in virtualdub? I have saved the file settings, but I guess there's no easy way to import from that?

3) I use handbrake because I have settled on processing settings I'm happy with. My OCD won't let me deal with something else doing the compression as I won't be confident it is giving me the same quality/size ratio. Sad I know, but that's just me!

[poisondeathray]

1) I'm assuming this would involve actually saving the output file? One benefit from having the avfs server was to avoid taxing the HDD too much. I'd prefer to avoid creating temporary files 100's of GB in size. If I've understood you wrongly, and I don't have to actually create large temporary files please let me know

Yes, physical file

2) I would be happy to do this, but I'm a bit of a novice with avisynth. How easy is it to replicate the settings I have created in virtualdub? I have saved the file settings, but I guess there's no easy way to import from that?

It depends on which filters you used, but there is LoadVirtualDubPlugin() in avisynth, and most of them work ok in avisynth
http://avisynth.nl/index.php/Plugins#LoadVirtualDubPlugin

Or you can try to "recreate" what you did in vdub using native avisynth filters

3) I use handbrake because I have settled on processing settings I'm happy with. My OCD won't let me deal with something else doing the compression as I won't be confident it is giving me the same quality/size ratio. Sad I know, but that's just me!

Ok, but handbrake is just a GUI or front end . You can get same or even better results with other programs

[manono]

It is set as huffyUV for output

I see that now. I apologize for not noting that in my earlier response.

You're saving as YV12, and not RGB? RGB will blow up the file size as well. I'm just thinking of ways for you to get the file sizes down under the impression you're blaming the crashes on the large file sizes you're getting.

[nick1977]

Just to let everyone know, I tried feeding the original file into virtualdub via the directshow filter in avisynth, and it worked fine.
Although I am not sure WHY the problem happened, I have found an acceptable workaround.
Thanks everyone for your help

[poisondeathray]

Just to let everyone know, I tried feeding the original file into virtualdub via the directshow filter in avisynth, and it worked fine.
Although I am not sure WHY the problem happened, I have found an acceptable workaround.
Thanks everyone for your help

In case other people have issues -

What method did you use to feed vdub originally that caused the issue ?

What was the original filetype and compression used ? (you can use mediainfo (view=>text) for more info )

Did you mean you opened the original file with an avs script (opened the avs script in vdub)? Because vdub also has a directshow input driver plugin

[nick1977]

The original file was just opened directly using virtualdub.
It was an mkv with x264 video stream, and 2xDTS soundtracks
I resolved by creating a basic avs file that just opened the mkv with the directshowsource command. I set audio=false

One thing I forgot to mention is that it is 2 versions of virtualdub. The first one I was trying with was virtualdubmod, whereas the 2nd was one I had managed to get to work successfully with avisynth (but could not open mkv files directly)

Hope this helps to clarify