Exporting 4K x.264 clips from Virtualdub... error on most presets

[pippas]

As the title suggests, I've been trying to export some edited 4K files from within Virtualdub, using the x.264vfw codec. The task always fails, except when using the 'Ultrafast' preset.

I have added the extra instructions recommended by Jagabo in post #3 of THIS THREAD and they work fine....but if I try and use any of the other presets, it will always crash Vdub with an 'unknown error'....

I know I'm pushing my luck try to edit and export 4K video on a 32 bit machine, but converting the original clips to an intraframe intermediate (I'm using Grass Valley HQX) makes the whole process possible... in fact the timeline scrubs are still quite good!

But which parameter - or parameters - in any of the presets other than Ultrafast is causing the process to guarantee to crash I've been unable to discover..... any ideas?...

[sneaker]

Try large address aware VirtualDub or use the "external encoders" feature to pipe to x264.exe.

https://support.microsoft.com/kb/291988

[pippas]

Thanks for that ... I think you must be right - I've discovered that I can in fact use the 'Superfast' preset, as well as the 'Ultrafast' - but checking the RAM usage on resmon shows the extra memory being used, and maxing out...

So, by the time I get to try 'Veryfast' - or any of the others - it just crashes.

I only have 4GB of RAM and a 32 bit system, so I'm not sure that using Virtualdub aa would be of any help?....

I've downloaded it (thanks for the link) but I'm afraid I've no idea how it differs from the standard Virtualdub1.10.4 - or how it's actually used?....

Thanks for the suggestions....

[poisondeathray]

Likely the reason is --rc-lookahead , which is zero for superfast, ultrafast, but 10 for veryfast . The lookahead means those frames are decoded and stored in memory, so the larger the lookahead value, the more memory consumption . The other settings have impact on memory too, but not as much as lookahead, especially with a 4K frame dimension. You can try veryfast with --rc-lookahead 0 , if that doesn't crash , maybe try faster with --rc-lookahead 0, and so forth

Piping to 64bit x264 won't help , since you're on a 32bit OS

[sneaker]

Piping to 32 bit x264 might, though. 2G for x264 alone, not shared with VirtualDub/AviSynth/audio encoder. Even with preset medium (--threads 4) x264 seems to stay below 2G when encoding 3840x2160 for me.

That VirtualDub version is used like the vanilla VirtualDub. On 64 bit OS it works out-of-the-box, on 32 bit OS I believe you might need to do this first.

[pippas]

Thanks again for the suggestions.....

@PDR -- I added the "--rclookahead 0" command, and indeed the Veryfast preset now works..... It does bring up the following warning though --

" :264vfw [warning]: lookaheadless mb-tree requires intra refresh or infinite keyint"

...I'm afraid I have no idea what that means!... I must look at the resultant clip, to see if anything horrible has happened..

@sneaker -- I shall have a read of the Project Reality thread .. although I think it may be time to seriously start thinking about upgrading to 64 bit!

I'm guessing that using the fastest presets primarily results in larger file sizes, rather than significantly poorer quality encodes?

....That seem to be the suggestion that Jagabo makes, in his suggestion to add the "-aq-mode=2 --aq-strength=2.0 --subme=6" commands to help correct minor artifacts that the faster encodes might create...

[poisondeathray]

Piping to 32 bit x264 might, though. 2G for x264 alone, not shared with VirtualDub/AviSynth/audio encoder. Even with preset medium (--threads 4) x264 seems to stay below 2G when encoding 3840x2160 for me.

Yes, good point ; worth a try

@PDR -- I added the "--rclookahead 0" command, and indeed the Veryfast preset now works..... It does bring up the following warning though --

" :264vfw [warning]: lookaheadless mb-tree requires intra refresh or infinite keyint"

...I'm afraid I have no idea what that means!... I must look at the resultant clip, to see if anything horrible has happened..

It shouldn't be a problem, but you can explicitly disable mb-tree with --no-mbtree

The idea behind mbtree is it analyzes macroblocks temporally across adjacent frames and adjusts to improve compression efficiency. The longer the lookahead interval, the better (with diminishing returns of course). But with zero lookahead, it's basically useless. mb-tree is disabled with ultrafast and superfast presets by default (because lookahead is zero)

[pippas]

I've decided that for the present I shall continue to use Ultrafast (Superfast and Veryfast do also work -- but only sometimes!)

I'm clearly right at the limit of my 32 bit 4GB RAM machine - understandably - and I think I shall look at moving up to a 64 bit machine soon, rather than try and 'squeeze' any more out of this one...

Thanks again both for your suggestions and advice...

[jagabo]

I'm clearly right at the limit of my 32 bit 4GB RAM machine - understandably - and I think I shall look at moving up to a 64 bit machine soon

That may not help unless you use 64 bit VirtualDub (assuming you continue to use VirtualDub). That means you'll also need 64 bit x264vfw, 64 bit AviSynth, and 64 bit versions of all the filters you're using.

[pippas]

I'm clearly right at the limit of my 32 bit 4GB RAM machine - understandably - and I think I shall look at moving up to a 64 bit machine soon

That may not help unless you use 64 bit VirtualDub (assuming you continue to use VirtualDub). That means you'll also need 64 bit x264vfw, 64 bit AviSynth, and 64 bit versions of all the filters you're using.

In that case I may stick with my 32 bit system for a little while longer!........I think I can live with only using the Ultrafast preset. There doesn't seem to be much quality loss between the various presets at 720p.... I have no problem with those..

I'm hope that the addition of the "-aq-mode=2 --aq-strength=2.0 --subme=6" commands (which you mentioned in the earlier thread I linked to) will mean that there is not too much quality loss from only using the fastest preset, against than any of the slower ones?...... I must try out some 720p comparisons.

I'm not bothered about any extra file size....

[sneaker]

I'm clearly right at the limit of my 32 bit 4GB RAM machine - understandably - and I think I shall look at moving up to a 64 bit machine soon

That may not help unless you use 64 bit VirtualDub (assuming you continue to use VirtualDub). That means you'll also need 64 bit x264vfw, 64 bit AviSynth, and 64 bit versions of all the filters you're using.

It can still help if he uses the VirtualDub 32 version I linked above (with large address aware flag). Then 64 bit Windows will use up to 4 GB per 32 bit process instead of 2 GB on 32 bit Windows with default settings. Of course 64 bit VirtualDub could use more but those 4 GB could be sufficient. (and there's still the "external encoders" solution to combine VirtualDub 32 bit with x264 64 bit)