Stabilizing Avisynth MT

[MrBiggles]

I have Avisynth 2.58 installed and I use the Avisynth MT version 2.60 Build May 16 2012. I'm an experienced Avisynth user but I don't know how to fix this issue. I know MT isn't completely stable and maybe once a week you'll get a failed encoding. That's normal and I'm fine with that. This is the MT setting I use with my 8 core CPU:

setmtmode(5,6)
File name here
setmtmode(2)

But the problem is after a few months have passed and I install several unrelated programs, Avisynth will crash constantly when encoding standard definition to 720 x 576i with McTemporalDenoise Medium and QTGMC Slower in MT Mode. The files I'm encoding each take 6 hours to encode and if I'm lucky 1 out of 5 videos will encode without crashing. The errors it gives are meaningless error codes.

I don't install loads of rubbish on my PC. I install everything I need then I create a backup image of the C Drive. If in later months I install later stuff then I just load the old backup, install the new stuff then create a new backup. So I can't see anything installed that would cause Avisynth to crash. I've also tried updating to the latest Avisynth MT but it didn't solve the problem.

I would like to fix this once and for all. Can anyone please help me find out the cause of the problem?

[BÌG•bµdd]

SetMTmode(int mode,int threads)
Place this at the first line in the avs file to enable temporal (that is more than one frame is processed at the same time) multithreading. Use it later in the script to change the mode for the filters below it.

mode int (2, default 1-6)
- there are 6 modes 1 to 6.
   * Mode 1 is the fastest but only works with a few filter
   * Mode 2 should work with most filters but uses more memory
   * Mode 3 should work with some of the filters that don't work with mode 2 but is slower
   * Mode 4 is a combination of mode 2 and 3 and should work with even more filter but is both slower and uses more memory
   * Mode 5 is slowest (slower than not using SetMTMode) but should work with all filters that don't require linear frameserving (that is, the frames come in order (frame 0,1,2 ... last)).
   * Mode 6 is a modified mode 5 that might be slightly faster

start like this
Setmtmode(5,X)#X is number of your cpu threads

setmtmode(2,6)#mode 6

hope this helps you , i do read about avisynth filter in free time ( http://avisynth.nl/index.php/MT#MT )

[MrBiggles]

Thanks but it's not the MT mode that's the problem. I've tried various settings and the ones I use is the most stable for me and others have verified that's the best setting to use. I use 5,6 instead of 5,8 so I have 2 cores spare for non-video editing tasks.

[BÌG•bµdd]

did u tried?
Setmemorymax(1024)
Setmtmode(5,4)

Setmode(2,6)

[poisondeathray]

I would like to fix this once and for all.

The fix once and for all is don't use MT with temporal filters


You've already tried all the other modes. The most common culprit for avisynth MT crashes is memory (e.g. HD processing with x86 avisynth), but that shouldn't be an issue on your 720x575 SD encodes. You can try splitting up the filters into separate stages and lossless encodes (QTGMC first, then MCTD on a 2nd stage)

If it seems to be occurring more frequently, you might want to do some hardware stability tests (if overclocked, go back to default settings) , and do some memory integrity checks

You think it's just a random failed encode here and there, but that would be good IMO - because then you'd know at least something was wrong.

Another insidious issue is corrupt and/or misordered frames that you don't know about until actually watching it later, despite using frame accurate source filters. "Temporal anything" and MT anything don't mix in avisynth IMO . So unless you want to go through each encode to double check... That non knowing if there is a problem is much worse IMO

When the "most stable" is 1/5 work, you got to wonder if it's worth it ... You'd be done without errors much faster using parallel processing (eg. encode 2 or more videos simultaneously)

[poisondeathray]

Another approach you might want to look at is vapoursynth, since QTGMC is natively ported now. I haven't seen full benchmarks for QTGMC, but some of the individual components like mvtools2, etc... are anywhere from 5-50% faster in vapoursynth because of native mulithreading, 64bit (some cases are >100% faster than avisynth MT)

[MrBiggles]

What I don't understand though is that I could be encoding SD videos 24 hours a day for months and I might only get 1 failure a week but months later I get constant failures. Why?

I've gone back to default non-overclocked motherboard settings. I'll have to wait a day or so to see if any encodings fail or not.

How do I do memory integrity checks? Is there anything else I can do to diagnose the problem or do you just think the problem is that MT is unstable?

If I recall correctly I previously tried encoding to lossless AVI with just McTemporalDenoise interlaced true and then I encoded the AVI to h264 with QTGMC and re-interlaced afterwards. I got interlacing artifacts or some other issue which I think is due to the video being deinterlaced twice which wouldn't happen if the plugins were all run in the same script. Tried doing McTemp first then QTGMC second and vice versa but it made no difference.

I don't really know much about Vapousynth.

I was wondering, do think anybody would be interested in contributing to a kickstarter to get McTemperalDenoise and QTGMC to be stable in MT if the author of MT would agree to it?

Do you think if I get an 8 core (16 virtual cores) that I could run McTemp Medium & QTGMC Slower at 6fps in non-multi-threaded?

[poisondeathray]

Does anything other than avisynth MT act "buggy" ? Do some games crash for example ?

Decreasing performance over time, high frequency of crashes definitely sounds like either hardware issues, unstable overclock (check your temps, it's summer in some parts of the world, so much hotter), or OS/system/driver issues

Run something like memtest 86+ to check the memory, but bad/corrupt memory usually gives you irrecoverable crash

Run some stress tests like prime 95, occt, linx, etc...

You should have known all this before you overclock a system

[newpball]

I've gone back to default non-overclocked motherboard settings. I'll have to wait a day or so to see if any encodings fail or not.

How do I do memory integrity checks? Is there anything else I can do to diagnose the problem or do you just think the problem is that MT is unstable?

I presume you did stability test for your overclocked rig?

How does prime95 fare?

[MrBiggles]

I've gone back to default non-overclocked motherboard settings. I'll have to wait a day or so to see if any encodings fail or not.

How do I do memory integrity checks? Is there anything else I can do to diagnose the problem or do you just think the problem is that MT is unstable?

I presume you did stability test for your overclocked rig?

How does prime95 fare?

Yes I did but I used Y-Cruncher. But it's irrelevant as I still get the same issues when NOT overclocked.