Out-of-bounds memory error exporting virtualdub

[vidphil]

I keep getting this error when exporting from virtualdub:

"An out-of-bounds memory access (access violation) occurred in module 'ffmpeg'...
...reading address 353C4000"

I'm trying to deinterlace with QTGMC on a new system. Here's a list of my installed programs:

VirtualDub v1.10.4
Avisynth v2.6 MT
QTGMC v3.32
QTGMC 32-bit Plugins [Vit-Mod]
ffdshow rev4532


Here are my QTGMC settings:

SetMTMode(5, 4) # multithread mode for source filter
SetMemoryMax(400) # Optional line. Leave it out at first. See below for value M

AVISource("E:\_DOCS\_WORKING\_unprocessed\140630_BLANK-02_SE-3_35.50-38.25_.avi")

SetMTMode (2) # multithread mode for processing filters

AssumeTFF() 

QTGMC( Preset="slow", EdiThreads=8 ) #  1/2 physical cores

SelectEven() # deletes every other frame

I've tried everything I can think of including running ffmpeg as an external encoder, tried running different encoders, and still getting the error.

The only thing that seems to work is if I run QTGMC without any compression then it is able to complete.

My system has 16 cores and 32GB RAM, so I'm assuming assigning 4 threads and 400 max memory shouldn't be a problem. Previously I was getting great results with a very small test file using QTGMC @ 18 threads and 800 max memory.

The file I'm trying to process is 985MB and is also encoded FFV1. I've spent all day installing, uninstalling, upgrading, restarting and trying different options, but I keep getting this out of bounds error. Attached is the crash info, any help is appreciated.

[LigH.de]

Hard to know for sure; the reason might be a data error in your AVI file. It might also be a conflict due to using ffdshow/ffvfw for both decoding and encoding at the same time. You could try FFMS2 or L-SMASH Works instead of AviSource, they should be able to decode FFV1 as well and are native AviSynth decoder plugins, independent of ffdshow/ffvfw installed in your system.

...

Furthermore, I wonder about the high number of EdiThreads; I would recommend reducing them to 4 and test with the benchmark of AVSMeter how the balance between filtering speed and memory consumption changes. For so many threads (8 EDI threads per QTGMC instance * 4 QTGMC threads = 32 EDI threads overall), I am surprised that 400 MB RAM are enough and the thread synchronization doesn't choke. But if you say it was all fine, you will be right... yet, issues may arise after a higher number of processed frames only.

[vidphil]

Thanks for the reply!

I was originally setting the threads at 18 and memory at 800-1000, I set the edi threads to 8 because the documentation suggested half the cores. The documentation said to reduce the amount of memory in order to have more headroom for the number of threads, so when I started getting crashes I thought that was what was effecting stability so I kept trying lower numbers, though really I don't understand how anything does what and the documentation is pretty sparse.

I'll have to look into AVSMeter and figure out how to balance those settings.

It would be nice if it is simply an issue of a decoding/encoding bottleneck. I'll try those suggestions and get back to you!

[sneaker]

In addition to the things LigH suggested try a VirtualDub version that can use more memory: (see attachment or https://www.mediafire.com/?5wdk1o6wm7c94zz)
By default 32 bit processes are limited to 2 GB even on a 64 bit OS. This patched version can use 4 GB on a 64 bit OS. To use more you need 64 bit AviSynth (incl. all plugins) or VapourSynth.

[LigH.de]

The technical term of this feature is LAA = "Large Address Aware".

VirtualDub2 should be made with LAA support for the 32 bit build already. Using a 64 bit build of VirtualDub2 would also require a 64 bit build of ffdshow/ffvfw, which is not very usual; here I would strongly recommend FFMS2 or L-SMASH Works as source plugin instead of AviSource.

[vidphil]

So I think setting the edithreads to 4 helped, but I'm still having issues.

I was able to get a few more files processed, but still getting crashes in batch process, and there are errors within in the exported files such as blank video and glitched audio along with crashes.

I tried using AVSMeter and this is what I got:

F:\_DOCS\_DOWNLOADS\_APPS\_video_tools\AVSMeter278 >avsmeter 140803_BLANK-06_SESH-4_2.13-5.30_x-comp~lag.avs

AVSMeter 2.7.8 (x86) - Copyright (c) 2012-2018, Groucho2004
AviSynth 2.60, build:Feb 20 2015 [03:16:45] (2.6.0.5)

Number of frames: 5977
Length (hh:mm:ss.ms): 00:03:19.433
Frame width: 640
Frame height: 480
Framerate: 29.970 (30000/1001)
Colorspace: YUY2
Active MT Mode: 2
Audio channels: 2
Audio bits/sample: 16
Audio sample rate: 44100
Audio samples: 8816197

Frames processed: 5977 (0 - 5976)
FPS (min | max | average): 15.16 | 3238 | 115.7
Memory usage (phys | virt): 926 | 954 MiB
Thread count: 81
CPU usage (average): 49%

Time (elapsed): 00:00:51.662

I'm noticing the thread count is at 81 which seems too high. My system is 16 cores so I'm assuming 32 thread? How does the math work in the script to determine 81 threads? Here is the script I ran:

SetMTMode(5, 16) # multithread mode for source filter
SetMemoryMax(800) # Optional line. Leave it out at first. See below for value M

AVISource("E:\_DOCS\_WORKING\_unprocessed\140803_BLANK-06_SESH-4_2.13-5.30_x-comp~lag.avi")

SetMTMode (2) # multithread mode for processing filters

AssumeTFF() 

QTGMC( Preset="slow", EdiThreads=4 ) #  1/2 physical cores

SelectEven() # deletes every other frame

Also I did try installing and using ffms2, but it also seemed very glitchy and caused vdub to run very slow.

[vidphil]

Oddly enough I tried running it again removing the SetMemoryMax line and it ran slower as well as using less memory:

F:\_DOCS\_DOWNLOADS\_APPS\_video_tools\AVSMeter278 >avsmeter 140803_BLANK-06_SESH-4_2.13-5.30_x-comp~lag.avs

AVSMeter 2.7.8 (x86) - Copyright (c) 2012-2018, Groucho2004
AviSynth 2.60, build:Feb 20 2015 [03:16:45] (2.6.0.5)

Number of frames: 5977
Length (hh:mm:ss.ms): 00:03:19.433
Frame width: 640
Frame height: 480
Framerate: 29.970 (30000/1001)
Colorspace: YUY2
Active MT Mode: 2
Audio channels: 2
Audio bits/sample: 16
Audio sample rate: 44100
Audio samples: 8816197

Frames processed: 5977 (0 - 5976)
FPS (min | max | average): 24.91 | 91297 | 80.86
Memory usage (phys | virt): 676 | 697 MiB
Thread count: 81
CPU usage (average): 42%

Time (elapsed): 00:01:13.916

Don't understand why it would be using virtual memory when I have 32GB, am I misunderstanding what it is saying? Not sure how I'm supposed to use this information to optimized settings in the script.

[LigH.de]

If you run AviSynth in a 32-bit process, this process can use at most 2 GB RAM conventionally, or 4 GB RAM with "Large Address Aware" executables. And then you limit the RAM usage even more by using the SetMemoryMax directive, to only 800 MB.

To use more RAM where available, in AviSynth+ you should omit the SetMemoryMax line; but SEt's AviSynth MT may default RAM usage to 512 MB maximum, here you may need to increase it. You may even use 64 bit applications with AviSynth+, then it can use even more than 4 GB RAM. But AviSynth+ MT uses a different syntax, compared with SEt's AviSynth MT, to control multi-threading.

Regarding the thread count: AviSynth(+) MT – and its used plugins additionally – may allocate many threads for specific features; not all threads have to be active in parallel at the same time, though. But the more threads have to share the CPU power, the higher the chance that some have to wait for intermediate results of others. Overall, the CPU utilization will saturate. Twice the number of threads will not result in twice the calculation speed. And too many threads per frame may result in a too narrow scope of each thread processing only a slice of the frame. But finding the optimum requires some experimenting with all parameters which deal with parallelization. The number of AviSynth MT filter threads = parallel QTGMC calls and the number of EDI threads inside each QTGMC instance are possibly only two factors, there may be more.

[Zeppo]

For best efficiency with multi-threaded Avisynth, edithreads and dftthreads should be set to 2 (max), sometimes even 1 is better. Use AVSMeter to see which performs better.

[vidphil]

So I ran through AVSMeter and everything seems to be working fine with Avisynth. I noticed reducing edithreads to 1 reduced CPU usage to around 30%.

When I go back to exporting with Virtualdub, I'm still getting glitchy audio and out of bounds memory errors when I use compression. I even got an out of bounds error encoding HuffYUV.

What software is handling the compression, is it Virtualdub or Avisynth?

I would consider using AviSynth+ but with all the issues I don't know if the plugins for QTGMC are all compatible, plus I'm already drowning in trying to figure out all the correct syntaxes for everything.

I tried setting up ffmpeg as an external encoder but get this error:

Write error occurred on file "<anonymous pipe>": The pipe is being closed.

This is the .vdprof content I put together:

{
"description": "VirtualDub external encoder profile collection",
"externalEncoders": {
  "sets": {
  "ffmpeg FFV1": {
  "videoEncoder": "ffmpeg FFV1",
  "audioEncoder": "ffmpeg AUDIO COPY",
  "multiplexer": "ffmpeg AVI",
  "description": "avi",
  "extension": "avi",
  "processPartial": true,
  "useOutputAsTemp": false
  }
  },
  "profiles": {
  "ffmpeg AVI": {
  "name": "ffmpeg AVI",
  "program": "C:\\Apps\\ffmpeg-20180630-9f0077c-win64-static\\bin\\ffmpeg.exe",
  "commandArguments": "-i \"%(tempvideofile)\" -i \"%(tempaudiofile)\"  -vcodec copy -acodec copy -r %(fps) -f avi \"%(outputname)\"",
  "outputFilename": "%(outputname).audio",
  "type": 2,
  "inputFormat": 0,
  "checkReturnCode": true,
  "logStdout": true,
  "logStderr": true,
  "bypassCompression": false,
  "predeleteOutputFile": true
  },
  "ffmpeg FFV1": {
  "name": "ffmpeg FFV1",
  "program": "C:\\Apps\\ffmpeg-20180630-9f0077c-win64-static\\bin\\ffmpeg.exe",
  "commandArguments": "-i - \"%(tempvideofile)\" -f rawvideo -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -vcodec ffv1 -level 3 -coder 0 -context 1",
  "outputFilename": "%(outputname).avi",
  "type": 0,
  "inputFormat": 0,
  "checkReturnCode": true,
  "logStdout": true,
  "logStderr": true,
  "bypassCompression": false,
  "predeleteOutputFile": true
  },
  "ffmpeg AUDIO COPY": {
  "name": "ffmpeg AUDIO COPY",
  "program": "C:\\Apps\\ffmpeg-20180630-9f0077c-win64-static\\bin\\ffmpeg.exe",
  "commandArguments": "-i - -acodec copy \"%(tempaudiofile)\"",
  "outputFilename": "%(outputname).wav",
  "type": 1,
  "inputFormat": 1,
  "checkReturnCode": true,
  "logStdout": true,
  "logStderr": true,
  "bypassCompression": true,
  "predeleteOutputFile": true
  }
  }
}
}

[LigH.de]

What software is handling the compression, is it Virtualdub or Avisynth?

Clearly VirtualDub. AviSynth only works with uncompressed video and audio.

VirtualDub2 – former VirtualDubFilterMod – already contains an internal FFV1 codec and does not need piping to an external ffmpeg.

Differences between AviSynth and AviSynth+ MT control are explained in the AviSynth Wiki: AviSynth+ — MT Notes. Most important: Don't care so much anymore about switching modes explicitly, it is handled implicitly per each function (AviSynth+ is already shipped with a lot of well tested default values). And instead of initializing an MT mode in the beginning, you will use Prefetch(threads) in the very end of a script (if you use an explicit return clip for the whole script {not in a function!}, then right in front of that).

[sneaker]

"program": "C:\\Apps\\ffmpeg-20180630-9f0077c-win64-static\\bin\\ffmpeg.exe",
"commandArguments": "-i - \"%(tempvideofile)\" -f rawvideo -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -vcodec ffv1 -level 3 -coder 0 -context 1"

The order is important. Input options like -f rawvideo, -s %(width)x%(height) etc. belong before the "-i". Otherwise ffmpeg will treat them as output options.

Look into the VirtualDub log (F8) when you have trouble. It should give you hints about what's going wrong. (Depending on encoder you need to redirect stderr/stdout to log.)

[vidphil]

Omg, I finally got the external encoder option to work using ffmpeg. I used example profiles provided here:
https://encodingtalk.com/threads/virtualdub-save-direct-to-mp4-fraps-to-youtube-with-e...e-2#post-13731

I tested the examples and was able to run fine. Then slowly switched out the video, audio, and multiplexer profiles to my own FFV1, audio copy, and AVI profiles. The main thing being able to use FFV1 v3 which runs multi-threaded which I was hoping would run more stable.

I was getting stable results, but for some reason when I switched over to my AVI multiplexer profile I started getting hangs. Now even when I switch back to the example profiles I'm getting out of bounds errors caused by lagarith, decoding the source file!

Tearing my hair out!

Will VDub2 work with QTGMC and plugins?

Should AVISynth+ work with QTGMC and plugins, or should I stick to AVISynth 2.58 or 2.6 + MT?

Which QTGMC plugin pack should I use, default, modded or 2.6 interface?
https://forum.doom9.org/showthread.php?t=156028

[vidphil]

Tried VDub2. Used the native FFV1 encoder, still got a crash - access violation by lagarith

[LigH.de]

Will VDub2 work with QTGMC and plugins?

VirtualDub2 receives uncompressed video from AviSynth. It cannot know whether QTGMC was used in the AviSynth script or not. Therefore I see no reason why it should not work with VirtualDub2 specifically, but with other applications. And on top: The developers of AviSynth+ and VirtualDub2 cooperate tightly, making either one support what the other needs to improve each other.

Should AVISynth+ work with QTGMC and plugins, or should I stick to AVISynth 2.58 or 2.6 + MT?

Of course!!! QTGMC works even better with the features AviSynth+ has more than the legacy AviSynth, especially with the advantageous new planar color modes (e.g. YV16 instead of YUY2).

Which QTGMC plugin pack should I use, default, modded or 2.6 interface?

Trust in the collections listed in the AviSynth Wiki: QTGMC – Requirements. The "Core Plugins and scripts" are necessary, "Optional Plugins and Scripts" are only required for a few specific operation modes. Also, don't forget the FFT3W libraries to be put in the matching Windows system directories...

AviSynth+ also has a more convenient system to handle plugins, especially having separate versions for 32 and 64 bit code.

Tried VDub2. Used the native FFV1 encoder, still got a crash - access violation by lagarith

Uhm, why?! ... Is your source video encoded with Lagarith, so actually the decoder crashes? Or why should the Lagarith encoder fail when you select the FFV1 encoder? Strange.

[vidphil]

Yea, the source is encoded lagarith, so I'm assuming the decoder is now causing an out of bounds error??

I am trying to deinterlace and bunch of older captures that are in various formats ranging from huffyuv, lagarith, to ffv1.

I like VDub2, the native ffv1 encoder makes me feel a little more confident, but I'm still getting the out of bounds error from lagarith.

Trying to do a little process of elimination this is what I'm finding:

Transcode Lagarith -> FFV1 [OK]
QTGMC (threads>2) -> No Compression [OK]
QTGMC (threads<=2)-> FFV1 [AUDIO GLITCHES] very slow
QTGMC (threads>2)-> FFV1 [ERROR]

I guess I'll try setting it up with AviSynth+ next, but need to take a break from this today.

Thanks for the input!

[LigH.de]

The reason might be that the Lagarith VfW codec does not like multiple threads accessing different frames in the video... the different MT code in AviSynth+ might indeed help. Or ... I am not sure whether FFMS2 or L-SMASH Works can decode Lagarith at all, and maybe even more stable than the VfW codec. Well, another bit of hope.

And another way: Convert the Lagarith AVI directly to FFV1 before you try to deinterlace that copy. FFMS2 and L-SMASH Works as AviSynth decoder plugins can surely decode FFV1, and it is possibly a more robust codec, even in multi-threaded access.

We will get it done!

[vidphil]

Ok, so I installed AviSynth+ and updated my script as follows:

SetFilterMTMode("DEFAULT_MT_MODE", 2)
SetFilterMTMode("AVISource", 3)

AVISource("E:\_DOCS\_WORKING\_VIDEO_EFFECTS\_2014\140702-vid_sesh_w_mark\_unprocessed\140803_BLANK-06_SESH-4_2.13-5.30_x-comp~lag.avi")
AssumeTFF() 
QTGMC( Preset="slow", EdiThreads=1 )
SelectEven()

Prefetch(4)

I'm getting an error:

Avisynth open failure:
Script error: There is no function named 'SetFilterMTMode'.

I followed the documentation here:
http://avisynth.nl/index.php/AviSynth%2B#MT_Notes

Confused

[Zeppo]

Run "AVSMeter -avsinfo" and post any reported errors.

[vidphil]

Here you go:

AVSMeter 2.7.8 (x86) - Copyright (c) 2012-2018, Groucho2004

VersionString: AviSynth+ 0.1 (r1576, x86)
VersionNumber: 2.60
File / Product version: 2.6.0.5 / 2.6.0.5
Interface Version: 5
Multi-threading support: No
Avisynth.dll location: C:\WINDOWS\SysWOW64\avisynth.dll
Avisynth.dll time stamp: 2014-01-02, 23:14:26 (UTC)
PluginDir2_5 (HKLM, x86): C:\Apps\AviSynth+\plugins
PluginDir+ (HKLM, x86): C:\Apps\AviSynth+\plugins+


[C 2.5 Plugins (32 Bit)]
C:\Apps\AviSynth+\plugins\yadif.dll [1.7.0.0]

[CPP 2.5 Plugins (32 Bit)]
C:\Apps\AviSynth+\plugins\AddGrainC.dll [1.5.2.0]
C:\Apps\AviSynth+\plugins\dfttest.dll [1.8.0.0]
C:\Apps\AviSynth+\plugins\EEDI2.dll [0.9.2.0]
C:\Apps\AviSynth+\plugins\eedi3.dll [0.9.1.0]
C:\Apps\AviSynth+\plugins\FFT3DFilter.dll [2.1.1.0]
C:\Apps\AviSynth+\plugins\mt_masktools-25.dll [2.0.48.0]
C:\Apps\AviSynth+\plugins\mvtools2.dll [2.5.11.2]
C:\Apps\AviSynth+\plugins\nnedi.dll [1.3.0.0]
C:\Apps\AviSynth+\plugins\nnedi2.dll [1.6.0.0]
C:\Apps\AviSynth+\plugins\nnedi3.dll [0.9.4.0]
C:\Apps\AviSynth+\plugins\RemoveGrainSSE2.dll [2011-04-19]
C:\Apps\AviSynth+\plugins\RepairSSE2.dll [2011-04-19]
C:\Apps\AviSynth+\plugins\SSE2Tools.dll [2005-04-11]
C:\Apps\AviSynth+\plugins\TDeint.dll [1.1.0.0]
C:\Apps\AviSynth+\plugins\VerticalCleanerSSE2.dll [2011-04-19]

[CPP 2.6 Plugins (32 Bit)]
C:\Apps\AviSynth+\plugins+\DirectShowSource.dll [2014-01-02]
C:\Apps\AviSynth+\plugins+\ImageSeq.dll [2014-01-02]
C:\Apps\AviSynth+\plugins+\Shibatch.dll [2014-01-02]
C:\Apps\AviSynth+\plugins+\TimeStretch.dll [2014-01-02]
C:\Apps\AviSynth+\plugins+\VDubFilter.dll [2014-01-02]

[Scripts (AVSI)]
C:\Apps\AviSynth+\plugins+\colors_rgb.avsi [2014-01-03]
C:\Apps\AviSynth+\plugins\FFMS2.avsi [2015-05-22]
C:\Apps\AviSynth+\plugins\QTGMC-3.32.avsi [2011-06-07]

[Uncategorized files]
C:\Apps\AviSynth+\plugins+\colors_rgb.txt [2014-01-03]

Why does it say multi-threading is not supported, am I supposed to replace avisynth.dll with the one here?
https://forum.doom9.org/showthread.php?t=148782

[LigH.de]

Oh, AviSynth+ r1576 ... that's the completely obsolete ancient version from their abandoned website. Not even an MT build.

Current MT versions of AviSynth+ are available in pinterf's releases. Due to very recent changes involving plugin DLL's too, try r2664-MT first, or read the linked thread in the doom9 forum from the end back to find MaskTools2 and other plugins matching the latest r2728-MT.

It's hard to keep up with their development efforts, sorry... but better than a lack of activity.

[vidphil]

Heeeeeeey, it worked! So I guess upgrading to Virtualdub2 (41980) and Avisynth+ MT (r2728) did the trick.

Finally getting stable QTGMC exports encoded with FFV1 with no glitches!

It was confusing for me because there was no singular source of information as to what is the lastest/greatest version with so many plugins and forks. I appreciate this is all for free being developed by folks on their spare time, but definitely a maze of information.

My only other question is optimizing. Increase EdiThread?


I saw this list of MT modes:
https://gist.github.com/tp7/8899021

Is this something that is worth including in my script? And if so how do I include it?

This is my current script:

SetFilterMTMode("DEFAULT_MT_MODE", 2)
SetFilterMTMode("AVISource", 3)

AVISource("E:\_DOCS\_WORKING\_unprocessed\140803_BLANK-06_SESH-4_2.13-5.30_x-comp~lag.avi")
AssumeTFF() 
QTGMC( Preset="slow", EdiThreads=1 )
SelectEven()

Prefetch(4)

I really appreciate you guys sticking with me to try and figure this out, hopefully if someone else comes along with the same problem they can take the shortcut.

If there are any other things to consider optimizing render times or stability feel free to let me know. Thaaaaaaaaaanks! ✌️✌️✌️✌️✌️

[Zeppo]

My only other question is optimizing. Increase EdiThread?

How about simply trying it out?

I saw this list of MT modes:
https://gist.github.com/tp7/8899021

Is this something that is worth including in my script? And if so how do I include it?

The latest version can be found here:
http://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest

Name it "mtmodes.avsi" and put it in your Avisynth auto-load directory.

[vidphil]

Okay, I'll give it a try. Thanks!