FFVideoSource vs LWLibavVideoSource

[midts]

I'm running a 5-6 year old system to encode .mkv or h264 files. I need to understand which approach is best? As I understand the L-SMASH (LWLibavVideoSource) that comes with MeGUI by default is newer than FFMS?

My problem is that when previewing videos in MeGUI indexed with L-SMASH and applying various changes to the .avs script file, it lags the preview and the computer like crazy, you can't even jump or navigate through frames normally. Meanwhile I get none of that with FFMS, preview runs real smooth when applying various filters to avs or jumping around frames.

So my questions are:

- So FFMS is older, but also faster, why?
- Which is better FFVideoSource vs LWLibavVideoSource?
- What do I lose using FFMS over L-SMASH?
- If no big loss over using FFMS, how do I make MeGUI select FFMS as a default indexer, because now it's always locked into L-SMASH when indexing.

I did a 5 minute test encode, the process rate was 1-2 fps difference between two, so no performance improvement. But what I noticed is that FFMS encode gave ~100-150kbps lower bitrate file than L-SMASH one with 100% same settings.

[LigH.de]

For a long time, FFMS2 depended on Haali Media Splitter. This time is over now. Since both FFMS2 and L-SMASH Works now mainly use the same libraries to access containers (libavformat) and to decode content (libavcodec) in current versions (for MeGUI, have the development update server selected!), differences should be negligible. The format of the index file differs, though, and FFMS2 also has an external indexer tool.

The different index format might be a reason for different seeking behaviour. But that's just guessing. AviSynth is not made for playback, primarily. To benchmark either decoding speed, please use AVSMeter only.

Because both work almost the same way, there should be no differences in your conversion results, because both are only decoders, and both should provide the same video content decoded from the same source. Neither should be "better" regarding decoding results and quality. At least in theory. I don't know where exactly any differences are left over.

But what I noticed is that FFMS encode gave ~100-150kbps lower bitrate file than L-SMASH one with 100% same settings.

For now, this is only a claim without proof. Let us compare both scripts and MeGUI logs. Let us compare the result of an Info() overlay at the end of both scripts.

[raffriff42]

I find FFMS sometimes gives smoother playback and searching, but I normally use LibAV these days.

* One advantage LSMASH has over the others is that it doesn't need to create an index file for its supported formats
(it can make an in-memory index only; advantage = no index files laying around; disadvantage = index must be rebuilt every time)

* Libav supports video codecs LSMASH does not – for example MPEG-4, UT Video, Lagarith, and animated GIF and PNG.

* FFMS2 requires the "10bithack" version to decode high bit depth; LibAV and LSMASH were made to support 10 bit color.

See the LSMASHSource Talk page for a codec/container compatibility test I posted back in 10/16.

EDIT it's not used too often, but FFMS can display the picture type (I-frame, B-frame etc):

FFmpegSource(...)
ScriptClip(Last, """Subtitle("[ "+chr(FFPICT_TYPE)+" ]", align=2) """, after_frame=true)

[LigH.de]

* One advantage LSMASH has over the others is that it doesn't need to create an index file for its supported formats
(it can make an in-memory index only; advantage = no index files laying around; disadvantage = index must be rebuilt every time)

Not exactly: If it uses L-SMASH as demultiplexer for supported ISO Media containers (instead of libavformat), then it relies on these containers providing an index chunk, which is read out of the file, so it does not scan the content streams on its own.

* Libav supports video codecs LSMASH does not – for example MPEG-4, ...

Which one, exactly? MPEG-4 is a specification "generation", not one codec (like MPEG-4 Part 2, a.k.a. {Advanced} Simple Profile, implemented as e.g. DivX, Xvid, 3ivx...).

[midts]

Hey team, I work only with .mkv container and h264 format. Like I said FFMS is smooth and no lag on older machine while L-SMASH is choking and while loading e.g. a preview of a 720p resolution file after applied some filters to .avs, MeGUI goes into frozen mode for ~1-3 seconds and if u clicked somewhere on the interface or something there's a chance for it to crash. I don't know how to explain this. With FFMS there is no lag and delay or anything when doing same type of work with HD files.

So I personally chose FFMS now, only problem is that MeGUI has LSMASH set as default indexer even after disabling the addon in the update window it still ticks LSMASH as default and re-enables it.

So if you work with 50 files a day you have to tick FFMS manually for each time indexing and there is no setting to tick default indexer...

edit: looks like LibAV does not support avisynth 2.6 according to the comments in the download page.

[LigH.de]

edit: looks like LibAV does not support avisynth 2.6 according to the comments in the download page.

You may be looking in the wrong place. This libav.exe is just like ffmpeg.exe, but not a source plugin for AviSynth. I believe "libav" is used ambiguously as an abbreviation for anything what might be L-SMASH Works or LAV Filters... I am not sure.

Both L-SMASH Works (LSMASHSource.dll, providing LwLibav{Video|Audio}Source in AviSynth) and FFMS2 (FFMS2.dll, providing FF{Video|Audio}Source in AviSynth) do support AviSynth 2.6, even AviSynth+. Finding the latest version may be a bit of challenge though.

FFMS2 is released on github; L-SMASH Works is available on DropBox (you may prefer the MSVC builds for one static DLL each for AviSynth).

Regarding your request to be able to change the preference of the source filter in MeGUI's File Indexer dialog, I pointed the developer (Zathor) over here from the doom9 forum. I hope he can implement an option in its Settings.

[midts]

Maybe it's an old comment https://www.videohelp.com/software/Libav/comments#12402

What about the lag I get with L-SMASH?

[LigH.de]

Again: "Libav 12" is not an AviSynth source plugin.

Regarding the lag: You still did not post a pair of equivalent scripts / logs to compare. Do they use threads=1 as limitation? And as I said, there are still small differences, even though they should be negligible now, but it's possible that they handle one or another container format differently; I remember interlaced AVC in TransportStreams being challenging.

[midts]

No, threads is set to 0. That's not the point, the lag is no in the encoding part, but on the preview part of working on the .avs script or simply clicking buttons like OK or adding new job to Queue the MeGUI always hangs/doesn't respond for 1-3 seconds with L-SMASH indexed file.

My scripts are simple e.g.

LoadPlugin("C:\MeGUIx64\tools\ffms\ffms2.dll")
FFVideoSource("C:\jobs\vid.mkv", cachefile="C:\jobs\vid.ffindex", fpsnum=30000, fpsden=1001)
LoadPlugin("C:\MeGUIx64\tools\avisynth_plugin\TIVTC.dll")
SetMemoryMax(7168)
tdecimate()
#crop
Spline64Resize(1280,720) # Spline64 (Sharp)
#denoise

With L-SMASH its same just replace FFMS lines with

LoadPlugin("C:\MeGUIx64\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("C:\jobs\vid.lwi")

and something like this

program --level 4.1 --preset slower --crf 23.0 --deblock -3:-3 --me umh --trellis 1 --sar 1:1 --output "output" "input"

But what it has to do with lag on the interface with L-SMASH indexed files while building the .avs script.

[LigH.de]

Index files are written once to prepare the decoding. But they are read each time you decode a file, used to calculate seek points to previous GOP starts. If many decoding threads are used, this might cause invalidation of a frame cache content, forcing a decoder to seek backwards and decode the same GOP again. Limiting the decoding threads might help.

SetMemoryMax(7168) is utter nonsense. This script does not need to allocate up to 7 GB RAM.

Also I wonder about fpsnum=30000, fpsden=1001 in your FFVideoSource call. Forcing a constant frame rate of material which already may have a constant frame rate might still delete or duplicate frames occasionally, confusing your IVTC attempt. Does MeGUI include that always for FFMS2? At least it might cause a larger number of pre-decoded frames.

But there are better ways to provoke a bigger frame cache than risking discontinuations due to rounding errors. TIVTC contains the function RequestLinear(), which might be useful to let the source plugin decode frames ahead of the output.

[midts]

MeGUI includes the fpsnum=30000, fpsden=1001 to a 29.976 fps .mkv by default. I don't know about SetMemoryMax(7168), I started using it after googling some old posts on doom9 and etc where people claimed it might prevent from crashing etc. So I just added this to my default avs script profile and it gets loaded by default, I don't think it hurts.

[LigH.de]

You don't use extremely complex and memory consuming scripts (like QTGMC) in a multithreaded run. No, you don't need that much memory. As simple as scripts generated by MeGUI are, trust in the AviSynth kernel allocating only as much as required. Remember, your encoder or player needs RAM too. When you don't keep some RAM free for other applications, the swap file may have to be utilized, and that will slow down the process for sure.

[hello_hello]

Do you have Avisynth installed (in addition to MeGUI's portable version)? If you do, try opening an LSmash script and an ffms2 script with MPC-HC or VirtualDub. Both scripts opening the same video, of course. Does the LSmash script still take longer to open? I found it did, but I'm still running XP and using the last XP compatible version of LSmash, which is older than the current ffms2, so you mightn't get the same result with a newer LSmash, although others have made the same complaint.

https://forum.doom9.org/showthread.php?p=1796312#post1796312
https://forum.doom9.org/showthread.php?p=1796877#post1796877

MeGUI includes the fpsnum=30000, fpsden=1001 to a 29.976 fps .mkv by default.

It only adds it for ffms2 (not L-Smash) but be careful with that. I'd hoped fpsnum & fpsden would have been removed from the scripts by now.

The idea, which seemed like a good one at the time (it was mine) was for fpsnum & fpsden to be added so MeGUI could encode variable frame rate sources by converting them to a constant frame rate, and while not necessarily ideal, it'd output something sensible. Without fpsnum & fpsden, both ffms2 and L-Smash output the average frame rate for variable frame rate sources, which at best puts the audio out of sync or at worst makes more of a mess. For constant frame rate sources, fpsnum & fpsden "should" have no effect if MeGUI gets the frame rate correct, but unfortunately, it's not that easy.

The problem is (probably due to frame rate jitter) both ffms2 and L-Smash will sometimes unnecessarily drop/repeat frames when fpsnum & fpsden are used, even for constant frame rate sources (when I say "sometimes," in my experience either most of an encode ends up "jittery", or the whole encode is fine). Fortunately though, if you do need to convert to constant frame rate, one of them usually works fine. If one indexer makes the output "jittery", try the other.

For "normal" sources though, try opening the video with ffms2 and checking the total frame count and duration using the preview, then delete fpsnum & fpsden from the script and refresh the preview. If the frame count/duration change, you probably have a variable frame rate source and need fpsnum & fpsden, or you need to encode as variable frame rate (another story). If they don't change the source is constant frame rate and you don't need fpsnum & fpsden in the script. If you don't need them, better to remove them, in case on occasion they make the output "jittery".

If you're running Avisynth in single threaded mode (standard Avisynth) I wouldn't bother with SetMemoryMax(). I never have. I get a very occasional crash, maybe due to a bug in a new plugin.... or the fact I haven't rebooted the computer in two weeks and I've been opening and closing lots of software etc.... that sort of thing.... but other than that I use fairly slow filtering such as QTGMC and LSFMod quite often, and it's rock solid without SetMemoryMax. MeGUI is a little more crash prone, but that's MeGUI.

[midts]

I get exactly what is described in your links.

Also I have Avisynth 2.6 installed, but with MPC-HC I get cannot render and VirtualDub gives 'Unable to open file' for both .avs scripts.

[sneaker]

If you want to open AviSynth 32 bit scripts you need to use MPC-HC 32 bit / VirtualDub 32 bit. Some VirtualDub input plugins might also interfere. Use "File"->"Open video file" and choose the AVIfile input driver in the drop-down menu.

[midts]

I tried with both 32/64 versions of VirtualDub 1.10.4 and I get

avisynth open failure: loadplugin: unable to load ffms2.dll (or lsmashsource.dll) error=0xc1

with x32 and same errors with x64 build.

[LigH.de]

As you may not have explicit "LoadPlugin" calls in your script, your system-wide AviSynth installation cannot find these DLL's (in contrast to the local version used inside MeGUI, which has its own plugin management). You will have to copy this DLL into the plugin directory which belongs to your system-wide AviSynth installation. Furthermore, error 0xc1 may mean that either you forgot to install matching MS Visual C++ runtime DLLs, or you tried to load a 64 bit plugin in 32-bit AviSynth (or vice versa).

How comes that VirtualDub for AMD64 works in general? Did you ever install AviSynth+ before?

[midts]

LoadPlugin is in the .avs for all dlls as shown in previous scripts pastes, I copied the plugins to C:\Program Files (x86)\AviSynth 2.5\plugins dir, does not work.

[LigH.de]

So check the rest of my suggestions... You cannot load the exactly same DLL for an AviSynth script running in both a 32 bit and a 64 bit environment, so a code type mismatch will be the reason in one case; in the other case, the reason might be a missing MSVC runtime.

[hello_hello]

I tried with both 32/64 versions of VirtualDub 1.10.4 and I get

avisynth open failure: loadplugin: unable to load ffms2.dll (or lsmashsource.dll) error=0xc1

with x32 and same errors with x64 build.

How odd. Which flavour of Windows are you using?

You're not alone, by the way. If it was only happening to you, it might be simply because computers are vicious bastards, but you're the second person with the same problem within a couple of days. A different error message in their case, but it appears to be the same problem. Maybe it's some sort of bug, but what's causing it, I have no idea.

https://forum.doom9.org/showthread.php?p=1805980#post1805980

[midts]

I'm on latest Win10 x64.

[hello_hello]

I quoted your post and added it to the doom9 thread. Keep an eye on the thread for any developments.
https://forum.doom9.org/showthread.php?p=1806075#post1806075

[LigH.de]

It might help to check your system with AVSMeter (or AVSMeter64, respectively):

a) AVSMeter64.exe -avsinfo -l — to check general health of your system-wide installation (produces a log file to attach or upload to a pastebin site).
b) AVSMeter64.exe yourscript.avs — to check specifically the execution of your script

[midts]

AVSMeter (x64) cannot load a 32 Bit avisynth.dll.

This is strange. I even did clean uninstall of avisynth 2.6 and install. The installer is 32bit, but it puts avisynth.dll inside syswow64 and not system32, and AVSMeter reports same error.

[LigH.de]

Of course not. AVSMeter64 can only work with a 64-bit avisynth.dll in system32. Practically, it works with AviSynth+ only (as there is no serious 64-bit AviSynth 2.5+ which is not "plus").

But AVSMeter (the 32-bit application) should be able to use a 32-bit avisynth.dll v2.6 in SysWoW64.

[midts]

You're right, I ran the .avs with the 32bit version and I get the same exact error like with VirtualDub.

[LigH.de]

So are you trying to load a 64-bit FFMS2.dll into a 32-bit AviSynth?

[hello_hello]

AVSMeter (x64) cannot load a 32 Bit [url=https://www.videohelp.com/software/Avisynth]avisynth[/url].dll.

This is strange. I even did clean uninstall of avisynth 2.6 and install. The installer is 32bit, but it puts avisynth.dll inside syswow64 and not system32, and AVSMeter reports same error.

As mental as it seems, 32 bit dlls go in the syswow64 folder and 64 bit dlls go in the system32 folder. I think it's a backwards compatibility thing.
http://www.samlogic.net/articles/32-64-bit-windows-folder-x86-syswow64.htm

[midts]

So are you trying to load a 64-bit FFMS2.dll into a 32-bit AviSynth?

I replaced them with 32 versions. and now it works, but what I'm supposed to look at?

$ AVSMeter.exe test-ffms.avs

AVSMeter 2.5.4 (x86) - Copyright (c) 2012-2017, Groucho2004
AviSynth 2.60, build:Mar 31 2015 [16:38:54] (2.6.0.6)

Number of frames:                 7040
Length (hh:mm:ss.ms):     00:04:53.627
Frame width:                      1280
Frame height:                      544
Framerate:                      23.976 (24000/1001)
Colorspace:                       i420

Frame (current | last):         5549 | 7039
FPS (cur | min | max | avg):    181.7 | 34.90 | 815.1 | 204.1
Memory usage (phys | virt):     237 | 239 MiB
Thread count:                   8
CPU usage (current | average):  56% | 68%

Time (elapsed | estimated):     00:00:27.191 | 00:00:34.490

Also then why x64 virtualdub does not open the .avs with x64 plugins?

[jagabo]

Also then why x64 virtualdub does not open the .avs with x64 plugins?

Everything must be 32 bit, or everything must be 64 bit. You can't mix 32 bit and 64 bit programs/plugins.

32 bit VirtualDub + 32 bit VirtualDub plugins + 32 bit AviSynth + 32 bit AviSynth plugins.

64 bit VirtualDub + 64 bit VirtualDub plugins + 64 bit AviSynth + 64 bit AviSynth plugins.