Encoding to AV1 - NotEnoughAV1Encodes

[Alkl]

Dear Community,

I am currently developing the Windows tool NotEnoughAV1Encodes, which helps people to have a more smooth start in AV1 Encoding.

I was baffled to see Comments like this, where people literally don't even try at all and just post a negative review. Don't get me wrong, I need critical reviews and tests, but in this case it's devastating to see people not get the most simple problem fixed, nor do they reach out at Github to get things sorted. Ok, enough whining now.

Whats AV1 you may ask (or not) Source:

AOMedia Video 1 (AV1) is an open, royalty-free video coding format designed for video transmissions over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia), a consortium founded in 2015 that includes semiconductor firms, video on demand providers, video content producers, software development companies and web browser vendors.

There are currently three open-source AV1 Encoders: aom,rav1e and SVT-AV1.

For decoding dav1d is mainly used, as it's current state is much faster than the reference decoders. VLC and MPV are by default using dav1d as decoder.
If you have issues playing AV1 content on Desktop with VLC, please try using the latest VLC Nightly, as these often include the latest dav1d commits.

The current state of AV1 is quite interesting, it is getting pushed by YouTube and Netflix, but the encoding process as is is still pretty slow, as the reference encoder (libaom/aomenc) is still not optimised for multi-threading.

Thats where tools like NotEnoughAV1Encodes comes in, which does make encoding easier and faster for AV1 encoders (and recently also libvpx-vp9).
It splits the Source Video into multiple chunks and encodes them parallel with the same given settings. At the end it will Concatenate the chunks into a single video. (It's basically a ffmpeg GUI which uses ffmpeg to pipe to the encoders)

Here is a screenshot of the latest NotEnoughAV1Encodes 1.0 Release (Background picture not included in the application):


You can also download the latest Builds, which are created after each commit on AppVeyor.

The installation process should be straight forward. (It does not matter if it is the Protable Version/Installer Version) After startup the program will check at three different locations for it's dependencies: 1. In the Subfolders /Apps/ffmpeg/ & /Apps/Encoder/ 2. in the root directory next to the exe 3. in the Windows PATH Environment.

Because I know people might get confused with it, I also added an integrated Updater with the 1.0 Release which should have accounted for people like dmagic1. This feature requires 7zip tho.

That being said, I would like to get some thoughts and feedback. You can also always DM me on Discord (or talk on the AV1 Discord Server) and I will gladly help. Addionaly you are also welcome to use the Issue Page on Github .

Many Greetings!

[Alkl]

Why?

[Attachment 54499 - Click to enlarge]


[Attachment 54500 - Click to enlarge]


[Attachment 54501 - Click to enlarge]

[Alkl]

@sophisticles

Do you have considered that NEAV1E, qencoder and AV1An is literally developed on the same Discord Server?


[Attachment 54502 - Click to enlarge]

Everyone here on this Discord is aware of each other, we talk to each other, we give constructive criticism to each other.
The qencoder and av1an DEV are also not happy about your childish behavior. I will admit that I might have reacted a bit childish by contacting you in the first place, but let me tell why:
All your "review" said was: Mimimi, bad programmer, mimimi, bad software, mimimi never tested but it's still bad.

[Alkl]

I am just leaving this here:

[Attachment 54503 - Click to enlarge]

[Ant_G]

Don't pay attention to that kind of comments, just do what you want to do, coding the way you want, as long you not breaking any rules and achieved your goals.

There are people who think they are master of everything and anything else that not follow their ways is considered as flawed.

[LigH.de]

If the choice of a specific programming language or compiler already makes software "bad", then I should be proud to have used a collection of the worst software out there. I still use a tool that needs a Visual Basic 5 Runtime. And it is still useful for me. Head up, keep going, fun and success!

[sophisticles]

Me thinks a certain someone is a bit butthurt.

As I've said in the past, there are programmers and then there are "programmers" and people that use stuff like C#, or VB (really, VB?_are "programmers".

Seriously though, I support your decision to reinvent the wheel and "code" an "application" that literally does the exact same thing that can be done with a simply shell script and ffmpeg.

Keep up the "good" "work".

[LigH.de]

(Bias and discrimination.)

Keep up the "good" "work".

Is that supposed to be sarcasm?

[sophisticles]

(Bias and discrimination.)

Keep up the "good" "work".

Is that supposed to be sarcasm?

Don't be dense, how is it "bias and discrimination"? I don't support using a programming language or library or API that ties an application to a specific OS, with very minor exceptions. In general, I believe in portability, because you never know when you may need to port your software to a different platform and in fact this is a fight I've been having at my job for years. There's an application that I use on a daily basis that was coded in VB decades ago and it has locked us into specific versions of Windows, and after the developer retired, we had to hire a full time person just to debug the kludge the original developer had "coded" in VB. My company finally listed to me after I don't know how many meetings and complaints and they have begone the process of coding a replacement in a portable language.

As for this silly software, apparently this guy doesn't seem to see the absurdity of his software. You can literally do what he's doing with a shell script and ffmpeg, while being cross platform and simpler to implement.

Furthermore, the problem with libaom, and all AV1 encoders, isn't the lack of multithreading or lack of optimizations, though that does hamper libaom, it's the fact that the algorithms are just so computationally intensive.

His approach will only work properly with closed GOP content, if he attempts to cut open GOP content to encode the segments in parallel, he will likely cause corrupted output.

On top of that, even if everything worked as planned, the expected speedup would be insignificant in the grand scheme of things. x265 is at least an order of magnitude slower than x264 and AV1 is at least an order of magnitude slower than x265. x265 is very well optimized, with excellent threading, including frame level and gop level threading, as well an extensive SIMD optimizations, including AVX-512 and it's still painfully slow with even the faster presets.

The fastest AV1 encoder, SVT-AV1, which has AVX-512 and great multi-threading support, is an order of magnitude slower than x265. Libaom, which I have tested, even in the fastest settings, encodes on my 6C/12T Ryzen 5 1600 at something like 1/10 of a frame/sec, meaning even if his software worked perfectly and resulted in a 12x speedup, it still would barely break the 1fps encoding barrier.

The answer to fast AV1 encoding isn't "software" like his, it's hardware encoders.

But if he feels like pretending to be a developer, more power to him.

[Alkl]

As for this silly software, apparently this guy doesn't seem to see the absurdity of his software. You can literally do what he's doing with a shell script and ffmpeg, while being cross platform and simpler to implement.

I never claimed otherwise. The same goes to other GUI's like vidcoder, handbrake, staxrip, qencoder, etc. Nearly everything can be done with a shell script. Is a shell script good for super users? Certainly yes. Is it good for the guy who never even touched such scripts? (aka 90% of the people who uses windows (this number is just an exaggeration)) Certainly no.

You don't notice the "absurdity" of your statements/opinions. You don't understand the key-point why I programmed this application.

The Point is, that WE (AV1 Discord Community/Reddit) wanted to bring AV1 to the users. My goal is not to deliver a software for the power user you state you are.

I purposely did NOT want to develop multi-platform, as the AV1AN developer was literally already doing so in the Discord channel above me. The qEncoder dev started his project, because he wanted to bring AV1AN to people who did NOT like CLI. His project is based on AV1AN.

Here a what the qEncoder dev wrote to you stating that I was copying qencoder:

Anyway today I learned neav1e is an imitation of a program that was literally made months later.
And which wouldn’t exist probably if not for neav1e

The fastest AV1 encoder, SVT-AV1, which has AVX-512 and great multi-threading support, is an order of magnitude slower than x265. Libaom, which I have tested, even in the fastest settings, encodes on my 6C/12T Ryzen 5 1600 at something like 1/10 of a frame/sec, meaning even if his software worked perfectly and resulted in a 12x speedup, it still would barely break the 1fps encoding barrier.

Let's be real. AVX-512 is not optimal for real world usage. It is barely usable on longterm because the CPU literally has to downclock itself to a point, that there is no real advantage of AVX-512 for the time being.
Libaom delivers decent speeds, when you run it in parallel. With CPU-Used 3 10bit 2-Pass Q30 I can get roughly 3.5 frames per second. Only running one instance results in 0.78 frames per second. In the fastest possible setting of libaom you can get more than 10 frames per second.

Here are some of my older Tests:
- https://www.reddit.com/r/AV1/comments/f1h2u1/aom_vs_rav1e_vs_svtav1_vs_x265_vs_vp9_vs_svtvp9/
- https://www.reddit.com/r/AV1/comments/e65dhl/libaom_vs_rav1e_vs_svtav1_test_results_with/

I can certainly say that you have less experience with AV1 Encoding than anyone on the unofficial AV1 Discord Server.

The answer to fast AV1 encoding isn't "software" like his, it's hardware encoders.

To be honest, thats one of the most dumbest statements I so far heard. Hardware encoders are bad on so many levels. I exhaustively tested the nvenc crap, which gives worse effeciency while delivering bad visual quality. We also don't expect that AV1 will have Hardware Encoder support anytime soon.

But if he feels like pretending to be a developer, more power to him.

You still not have understood it. Technically I am a developer, practically I am just a student who has nothing to do, because of Corona.

My company finally listed to me after I don't know how many meetings and complaints and they have begone the process of coding a replacement in a portable language.

I am surprised that you, a guy who is programming for a company on a professional level, is shitting literally on a guy on the internet about him writing a program in c# in his freetime, who never touched code prior to 7 month ago, without getting payed. I have no, absolutely no reason to follow your opinions in how I have to write a open source software.

[_Al_]

Alkl - discussion with academic sophisticles is waste of time, ..., have fun coding, forget this

[megapro17]

As I've said in the past, there are programmers and then there are "programmers" and people that use stuff like C#, or VB (really, VB?_are "programmers".

Can you show your repositories, or name any projects where you made a significant impact?

[sophisticles]

Just to be clear, I did not say I currently work as a software developer, though I do have a background in various areas of computer science and IT. I currently work as a data analyst for a very large medical lab, and I analyse large data sets, in order to gauge efficiency of SOPs as well as make predictions about future trends and most of this is done through the prism of how it impacts the bottom line.

As for Alkl's claims, I also never said that he copied Qencoder, I said that Qencoder does it better because it's cross platform and doesn't rely on MS run-times or a proprietary language.

And yes, I have the same problem with Staxrip, even though that is a vastly superior piece of software, and I can almost forgive it because it is vastly more functional that this waste of 1's and 0's.

As for "hardware encoders are bad on so many levels", this guy clearly has no idea what he is talking about, none.

https://forum.videohelp.com/threads/398346-H266

Just some tests I have done, feel free to use the search function to see all the tests I have contributed over the years.

[Alkl]

As for Alkl's claims, I also never said that he copied Qencoder, I said that Qencoder does it better because it's cross platform and doesn't rely on MS run-times or a proprietary language.

[Attachment 54510 - Click to enlarge]

Question: Are you talking about my programming skills or are you talking about my knowledge about AV1? If it's first: I never denied to be a bad programmer. I don't see the problem tho, if it works. If you mean that I have no idea about AV1, then I will kindly ask you to join the AV1 Discord from the Subreddit. They will tell you otherwise.

As for "hardware encoders are bad on so many levels", this guy clearly has no idea what he is talking about, none.

The differences in these encoders will show most prominently across multiple hardware generations where some features are simply not present. For instance, since Skylake, QSV has supported HEVC encoding with B-frame support, whereas it took NVENC up to Turing to implement the same. Making comparisons across hardware-based encoders must take into account the multiple variances introduced across hardware generations and the APIs tapping into the accelerators in question.


I would personally argue that there is no way for a hardware encoder will reach software encoder effecinecy levels. Most Hardware encoder don't have all the functions the reference software encoders have. The speed of a hardware encoder will always result in some way of effeciency reduction, especially on low bitrates. The nvenc encoder has only a very small portion of functions which the x265 software encoder has. You could literally see with the eyes, that it completely made a mess of sharpness and details. It introduced blocking and it looked like it applied a water color filter on top of the encode.

Hardware encoders are only intersting for people who livestream or to people who want to quickly give a friend an encode. From a professional standpoint, nobody would use a hardware encoder to compress their video library, especially not companies like netflix (who developed vmaf).

Just some tests I have done, feel free to use the search function to see all the tests I have contributed over the years.

I did some research about you and I was compltetly stunned by you. (in a bad way) You are quite unpopular by your agressive manners towards other people.
I also can't find any tests you claim to have done. Does these "tests" you claim to have done, include objective comparisons (vmaf) / subjective comparisons (pixel peeping until you go crazy) while taking different video types into account? PSNR and SSIM are by the way bad metrics to start with if you test, as they are not perceptual. I had lots of encodes with good psnr/ssim results, but the results still looked like crap. I saw a post where you said that you haven't done vmaf. Do it.

[sophisticles]

I did some research about you and I was compltetly stunned by you. (in a bad way) You are quite unpopular by your agressive manners towards other people.

Oh, no!!! I am unpopular?!? I'll call my therapist right now and see if he can help me come to grips with this realization.

I also can't find any tests you claim to have done.

You must have gone to the O.J. School of Searching ( I wonder if you are old or savvy enough to get this reference):

https://forum.videohelp.com/threads/397870-The-most-comprehensive-AV1-encoding-test

https://forum.videohelp.com/threads/397888-SVT-HEVC-vs-X265

https://forum.videohelp.com/threads/397344-Most-comprehensive-x264-test-you-will-ever-see%21%21%21

https://forum.videohelp.com/threads/397165-X264-vs-Intel-QSV-on-Ubuntu-Linux

https://forum.videohelp.com/threads/396998-x264-slow-or-x265-veryslow-psnr

https://forum.videohelp.com/threads/394569-I-there-any-benefit-to-encoding-to-10-bits

https://forum.videohelp.com/threads/393286-Can-NVENC-beat-x264-slow-preset

https://forum.videohelp.com/threads/392745-Kaby-Lake-Quick-Sync-quality-test

https://forum.videohelp.com/threads/392943-QSV-beats-x264-and-here-s-the-proof%21%21%21

https://forum.videohelp.com/threads/392544-Intel-SVT-HEVC-encoder-test

https://forum.videohelp.com/threads/384071-Pascal-NVENC-vs-X264-x265-test

https://forum.videohelp.com/threads/388642-Kaby-Lake-VP9-hardware-encoder-tested%21%21%21

If only I had been working with video since before you were born,

[natis1]

Originally I wasn't going to feed the trolls, as it were, but I realized that it just wasn't right to let you go on like this. It brings me no pleasure to see you use my software, qencoder, as a cudgel against other developers, ones whom I deeply respect and look up to. When someone writes free software that doesn't do exactly what you want, or has some other problem that makes it not work for you (eg it doesn't work on your platform), you have basically no ground to stand on, or complain. If you hate someone's code and program that they're giving out for free so much make your own program. Don't be a Karen.

[Baldrick]

I will delete all posts that doesn't come with some useful information. If it sucks then give the programmer some tips instead.

I'm a real programmer that still use PHP! And I love spaghetti code....

[sophisticles]

I will delete all posts that doesn't come with some useful information. If it sucks then give the programmer some tips instead.

I'm a real programmer that still use PHP! And I love spaghetti code....

I did give both of them useful information, they just chose to act like children and complain instead.

1) Don't use a proprietary language that depends on proprietary run-time environments, C# being the one that is currently in favor by many so-called developer's. For the record I have the same problem with Visual Basic and Swift.

2) Don't distribute a program that has dozens of dependencies.

NotEnoughAV1Encodes was made needlessly complicated by coding it in C# and requiring an end-user to install both a specific .NET version and the MS Visual C++ Redistributable.

Qencoder on the other hand is coded to be cross platform, but that's in name only. There are literally 12 dependencies required in order to get it to work on Linux and even if you install all the dependencies it still fails to work on many distributions. Here's a pro-tip from a long time Linux user, distribute it, along with all the dependencies, as an appimage, see AviDemux and ShotCut for 2 applications done right.

As for PHP, I have no problem with it, in fact it's absolutely crucial for running the internet as part of a LAMP stack.

[Baldrick]

I will keep delete off topic posts.

[sophisticles]

Fine, it's your forum, you are free to do as you please. How's this for on topic:

According to this:

https://www.videohelp.com/software/NotEnoughAV1Encodes

This software requires .NET Framework 4.7.2 to work; according to Microsoft, there are 12 different variants of .NET Framework:

https://dotnet.microsoft.com/download/dotnet-framework/net472

According to Microsoft only 1 version of .NET 4 may be installed at any one time:

https://docs.microsoft.com/en-us/dotnet/framework/get-started/system-requirements

Staxip, a program also coded in C#, requires NET 4.6.1:

https://www.videohelp.com/software/StaxRip

So, since you can only have 1 version of .NET 4 installed on a Windows system at any given time and since both of these programs depend on a different version of .NET, then it's impossible, according to what Microsoft is saying, to have both this software and Starip installed on a Windows PC at the same time.

Media Coder on the other hand, a vastly more capable piece of software than either of these 2, does not have any such dependencies and will not prevent you from installing or using any other software.

Both of these guys, the author of this, and the author of Staxip, should consider rewriting their software in order to prevent incompatibilities as well as to not result in people losing functionality and freedom of their OS.

[Alkl]

So, since you can only have 1 version of .NET 4 installed on a Windows system at any given time and since both of these programs depend on a different version of .NET, then it's impossible, according to what Microsoft is saying, to have both this software and Starip installed on a Windows PC at the same time.

Wrong.

Microsoft: "If you install a later version, you don't have to first uninstall the previous version."

You don't have the basic understanding how NET works. If you have .NET 4.7.2, it will also work with applications written in .NET 4.6.1

The .NET Framework 4 is highly compatible with applications that are built with earlier .NET Framework versions, except for some changes that were made to improve security, standards compliance, correctness, reliability, and performance.

The .NET Framework 4.5 and later versions are backward-compatible with apps that were built with earlier versions of the .NET Framework. In other words, apps and components built with previous versions will work without modification on the .NET Framework 4.5 and later versions.

I am running NotEnoughAV1Encodes and StaxRip with the latest .NET Framework and without any problems.

[sophisticles]

I am not going to edit my above post, because I am man enough to admit when i am wrong. It seems that Microsoft open sourced many parts of .NET, so that you can use it with Linux and OSX as well as Windows:

https://dotnet.microsoft.com/download

https://docs.microsoft.com/en-us/dotnet/core/install/linux

NotEnoughAV1Encodes still needs the VC++ runtime, but in theory, if one were to recompile Staxrip and this software with GCC, one could get it to run on Linux.

I intend to test out my theory this weekend.

[Alkl]

sophisticles kudos for admitting it.

I can sadly tell you that it won't be possible to run neav1e on Linux, even tho most of .NET is open source.

The GUI is programmed with the WPF (Windows Presentation Foundation) UI Framework, which is not and won't be supported on Linux, as Microsoft does not have any plans in doing so. The Mono project also has no intensions in implementing the API.

There are other open source cross platform alternatives like AvaloniaUI (https://avaloniaui.net/ - uses .NET Core instead of .NET Framework) which is an imitation of WPF.

I have to admit that I was too lazy to dig further in this alternative, as I wanted to bring neav1e as fast as possible to a running state.

You also probably won't be able to get StaxRip Running at all, because alot of the inbuild functions are entirely designed for Windows.

When I have the nerves I can look further into AvaloniaUI, but this should not be seen as promise.

[jamespoo]

i love this app but have a problem when i deinterlace mpeg2 vob why does it make the fps 29.976fps to 59fps or 25fps is now 49fps?

what am i doing wrong also is there a way to ivtc ntsc film to 23.976fps?

also could you add unless its already in the app and i missed it anamorphic automatic setting like in handbrake

where i crop lets say a 720x576 4:3 aspect video where it displays at 768x576 when playing the original video in mpc but i crop the black bars so now it is 704x496 but it will save the video as 704x496 but still desplaying it at 752x496

hope you can understand what i mean thanks

[LigH.de]

Deinterlacers may interpolate woven fields to independent full frames, doubling the nominal frame rate (29.97 becomes 59.94 fps, 25.0 becomes 50.0 fps). But it works only if the material has a linear temporal progression from one field to the next to the next... You may discard half of them to return to the original frame rate. The side effect will be a "short shutter" effect, a lack of motion smoothness.

Instead, if the material has 29.970 fps because it has been telecined, use Inverse Telecine (IVTC) to revert it to 23.976 fps. But only if. Make sure it is telecined, not TV-interlaced, before you attempt to do so.

If you are not sure about the difference between interlacing (caused by cathode ray tube TV cameras) and Telecine (processing film to match the frame rate for NTSC TV sets), learn about them, how they work, why they were developed, how you can tell one from the other.

Unfortunately, there are also many perverted kinds of norm conversions which are much harder to fix. Or not fixable at all.

[jamespoo]

Deinterlacers may interpolate woven fields to independent full frames, doubling the nominal frame rate (29.97 becomes 59.94 fps, 25.0 becomes 50.0 fps). But it works only if the material has a linear temporal progression from one field to the next to the next... You may discard half of them to return to the original frame rate. The side effect will be a "short shutter" effect, a lack of motion smoothness.

Instead, if the material has 29.970 fps because it has been telecined, use Inverse Telecine (IVTC) to revert it to 23.976 fps. But only if. Make sure it is telecined, not TV-interlaced, before you attempt to do so.

If you are not sure about the difference between interlacing (caused by cathode ray tube TV cameras) and Telecine (processing film to match the frame rate for NTSC TV sets), learn about them, how they work, why they were developed, how you can tell one from the other.

Unfortunately, there are also many perverted kinds of norm conversions which are much harder to fix. Or not fixable at all.

the videos im encoding are not from video camera they are from film so they are just adding duplicate frames

[LigH.de]

If that's correct, they duplicate fields, not frames, in a 3:2 pattern. And then you use IVTC, not a deinterlacer. And Telecine with 3:2 Pulldown never happens for PAL, only for NTSC.