The most comprehensive AV1 encoding test

[sophisticles]

Is not what you find will here. What I am about to post is a test of Intel's/NetFlix's SVT-AV1 encoder, compared against x264+placebo and x265+placebo.

For those of you that don't know, Intel and NetFlix have been working together on a highly optimized software based AV1 encoder for a while now:

https://netflixtechblog.com/netflix-now-streaming-av1-on-android-d5264a515202

https://netflixtechblog.com/svt-av1-an-open-source-av1-encoder-and-decoder-ad295d9b5ca2

There a few claims to remember about AV1, first is that SVT-AV1 can match the reference AV1 encoder, libaom, in theory the best AV1 encoder possible, second that libaom is 20% more efficient than libvp9 and third that it is extremely fast for a AV1 encoder.

To test it I chose my R5 1600 system with 16GB ddr4 2666, Ubuntu 18.04.4 LTS with all the updates, with 2 different ffmpeg versions, x264 and x265 were encoded with the latest ffmpeg+libx264+libx265 available for Ubuntu and ffmpeg+svt-av1 was compiled from the latest dev and patched to allow ffmpeg to use SVT-AV1,

Source and destination drives were 2 separate SSDs.

The video source was put together by going to Xiph Derf and downloading all the files that listed as 4:2:2 1080p, these in turn were loaded into Shotcut and exported as x264 lossless (CRF 0) using the ultrafast preset for a resultant 5.1Gb 2min32s file.

The command lines used are as follows:

time ffmpeg -i "1080p.mp4" -c:v libx264 -preset placebo -tune film -an "/media/XX/960 GB SSD/1080p x264.mp4" 43m21.807s

time ffmpeg -i "1080p.mp4" -c:v libx265 -preset placebo -an "/media/XX/960 GB SSD/1080p x265.mp4" 152m12.083s

time ./ffmpeg -i "1080p.mp4" -c:v libsvt_av1 -an "/media/XX/960 GB SSD/1080p SVT-AV1.mp4" 17m7.297s

time ./ffmpeg -i "1080p.mp4" -c:v libsvt_av1 -preset 7 -an "/media/XX/960 GB SSD/1080p SVT-AV1 2.mp4" 26m14.989s

time ./ffmpeg -i "1080p.mp4" -c:v libsvt_av1 -an -rc 2 -b:v 3677K "/media/XX/960 GB SSD/1080p SVT-AV1 3.mp4" CVBR 17m7.007s

time ./ffmpeg -i "1080p.mp4" -c:v libsvt_av1 -an -rc 2 -b:v 9662K "/media/XX/960 GB SSD/1080p SVT-AV1 4.mp4" CVBR 17m57.796s

For the x264 and x265 encodes aside from choosing the preset and in the case of x264, the tune, I left everything at the defaults, letting the encoder decide how it wanted to create the final product. The interesting thing is that I have done the x264 test with this source numerous times and no matter what preset I use, it always uses right around 10 mb/s, x265 wants to use around 5 mb/s, settling to under the 4 mb/s mark with the placebo setting.

This falls in line with the promises made about HEVC in general, that it offered 50% bit rate savings at the same quality compared to AVC.

The encode times were shocking, to say the least. I knew the placebo settings were slow, but it still caught me by surprise that x264 would take over 17 times the length of the clip to finish and that x265 would take over 60 times the length of the clip to finish.

For the SVT-AV1 encode I did a couple of encodes, I did one with the default settings (it defaults to the fastest preset), I did one at the next fastest preset, I did one at the fastest preset using constrained vbr (meaning it attempts to match the average chosen bit rate within any given gop, a opposed to across the entire file) targeting the bit rate x264 used and once with constrained vbr targeting the bit rate x265 used.

For playback and evaluation I used the latest dev snap of VLC.

[Selur]

Using av1 without using any of the option available for grain/noise handling, seems like ignoring one of the main points for av1.
comprehensive and nothing about aomenc and rav1e?

[Sharc]

Sidenote: The scene with the House on Fire is actually interlaced with 59.94 fields per second. All encodes seem to have preserved the fields structure for this scene, but the clip is flagged as progressive. I don't know how the playback device will deal with this.

[sophisticles]

Using av1 without using any of the option available for grain/noise handling, seems like ignoring one of the main points for av1.
comprehensive and nothing about aomenc and rav1e?

The ffmpeg does not expose all that many options through ffmpeg:


libsvt_av1 AVOptions:
-hielevel <int> E..V..... Hierarchical prediction levels setting (from 3 to 4) (default 4level)
3level E..V.....
4level E..V.....
-la_depth <int> E..V..... Look ahead distance [0, 120] (from -1 to 120) (default -1)
-preset <int> E..V..... Encoding preset [0, 8] (from 0 to 8) (default 8)
-profile <int> E..V..... Set profile restrictions (from 0 to 2) (default main)
main E..V.....
high E..V.....
professional E..V.....
-tier <int> E..V..... Set tier (general_tier_flag) (from 0 to 1) (default main)
main E..V.....
high E..V.....
-level <int> E..V..... Set level (level_idc) (from 0 to 31) (default 0)
2.0 E..V.....
2.1 E..V.....
2.2 E..V.....
2.3 E..V.....
3.0 E..V.....
3.1 E..V.....
3.2 E..V.....
3.3 E..V.....
4.0 E..V.....
4.1 E..V.....
4.2 E..V.....
4.3 E..V.....
5.0 E..V.....
5.1 E..V.....
5.2 E..V.....
5.3 E..V.....
6.0 E..V.....
6.1 E..V.....
6.2 E..V.....
6.3 E..V.....
7.0 E..V.....
7.1 E..V.....
7.2 E..V.....
7.3 E..V.....
-rc <int> E..V..... Bit rate control mode (from 0 to 3) (default cqp)
cqp E..V..... Const QP
vbr E..V..... Variable bit rate, achieve the target bitrate at entire stream
cvbr E..V..... Constrained variable bit rate, achieve the target bitrate at each gop
-qp <int> E..V..... QP value for intra frames (from 0 to 63) (default 50)
-sc_detection <boolean> E..V..... Scene change detection (default false)
-forced-idr <boolean> E..V..... If forcing keyframes, force them as IDR frames. (default true)

As you can see there are very few options, I have tested libaom and it's so slow as to be unusable, and according to NetFlix SVT-AV1 is very close to libaom; I would love to test rav1e but I have never been able to build ffmpeg with rav1e, it fails every time, can't figure out what I am doing wrong.

Regardless, SVT-AV1 is most likely the future if NetFlix has chosen it as their encoder of choice for future use. Not to mention I really believe the SVT family of encoders are really a test bed for Intel's upcoming hardware AV1 encoder that will be introduced with the Gen12 Xe graphics chip, that will be found in Tiger Lake, Rocket Lake and their DG1 discreet gpu.

[Selur]

okay,... so the title is simply click bait,..

[KarMa]

There a few claims to remember about AV1, first is that SVT-AV1 can match the reference AV1 encoder, libaom, in theory the best AV1 encoder possible

Maybe at the fastest Libaom setting against the slowest SVT-AV1. Maybe.

[sophisticles]

There a few claims to remember about AV1, first is that SVT-AV1 can match the reference AV1 encoder, libaom, in theory the best AV1 encoder possible

Maybe at the fastest Libaom setting against the slowest SVT-AV1. Maybe.

Did you bother to read through NetFlix's blog where they post their test data? Have you tried both SVT-AV1 and libaom? If not, then as usual, you are talking out of your ass.

[sophisticles]

okay,... so the title is simply click bait,..

Actually it was a joke, as evidenced by the very first line of my post where I said:

"Is not what you find will here."

[megapro17]

WHY are you using x265 placebo?? To show that x265 is allegedly slower than av1? Time matters, there is no point to use slower preset than "slow" in x265
obviously your x265 encodes looks better than av1 especially at the end

[sophisticles]

WHY are you using x265 placebo?? To show that x265 is allegedly slower than av1? Time matters, there is no point to use slower preset than "slow" in x265
obviously your x265 encodes looks better than av1 especially at the end

I chose the placebo presets because I wanted the best that x264 and x265, in theory, are capable of. SVT-AV1 at the slowest setting would be slower than either.

[KarMa]

Did you bother to read through NetFlix's blog where they post their test data? Have you tried both SVT-AV1 and libaom? If not, then as usual, you are talking out of your ass.

Show me another thread where I've been talking out of my ass?

My constant threads of how much better Intel QS H.264 is than x264 (with the eventual caveat that the bitrate is 50Mbit, or the caveat that the quality is good enough and speed is all that matters, or the eventual goalpost moving to Intel H.265 QS vs x264)? The threads where I change TV range video to PC range as a great way to give Nvidia's encoder an advantage on TV range decoders? Where I justify VMAF results that show x264 Fast beating out x264 Slow. Where I constantly demand that until obscure commercial H.264 encoders are tested (not by me of course), then I will nitpick any suggestion of noobs using x264 because I'm so bent out of shape over x264. Where I create the false narrative that Youtube's H.264 quality sucks because they use x264, when in actually they reserve x264 for their most popular of content.

No wait, that's you.

As for my experience with AV1. I can remember testing out the early versions of AOMENC in early 2017, with mixed success. I can remember making my first 1 hour long 1080p encoding via Libaom in early 2019, took me weeks. Now I have dozens of 1080p AOMENC videos encoded by me, and I've only stopped as I have run out of stuff to encode. As for SVT-AV1 I've messed with it, it has improved, and it can be considered faster when you don't want to run AOMENC encodings in parallel and if you only want to run SVT at the faster presets. As once you start ramping up the preset the less CPU usage you get. I still think SVT-AV1 is far from being considered just as good as AOMENC, no matter if Netflix says it.

[sophisticles]

Show me another thread where I've been talking out of my ass?

I would but the forum's search function said it couldn't return that many results in one shot.

My constant threads of how much better Intel QS H.264 is than x264 (with the eventual caveat that the bitrate is 50Mbit, or the caveat that the quality is good enough and speed is all that matters, or the eventual goalpost moving to Intel H.265 QS vs x264)? The threads where I change TV range video to PC range as a great way to give Nvidia's encoder an advantage on TV range decoders? Where I justify VMAF results that show x264 Fast beating out x264 Slow. Where I constantly demand that until obscure commercial H.264 encoders are tested (not by me of course), then I will nitpick any suggestion of noobs using x264 because I'm so bent out of shape over x264. Where I create the false narrative that Youtube's H.264 quality sucks because they use x264, when in actually they reserve x264 for their most popular of content.

Hysterical. If you actually go back and read my posts, you will note that I have repeatedly complained about Intel's H264 QSV, stating that I believe it was fundamentally broken because there are some test clips where no matter how much bit rate I throw at it the scene still ends up falling apart.

I have stated that Intel's QSV HEVC beats x264, and I have stated that Intel's QSV 10-bit is fantastic and when you take into account the power and time savings along with the quality, it's unbeatable, and I stand by that. i am fairly certain that it was either you or someone like you, i.e. an x264 worshiper, that attempted to rebut me by stating that it was unfair to compare x264, an AVC encoder against an HEVC encoder and that it was also unfair because supposedly AVC was never meant to be used with 4k.

Do you recall these statements or do we have to go grave digging old threads?

The NVIDIA encoder test you are referring to in context, was me trying to demonstrate that the encoder is less important than important in many instances than many people try to make it, primarily because people routinely use filters and high bit rates when they are preparing content for commercial consumption.

Regarding the VMAF claim, I have no idea what you are referring to, I have never used VMAF nor do I know how to use it.

I do demand that commercial encoder be tested before crowning any free encoder as the "best". This goes for x265, lib-vp9, Intel's SVT encoders, all of them, you can't claim any is "best" until you have compared it against all other available options.

I would have the same objection if you said such and such pizzeria has the best pizza, if you haven't tried every other pizza available then you can't say it's the best.

As for my experience with AV1. I can remember testing out the early versions of AOMENC in early 2017, with mixed success. I can remember making my first 1 hour long 1080p encoding via Libaom in early 2019, took me weeks. Now I have dozens of 1080p AOMENC videos encoded by me, and I've only stopped as I have run out of stuff to encode. As for SVT-AV1 I've messed with it, it has improved, and it can be considered faster when you don't want to run AOMENC encodings in parallel and if you only want to run SVT at the faster presets. As once you start ramping up the preset the less CPU usage you get. I still think SVT-AV1 is far from being considered just as good as AOMENC, no matter if Netflix says it.

I take it you are not hip to the fact that not only are NetFlix and Intel part of the Alliance for Open Media but they helped found it and helped create AV1:

https://aomedia.org/about/