x264 Ultra HIgh Compression settings

[somespirit]

Maximum GOP Size = 10 000 - even more, depends how long we watch voiceman speaking about forecast on TV
Minimum GOP Size = 1

it is obviously why this is very optimal variant and why:

Maximum GOP Size = 250
Minimum GOP Size = 25

is like noob driving his shiny new car ..... or jet engine powered with house coals

-------------------------------------------------------------
Open GOP - enabled
- this will give around 2% smaller file size of final encoding

[jagabo]

GOPs larger than 250 will make very little difference in the file size. And don't try seeking with 10,000 frame GOPs.

[somespirit]

with 10 000 GOP i have 1050 I frames (100 minutes movie) for example and it is very enough for seeking ....

GOPs larger than 250 will make very little difference in the file size

but the quality of picture will be beoynd Blu-ray, because we will use only P frames instead large I frames in every GOP sequence

[jagabo]

but the quality of picture will be beoynd Blu-ray, because we will use only P frames instead large I frames in every GOP sequence

GOPs larger than 250 will make very little difference in file size (CRF encoding) or picture quality (bitrate encoding).

with 10 000 GOP i have 1050 I frames (100 minutes movie) for example and it is very enough for seeking ....

That's because you don't really have 10,000 frame GOPs. You're average GOP size is about 137 frames.

[somespirit]

I frame, B frame B-frames set to 5

real GOP of video in this moment is 200 for example

max GOP 15
I BBBBB P BBBBB P BB then new GOP - I BBBBB P BBBBB P BB

the same situation with max GOP 30 will give us
I BBBBB P BBBBB P BBBBB P BBBBB P BBBBB

so we will have one exchange of I to P so imagine for large movie what will be the effect - ще стане ефекта на заменките

[jagabo]

We weren't talking about the difference between 15 and 30 frame GOPs. We were talking about the difference between 250 and 10,000 frame GOPs.

[somespirit]

tra la la - you are not so bad smart

we are talkling for the same things in both cases

with 10 000 GOP we will get 6 minutes still movement in one GOP and will use only P frames (with one I frame in begining of GOP) instead of I frames which are much larger

[jman98]

tra la la you are not so bad smart

That's big talk.

Ty ne mozesh pisat' khorosho na angliskom.

[somespirit]

я могу everything ........

[pandy]

я могу everything ........

prove it... pебёнок

[somespirit]

no need

[jagabo]

with 10 000 GOP we will get 6 minutes still movement in one GOP and will use only P frames (with one I frame in begining of GOP) instead of I frames which are much larger

Yes, it's trivially obvious that longer GOPs have the potential to reduce the bitrate requirement. But with your typical video that will happen so infrequently that the savings is negligible be negligible. And you are ignoring the negative aspects of allowing extremely long GOPs. They are slow to seek. Picture quality slowly degrades over the course of long GOP (I frames are encoded at higher quality settings so they restore the picture quality every now and then). Then setting the minimum GOP to 1 will cause some shots to eat up more bitrate, just to keep the quality of one frame, which you're only going to see for 1/24 second.

Are there situations where you might want to use very long GOPs? Yes. But it's not a panacea for increasing the quality of the average video.

[somespirit]

Picture quality slowly degrades over the course of long GOP (I frames are encoded at higher quality settings so they restore the picture quality every now and then).

absolutly wrong is that statement, we have very still movement in that long big GOP of 10 000 and P frames they are like B frames but only in previous direction looking, so quality will be the same ~ and thus looking behind will copensate quality of pictire, so bad that I frames dont looking anywhere and for this reason we use P frames with long GOP

[jagabo]

Like I said, there are some exceptions. But in the real world video has noise and small motions all the time. Your 100 minute movie with 1050 I frames probably only has a handful of GOPs over 250 frames.

[poisondeathray]

He's probably talking about a specific scenario, like the interview/talking heads static scenes and content . This won't work well for "normal" content for the reasons stated above

If there are no compatibility target concerns , for this type of scenario you can even use --keyint infinite --b-adapt 2 --bframes 16 . Also you can use VFR encoding for even more compression, and denoise, preprocess etc...

The reason why the default is --keyint 250, is because it's set for general purpose usage - it has nothing to do with "noob driving shiny car"

[somespirit]

you can't watch movies, 250 GOP it is only 10 seconds, so every movie on Earth have much longer still many scenes than you are thinking at first glance

[jagabo]

I took a 200 frame shot from a movie. I appended a reversed version of the same video to create a 400 frame video. Reversing the video prevents x264 from detecting a scene change at the "seam". I appended this forware/reverse video to itself several times to create a ~25000 frame video. I encoded that with 250 frame GOPs and 10000 frame GOPs at CRF 18.

For the 250 frame GOP video x264 reported
x264 [info]: frame I:107 Avg QP:14.51 size: 32971
x264 [info]: frame P:6952 Avg QP:17.82 size: 13895
x264 [info]: frame B:18669 Avg QP:18.53 size: 6644
encoded 25728 frames, 53.58 fps, 1671.16 kb/s

For the 10000 frame GOP video
x264 [info]: frame I:4 Avg QP:14.43 size: 38732
x264 [info]: frame P:6828 Avg QP:17.80 size: 13983
x264 [info]: frame B:18896 Avg QP:18.53 size: 6680
encoded 25728 frames, 56.54 fps, 1654.02 kb/s

So, in this near-best-case scenario for 10000 frame GOPs there was a bitrate savings of a whopping 1 percent. This video was pretty noisy so the P frames weren't as small as they might be otherwise. So cleaner video will likely show more bitrate savings. Random seeks (this was a standard definition video) on an i5 2500K took as long as 7 seconds. My WDTV Live only seeks to key frames so it had a ~7 minute granularity for seeks or fast forwarding.

I had a chance to run a full length movie through the same test (a fairly slow paced film full of long shots, not a Hollywood blockbuster full of 2 second shots):

max 250 frame GOP:
x264 [info]: frame I:1002 Avg QP:15.66 size: 74725
x264 [info]: frame P:51096 Avg QP:18.71 size: 25205
x264 [info]: frame B:122571 Avg QP:22.45 size: 5320
encoded 174669 frames, 75.25 fps, 2212.57 kb/s

max 10000 frame GOP:
x264 [info]: frame I:616 Avg QP:15.78 size: 79208
x264 [info]: frame P:50925 Avg QP:18.70 size: 25333
x264 [info]: frame B:123128 Avg QP:22.45 size: 5324
encoded 174669 frames, 43.43 fps, 2190.07 kb/s

Again, about 1 percent difference in bitrate. I wouldn't call this ultra high compression.

[Cornucopia]

Yeah, "the proof is in the pudding" and somespirit obviously doesn't know what he/she is talking about re: compression. There is no such thing as a free lunch, as they say.

Nice test, jagabo!

Scott

[somespirit]

max 250 frame GOP:
x264 [info]: frame I:1002 Avg QP:15.66 size:74725
x264 [info]: frame P:51096 Avg QP:18.71 size: 25205

max 10000 frame GOP:
x264 [info]: frame I:616 Avg QP:15.78 size: 79208
x264 [info]: frame P:50925 Avg QP:18.70 size: 25333

saved bitrate goes there, with bold green, we have better I frames (79208 vs. 74725) and with bold red P frames (25333 vs. 25205) - so we got improvement in quality of I frames, and nothing bad happend to P frames even a bit high size --> and B frames will be based on better I and P frames, so overall quality will go high

and question about x264 stats ........--->

I frames -
1.Avg QP:15.66 size:74725 - 2.Avg QP:15.78 size: 79208 - this second (2.) stats should have better average QP because Average size of I franes is bigger, but how we can see it shows that it is lower (15.66 <->15.78)............. the number of I frames is meaningless because we have average size of I frames


no need to set max GOP to 10 000, 1000 - 2000 is enough

and also very important is Minimum GOP Size = 1, so where we have big change in scene we will always use I frame as first (in new scene) which have better quality than P frame as first


analyse by Director X

[poisondeathray]

saved bitrate goes there, with bold green

Saved bitrate is at the end, the total filesize difference

we have better I frames (79208 vs. 74725) and with bold red P frames (25333 vs. 25205) - so we got improvement in quality of I frames, and nothing bad happend to P frames even a bit high size --> and B frames will be based on better I and P frames, so overall quality will go high

The problem is no measure of objective or subjective "quality" was taken. How can you say what was "better quality" without looking at it or measuring it ? The assumption that equal CRF means equal quality isn't necessarily correct . In fact it's NOT correct - CRF is just an approximation of "quality". So what is being tested here is adjusting --keyint at a given CRF and measuring filesize and average macroblock quantizers of frametypes . But does a lower quantizer always mean better "quality ? " There is a strong positive relationship when all other variables are equal, but it's not always the case

Not to make jagabo do more work , but one way you might but a way to do this improve this testing is to add --ssim or --psnr

It might show something like this (random example)

x264 [info]: frame I:461 Avg QP:16.16 size: 51489 PSNR Mean Y:45.96 U:50.20 V:49.59 Avg:46.85 Global:44.69
x264 [info]: frame P:11836 Avg QP:17.35 size: 28198 PSNR Mean Y:42.89 U:47.43 V:46.95 Avg:43.81 Global:42.82
x264 [info]: frame B:28700 Avg QP:16.99 size: 11522 PSNR Mean Y:41.85 U:45.69 V:45.32 Avg:42.68 Global:42.27

Yes , there are problems with PSNR and SSIM measures, but at least you can have some information on "quality" and compare. At least you have a rough measure of "quality" and can say, ok I frames are higher "quality" here etc...you get the idea...

and question about x264 stats ........--->

I frames -
1.Avg QP:15.66 size:74725 - 2.Avg QP:15.78 size: 79208 - this second (2.) stats should have better average QP because Average size of I franes is bigger, but how we can see it shows that it is lower (15.66 <->15.78)............. the number of I frames is meaningless because we have average size of I frames

Avg QP is the mean average quantizer of macroblocks in the frame type , NOT the Avg frame QP . There should be a strong positive corrleation (ie. higher avg macroblock quantizers should result in result in higher average coded frame sizes for that frame type) , but it's not always the case

and also very important is Minimum GOP Size = 1, so where we have big change in scene we will always use I frame as first (in new scene) which have better quality than P frame as first

Not so important because scenecut is enabled by default (an IDR frame will always be inserted for the GOP on a big scene change, but not necessarily placed at the beginning of the GOP . x264 is smart enough to sometimes place coded order different that display order)

[somespirit]

take a walk in some park ..........fresh air better thoughts

[deadrats]

this is so ridiculous, i'm amazed that some of the more knowledgeable veterans around here have actually taken the time to address these ludicrous claims and i'm amazed at myself for actually not being able to keep myself from chiming in.

I frames, as we all know, are inherently of higher quality than either P or B frames, P frames are higher quality than B frames, so in a nutshell B frames are the redheaded step child of a GOP. I frames act as the defacto reference frame in a GOP under most conditions, using one I frame every 10,000 frames is absurd, aside from the ridiculously long seek points that you end up with, you also end up with a final encode that only uses a handful of high quality I frames. you couple that with the fact that the OP clearly has a hard-on for bitrate starving his encodes and the result will be a movie composed primarily of piss poor P and B frames.

i'm left to wonder why? is there a great bitrate shortage of 2012 that i am not aware of? have you deposited the bitrate in the bank in the hopes that you can collect interest? are you some kind of bitrate racist that things we need to keep as much bitrate as possible out of our encodes? do you have some kind of bizarre minimalistic fetish where you get off on bitrate starving your encodes?

in my own personal tests, even GOP's of 250 are too long, there's a reason why professional blu-rays and hd-dvd's use high bitrates and small GOP's, so that maximum quality can be ensured (the ability to seek to within a second is a side bonus). i have been running hundreds of test encodes, with some of the shittiest quality dvd sources imaginable, with xvid, divx and x264 and there's a few conclusions i have come to:

1) there ain't no substitute for bitrate, none. if you want max quality let the bitrate fly.

2) 2 pass encodes are the way to go, i don't care what the x264 developers say about crf using the same algorithm as 2 pass for rate control, the final output is not the same.

3) skip the quarter pixel detection, it kills visual quality.

4) crank up all the quality settings, sans the quarter pixel detection, to the max, even if they seem absurdly high.

5) no more than 2-3 b-frames and no more than 3-4 reference frames, higher reference frames slow down encoding speeds to a crawl and do nothing for quality.

6) a GOP of 60 or so offers the best quality, higher lead to more P and B frames and the quality drop is noticeable.

7) turn off x264 deblocking filter, it kills fine details, do use x264's denoise filter but not too high, a value of about 100 seems to offer nice encodes (unless you have really clean source in which case don't use it).

8) use psychovisual enhancements if the codec offers them, divx's shaping filter works great.

9) use some color correcting filter from most encodes, unless again you have really clean source, and even then there are times the encode will benefit.

10) if the codec offers some kind of motion tracking feature, such as GMC in divx and xvid and x264's mb-tree, use it.

11) use the highest trellis setting that a codec offers.

but long GOP's and low bitrates and think that somehow that leads to high quality encodes? LOL.

anytime i here someone talk about compressibility i think to myself that this guy loves shitty encodes.

[poisondeathray]

in my own personal tests, even GOP's of 250 are too long, there's a reason why professional blu-rays and hd-dvd's use high bitrates and small GOP's, so that maximum quality can be ensured (the ability to seek to within a second is a side bonus). i have been running hundreds of test encodes, with some of the shittiest quality dvd sources imaginable, with xvid, divx and x264 and there's a few conclusions i have come to:

I'd be very careful about make broad based generalized conclusions - because there are situations where your comments are clearly wrong . It's going to be situation dependent and source dependent for many things

1) there ain't no substitute for bitrate, none. if you want max quality let the bitrate fly.

Pretty much , yes, but the point of this thread (I think) was about getting better compression. If you don't care about bitrate, why even compress anything? Hell , lets use uncompressed video

2) 2 pass encodes are the way to go, i don't care what the x264 developers say about crf using the same algorithm as 2 pass for rate control, the final output is not the same.

Yes it's not identical, but produces very similar results overall (If you've done the testing correctly there will be some sections better, some worse). If you can show an example that clearly demonstrates otherwise, go ahead

3) skip the quarter pixel detection, it kills visual quality.

Can you explain what you mean by this ?

4) crank up all the quality settings, sans the quarter pixel detection, to the max, even if they seem absurdly high.

Which settings specifically? Later on you complain about encoding speeds being slow but then here you suggest using slow encoding settings ? A bit contradictory ? (I can tell you right now some of them are completely useless and just reduce encoding speeds because they aren't used)

5) no more than 2-3 b-frames and no more than 3-4 reference frames, higher reference frames slow down encoding speeds to a crawl and do nothing for quality.

It's source and situation dependent. Statement #5 here is problematic. This wouldn't be applicable for a low bitrate scenario for example. Or maybe a low bitrate streaming flash scenario

6) a GOP of 60 or so offers the best quality, higher lead to more P and B frames and the quality drop is noticeable.

Again, source and situation dependent. Be careful of making broad generalizations

7) turn off x264 deblocking filter, it kills fine details, do use x264's denoise filter but not too high, a value of about 100 seems to offer nice encodes (unless you have really clean source in which case don't use it).

Again, It's going to depend on the situation. If you're attempting to refine fine details, why are you using a denoise filter at all? Moreover, if your goal was to denoise, you get better result using customized denoise filters. In order to retain fine details you must be on the higher end of the compression curve (higher relative bitrates) - in those cases it might be applicable to disable the inloop deblocking .

[deadrats]

Pretty much , yes, but the point of this thread (I think) was about getting better compression. If you don't care about bitrate, why even compress anything? Hell , lets use uncompressed video

i personally wouldn't mind if all video was uncompressed, the only thing really holding us back is hdd size, but once we get to the 100's of terabyte size i can certainly get on board with using lossless codecs.

me: 3) skip the quarter pixel detection, it kills visual quality.
poisondeathray: Can you explain what you mean by this ?

sure, testing using divx with the quarter pel option enabled and disabled, when i use the qpel option there is a definite degradation of visual quality, almost like actually being able to see the pixel being broken apart, i'm struggling to find a word to accurately reflect what i have seen, almost like a vertical rectangular shaped macroblocking artifact, at bitrates on the lower end of the spectrum, obviously with higher bitrates this doesn't happen.

with x264, and i have run no exaggeration hundreds of test encodes using various dvd sources since i got my 3770k, the best picture quality i have seen is with sub me = 0 (i.e. full pel), all the others result in very noticeable artifacts that resemble a picture that has been broken up and then glued back together and from a distance you don't see the seams but if you go right up to the picture you can see where it was glued back. this effect becomes less and less apparent as sub me increases, at sub me 9 with trellis 2 it's almost completely gone and at sub me 10 (which necessitates trellis 2) it's completely gone, but sub me 10 is brutally slow even a processor like this i7 is brought to it's knees and since it doesn't happen with sub me = 0 then and sub me = 0 is lighting fast, then why bother with sub me at all.

but this brings me to a more general thought, what is the use of quarter pixel detection and why would anyone think that it would be beneficial? a pixel by definition is the smallest picture element that exist and is an 8 bit entity, in h264 from what i understand pixels are arranged in 16x16 partitions under normal conditions and can be 8x8 or in the case of I frames 4x4, why would one conclude that breaking up an 8 bit entity into 4x2bit entities would somehow result in improved picture quality? the only way to get a proper analysis of a pixel is to look at the whole pixel not break it up into 4 different pieces and examine each one individually. to me it makes no sense and to my eyes it results in horrible encodes, maybe it's beneficial if one was to bitrate starve an encode like using 500kb for full d1 resolution video but for any bitrate in the sane range it hurts quality and i can't think of a single reason why it would be beneficial.

me: 4) crank up all the quality settings, sans the quarter pixel detection, to the max, even if they seem absurdly high.

poisondeathray: Which settings specifically? Later on you complain about encoding speeds being slow but then here you suggest using slow encoding settings ? A bit contradictory ? (I can tell you right now some of them are completely useless and just reduce encoding speeds because they aren't used)

first things first, the "completely useless" thing drives me crazy and i'm not talking about in your context, DS, lord mulder and whoever the x264 wiki author have said this numerous times and the obvious question is if the settings are completely useless then why bother coding them into the encoder in the first place? what's the point of writing the code, including it as a configurable parameter and then saying "don't use it, it's useless", that to me is the sign of a programmer that needs to lay off whatever the hell he's smoking.

with regards to what settings i'm talking about i'm specifically talking about motion search type, tesa, no matter what the wiki tells you and ranges of up to 64, in my tests, substantially improve quality, you need a beefy cpu if you're going to use tesa and range=64 with HD content (or a lot of patience).

in fact i ran a battery of tests recently, using dvd sources and media coder, xmedia recode and video mastering works 5, i loaded up the dvd source (i used progressive dvd) and i created a batch job of dozens of test encodes and i configured the test encodes thus: i disabled every single setting or turned it down to the lowest setting available, e.g weighted p, weighted b, partitions, sub me, you get the idea. then on each test encode i started cranking up one setting until the max, so i set up a control test encode with everything at the bare minimum, then i set up a test encode with sub-me=5, =7, =9, =10, motion search i went through all 5 options, with diamond and hexagonal i went with a range of 16, for umh i went with a range of 24, for esa i went with 32 and for tesa i went with 64. you get the idea, one test encode with the only quality option enabled being all partitions, for the mb-tree i tested with lookahead=60 and weighted p=2 and so on.

the biggest single improvement, by a mile, was using tesa and 64, the difference between everything at bare minimum and everything at bare minimum sans tesa+64 was like night and day.

trellis=2 also made a significant impact, mb-tree for the most part was well worth it, the psychovisual enhancements by themselves didn't really seem to help and in some case hurt but psychovisual+mb-tree+trellis=2 offered significant benefits, i'm almost tempted to say that you shouldn't use one without the others.

subme seemed to hurt visual quality up until i started getting to subme=7+trellis=2 and then it started getting better up until as i said subme=10 brought me back to were i have been with subme=0.

me: 5) no more than 2-3 b-frames and no more than 3-4 reference frames, higher reference frames slow down encoding speeds to a crawl and do nothing for quality.

poisondeathray:
It's source and situation dependent. Statement #5 here is problematic. This wouldn't be applicable for a low bitrate scenario for example. Or maybe a low bitrate streaming flash scenario.

i didn't really test low bitrates, but i can tell you that in mid to high bitrates higher numbers of reference frames and b frames hurt visual quality and 16 reference frames severely slow down encoding speeds, in test after test, encodes that would take 30 minutes with a i7 3770k would take over 90 minutes simply by going from reference frames=3 to 16, with no other changes. that's just ridiculous.

me: turn off x264 deblocking filter, it kills fine details, do use x264's denoise filter but not too high, a value of about 100 seems to offer nice encodes (unless you have really clean source in which case don't use it).

poisondeathray: Again, It's going to depend on the situation. If you're attempting to refine fine details, why are you using a denoise filter at all? Moreover, if your goal was to denoise, you get better result using customized denoise filters. In order to retain fine details you must be on the higher end of the compression curve (higher relative bitrates) - in those cases it might be applicable to disable the inloop deblocking.

i know it seems contradictory, but in my tests, using external denoise filters did kill fine details but i found that if i set deadzones to 0,0 with nr=100 and inloop deblocking=0 then noise was reduced but fine details were maintained, a result that surprised me.

now i'm sure if one were so inclined one could find some source somewhere that would behave differently that i have laid out but based on my tests, i believe that for the majority of encodes what i said holds true.

[poisondeathray]

me: 3) skip the quarter pixel detection, it kills visual quality.
poisondeathray: Can you explain what you mean by this ?

sure, testing using divx with the quarter pel option enabled and disabled, when i use the qpel option there is a definite degradation of visual quality, almost like actually being able to see the pixel being broken apart, i'm struggling to find a word to accurately reflect what i have seen, almost like a vertical rectangular shaped macroblocking artifact, at bitrates on the lower end of the spectrum, obviously with higher bitrates this doesn't happen.

So this was with divx h.264 encoder ? Can you be more specific ? or post a screenshot or video to illustrate ?

In theory , for encoders that use qpel, but do not have some type of RDO calcuation , it can actually decrease quality

with x264, and i have run no exaggeration hundreds of test encodes using various dvd sources since i got my 3770k, the best picture quality i have seen is with sub me = 0 (i.e. full pel), all the others result in very noticeable artifacts that resemble a picture that has been broken up and then glued back together and from a distance you don't see the seams but if you go right up to the picture you can see where it was glued back. this effect becomes less and less apparent as sub me increases, at sub me 9 with trellis 2 it's almost completely gone and at sub me 10 (which necessitates trellis 2) it's completely gone, but sub me 10 is brutally slow even a processor like this i7 is brought to it's knees and since it doesn't happen with sub me = 0 then and sub me = 0 is lighting fast, then why bother with sub me at all.

So you're saying best quality at subme 0 ? you're suggesting all values in between 1-9 inclusive essentially useless? Can you post a comparison example that illustrates this?

Certainly at lower bitrates the differences between low subme and higher subme values are clearly evident, but between something 9 and 10 it's very difficult to distinguish. But you're saying skip everything except 0 or 10? Of course at high bitrates everything looks fine

How did you do these tests? 2pass ? CRF etc... ? what bitrate if 2pass? what were the test conditions ?

but this brings me to a more general thought, what is the use of quarter pixel detection and why would anyone think that it would be beneficial? a pixel by definition is the smallest picture element that exist and is an 8 bit entity, in h264 from what i understand pixels are arranged in 16x16 partitions under normal conditions and can be 8x8 or in the case of I frames 4x4, why would one conclude that breaking up an 8 bit entity into 4x2bit entities would somehow result in improved picture quality? the only way to get a proper analysis of a pixel is to look at the whole pixel not break it up into 4 different pieces and examine each one individually. to me it makes no sense and to my eyes it results in horrible encodes, maybe it's beneficial if one was to bitrate starve an encode like using 500kb for full d1 resolution video but for any bitrate in the sane range it hurts quality and i can't think of a single reason why it would be beneficial.

In long GOP encoding, you're storing the difference between what is predicted and the original, that is the stored "residual" that is what takes up actual data and bitrate . When you have more accurate motion vector predictions, there is less difference, thus less data needs to be stored in the residual = smaller filesize . Since you have interpolated "inbetween" values with the energy for the predicted macroblock movement is less. But the motion vector data is more complex and costs more - that's why you need RD optimization models to reduce the size - so this is sort of trade off (which is what video compression is really all about)

me: 4) crank up all the quality settings, sans the quarter pixel detection, to the max, even if they seem absurdly high.
poisondeathray: Which settings specifically? Later on you complain about encoding speeds being slow but then here you suggest using slow encoding settings ? A bit contradictory ? (I can tell you right now some of them are completely useless and just reduce encoding speeds because they aren't used)

first things first, the "completely useless" thing drives me crazy and i'm not talking about in your context, DS, lord mulder and whoever the x264 wiki author have said this numerous times and the obvious question is if the settings are completely useless then why bother coding them into the encoder in the first place? what's the point of writing the code, including it as a configurable parameter and then saying "don't use it, it's useless", that to me is the sign of a programmer that needs to lay off whatever the hell he's smoking.

In this context certain values are "completely useless" because they are not even used, even though they have "cost" (computational cost, CPU cycles, slow encoding) . e.g. High merange settings for a DVD source is "effectively" useless. Why? There is a hard limit of 720x480 pixels. You can go with an analyzer and figure out if where you might have missed 1 or 2 macroblocks with higher settings. In the end, 1 or 2 macroblocks which might have been "caught" with a higher radius equates to about 0.0000000001% difference . So I guess it might not be "completely" useless (but probably is if you look with an analyzer), Higher --merange can make a difference on higher resolution sources like 1080p , 4k .

Similar to something like b-frames, reference frames which can be completely useless. It's source dependent. Some can actually use 16 reference, 16 b. Some do not. Entering a value doesn't necessarily mean they are used - it's source dependent . For b-frames and references - you have to look at your logs to see what can actually be used on that source

These options are available because people want them. It's up to the user if they want to increase encoding time by 18x for a 0.000001% improvement in quality or something in between.

It's up to end user to make those decisions and tradeoffs - and that's what video compression really boils down to at the end of the day - trade offs

with regards to what settings i'm talking about i'm specifically talking about motion search type, tesa, no matter what the wiki tells you and ranges of up to 64, in my tests, substantially improve quality, you need a beefy cpu if you're going to use tesa and range=64 with HD content (or a lot of patience).

If you compare best to worst settings at a given bitrate, of course you will see a difference. But if you're seeing a difference with --me tesa and --merange 64 on a DVD source vs "typical" settings .... prove it. Unless this is a very low bitrate encode (like <100-200kb/s) . If you're seeing differences, I'm going to guess you're not doing these tests in a controllled fashion or
there is problem with your testing methods

in fact i ran a battery of tests recently, using dvd sources and media coder, xmedia recode and video mastering works 5, i loaded up the dvd source (i used progressive dvd) and i created a batch job of dozens of test encodes and i configured the test encodes thus: i disabled every single setting or turned it down to the lowest setting available, e.g weighted p, weighted b, partitions, sub me, you get the idea. then on each test encode i started cranking up one setting until the max, so i set up a control test encode with everything at the bare minimum, then i set up a test encode with sub-me=5, =7, =9, =10, motion search i went through all 5 options, with diamond and hexagonal i went with a range of 16, for umh i went with a range of 24, for esa i went with 32 and for tesa i went with 64. you get the idea, one test encode with the only quality option enabled being all partitions, for the mb-tree i tested with lookahead=60 and weighted p=2 and so on. the biggest single improvement, by a mile, was using tesa and 64, the difference between everything at bare minimum and everything at bare minimum sans tesa+64 was like night and day.

Of course worse possible settings vs. very highe settings is going to show a difference

I'm saying you could probably get similar results in 5-10x less encoding time . --me tesa and --merange 64 on a DVD source using "medium to high bitrates" is just plain crazy

me: 5) no more than 2-3 b-frames and no more than 3-4 reference frames, higher reference frames slow down encoding speeds to a crawl and do nothing for quality.

poisondeathray:
It's source and situation dependent. Statement #5 here is problematic. This wouldn't be applicable for a low bitrate scenario for example. Or maybe a low bitrate streaming flash scenario.

i didn't really test low bitrates, but i can tell you that in mid to high bitrates higher numbers of reference frames and b frames hurt visual quality and 16 reference frames severely slow down encoding speeds, in test after test, encodes that would take 30 minutes with a i7 3770k would take over 90 minutes simply by going from reference frames=3 to 16, with no
other changes. that's just ridiculous.

Yes, in most cases it is.... But then you advocate --me tesa --merange 64 on a DVD source! Talk about contradiction. I agree --ref 16 is usually useless (it's about as useful as --me tesa) . If you analyze your encode logs, you can fine tune what the actual used values are for that source. Same as b-frames. Entering 16 doesn't mean 16 are always used in the highest string

me: turn off x264 deblocking filter, it kills fine details, do use x264's denoise filter but not too high, a value of about 100 seems to offer nice encodes (unless you have really clean source in which case don't use it).

poisondeathray: Again, It's going to depend on the situation. If you're attempting to refine fine details, why are you using a denoise filter at all? Moreover, if your goal was to denoise, you get better result using customized denoise filters. In order to retain fine details you must be on the higher end of the compression curve (higher relative bitrates) - in those cases it might be applicable to disable the inloop deblocking.

i know it seems contradictory, but in my tests, using external denoise filters did kill fine details but i found that if i set deadzones to 0,0 with nr=100 and inloop deblocking=0 then noise was reduced but fine details were maintained, a result that surprised me.

I'm guessing you're probably not using proper filters. Let's say for example have ample bitrate - I'm 100% certain you can get better results with customized filters. x264's nr is a "dumb" algorithm

now i'm sure if one were so inclined one could find some source somewhere that would behave differently that i have laid out but based on my tests, i believe that for the majority of encodes what i said holds true.

I see so you're making test statements based on tests from 1 DVD source :P ... Your 1 set of test has positive predictive value over all the other types of sources (anime, action, CGI, drama, high action, sports, newcasts...) :P

[deadrats]

@poisondeathray

with regards to tested codecs, i did not use divx's h264, i used divx (asp) and x264; i used one progressive dvd source for the formal tests i outlined but as i mentioned in another thread i have been doing dozens of test encodes with those greek dvd's i mentioned and my findings hold up when applied to those dvd's.

with regards to subme, yes, i am saying that all subme between 1 and 9 inclusive are useless, at 6 when rdo starts coming into play they become less useless, but if you add trellis=2 to the mix they start becoming less and less useless, but from what i have seen subme=0 results in the best quality from what i have seen, though it's possible that subme=11 would beat it but i haven't tested it out because it's painfully slow, so yes, i'm saying skip everything except subme=0.

with regards to bitrate used i used 4mb/s for video with 2pass.

In long GOP encoding, you're storing the difference between what is predicted and the original, that is the stored "residual" that is what takes up actual data and bitrate . When you have more accurate motion vector predictions, there is less difference, thus less data needs to be stored in the residual = smaller filesize . Since you have interpolated "inbetween" values with the energy for the predicted macroblock movement is less. But the motion vector data is more complex and costs more - that's why you need RD optimization models to reduce the size - so this is sort of trade off (which is what video compression is really all about)

perhaps this is why i got the results i got, i didn't use long GOP's, as i stated i found long GOP's to result in lower quality, i used a GOP of 60 and don't think that GOP's much longer than that should be used.

with regards to tesa+64 and dvd resolutions, i can clearly see a difference in my tests between umh+24 and tesa+64, especially in well lit scenes, like an outdoor shot or lots of colors. in dark scenes, where detail isn't visible anyway, even hex+16 is fine.

it's late right now, tomorrow i'll do a number of test encodes and post samples so that you can judge the difference for yourself; if you like you can supply the test footage, preferably something bright with lots of colors and decent motion.

[somespirit]

on picture is standard profile provided in MeGUI and min GOP size = 2 (it is ok), but max GOP size = 24 maybe they have video where change of scenes is every second ha ha ha ........ and this is Blu-Ray specification profile .........who knowz

[poisondeathray]

]perhaps this is why i got the results i got, i didn't use long GOP's, as i stated i found long GOP's to result in lower quality, i used a GOP of 60 and don't think that GOP's much longer than that should be used.

You did use long GOP encoding - Technically, any non-intra encoding is "long GOP"

The P-frame is called a "P" frame because it's a "predicted" picture. It's storing the residual predicted from the IDR frame (or other P-frames if it's not the 1st P frame)

with regards to tesa+64 and dvd resolutions, i can clearly see a difference in my tests between umh+24 and tesa+64, especially in well lit scenes, like an outdoor shot or lots of colors. in dark scenes, where detail isn't visible anyway, even hex+16 is fine.

Very interesting - I'd like to see the comparisons

[poisondeathray]

on picture is standard profile provided in MeGUI and min GOP size = 2 (it is ok), but max GOP size = 24 maybe they have video where change of scenes is every second ha ha ha ........ and this is Blu-Ray specification profile .........who knowz

Yes, blu-ray has many rules for compliance.

Each restriction that you have to follow in order to meet compliance potentially reduces quality

[deadrats]

ok guys, i just finished doing a bunch of tests, here's the links for 6 test encodes:

http://www.filedropper.com/sintel20101128x480x2642456subme9

http://www.filedropper.com/sintel20101128x480x2643132esa

http://www.filedropper.com/sintel20101128x480x264baseline2145

http://www.filedropper.com/sintel20101128x480x264trellisalways2155

http://www.filedropper.com/sintel20101128x480quanttypempegmki60kt50amc2705

http://www.filedropper.com/sintel20101128x480xvid3431

a word about the tests, i used Sintel 4k, here's the media info for the source:
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 14mn 48s
Bit rate : 40.0 Mbps
Width : 4 096 pixels
Height : 1 744 pixels
Display aspect ratio : 2.35:1
Frame rate mode : Constant
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.233
Stream size : 4.05 GiB (96%)
Writing library : x264 core 115 r1947 b5a8ad7
Encoding settings : cabac=1 / ref=4 / deblock=1:-3:-3 / analyse=0x3:0x133 / me=tesa / subme=10 / psy=1 / psy_rd=0.80:0.30 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-4 / threads=3 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=6 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc=2pass / mbtree=0 / bitrate=40000 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00 / zones=80,751,b=2.0/1572,1954,b=2.0/12210,12303,b=2.0
Language : English
Default : Yes
Forced : No
Matrix coefficients : BT.709

Audio
ID : 2
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Format settings, Endianness : Big
Codec ID : A_AC3
Duration : 14mn 48s
Bit rate mode : Constant
Bit rate : 640 Kbps
Channel(s) : 6 channels
Channel positions : Front: L C R, Side: L R, LFE
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Stream size : 67.8 MiB (2%)
Title : AC3 5.1 @ 640 Kbps
Language : English
Default : Yes
Forced : No

interestingly enough, i can't seem to find the link for this particular file, it's almost like they pulled it from the site, at any rate, the test encodes i provided are as follows:

i used tmpg video mastering works and i resized to 1128x480 at 2mb/s (much lower than what i would normally use for my regular encodes).

i did a baseline test with x264 with all the settings either disabled or turned to the bare minimum, this encode took 21:45, i then did a test encode where i only changed "diamond" to "esa" and that encode took 31:32, i then put the setting back to "diamond" and changed subme=0 to subme=9 and that encode took 24:56, i then changed it back down to subme=0 and set trellis=2 (always) and that encode took 21:55, the gop for all encodes was set to 60, as was b-frames set to 3 and reference frames also equaled 3 and 1 pass vbr with a max bitrate of 4mb/s (as i said target bitrate was 2mb/s) and all x264 encodes had chroma me enabled.

for the divx encode i used the "balanced" preset, cbr, bidirectional encoding was set to "adaptive multiple consecutive" and quantization type was set to "mpeg", note this was not with the vfw divx version but rather the version built into mastering works. maximum keyframe interval was set to 60 and keyframe threshold was set to 50%. this encode took 27:05.

for the xvid i used the latest vfw build and configured it as follows: profile=unrestricted, quantization type=mpeg, max consecutive b-vop=3, closed gov, packed bit stream, chroma optimizer enabled, motion search was set to "ultra high", vhq mode was set to "wide search", vhq was enabled, as was "use chroma motion" and "use vhq for b-frames", max I frame interval was set to 60 and trellis was enabled, everything else was left at default, total encode time was 34:31.

the test hardware was an i7 3770k with 8gigs of ddr1333, cpu usage for all of the encodes stayed under 50% and ram usage was about 1.3gigs.

it should be noted that if you're planning on playing around with 4k sources, todays cpu's are not fast enough, the movie is just under 15 minutes and despite only targeting 2mb/s with sub 720p resolutions the fact that i was using a 4k source was still enough to ensure that i didn't see real time encoding and this is with x264's settings being turned all the way down. if you're planning on encoding anything at max settings and 4k resolutions you better count on measuring encoding times with an hourglass.

as a side note, i noticed that the 4k source i used list threads=3, that means these guys encoded that monstrosity using a dual core, i admire their patience.

as far as which settings won the quality test, i'll let everyone make up their own mind.