TEST: Multi-pass VBR vs. CQ VBR

[therick]

on the contrary kwag, I think there is a very good reason to continue the tests...and continue them I have-

I am not going to use the web to post the samples any longer, I am going to use my FTP site:

robbins.dns2go.com l/p : vcdhelp/vcdhelp

Now I have finished the 2nd round of tests and put them out on the site...there are now four samples:

one was encoded with TMPEGEnc in 2-pass VBR mode
one was encoded with CCE in 2-pass VBR mode
one was encoded with TMPEGEnc using kwag's full D1 CQ template
one was encoded with CCE in its CQ mode

All files are (approximately) the same size...about 48.4 MB. So they are all even on the filesize side of things.

So download them and judge for yourself...

@adam - I am prefectly willing to do another high-quality round of tests at DVD reslution... or another round using the 2 mbit (CVD)range...

@kwag - will you supply me with a version of your CQ template that matches the "high quality" parameters given by adam?

Also, something I must note for those of you who use CQ because you claim that it is a lot faster than TMPEGEnc's 2-pass...I've got news for you...using kwag's template (over MANY test encodes) I am not seeing a huge time difference. And as for speed, CCE still absolutely destroys TMPEGEnc...

[adam]

Kwag, In regards to SVCDS, I don't consider 2mbits low bitrate, I consider 1.5 mbits and under low bitrate, especially with a resolution of 704x480!

Most people who encode SVCDS use the bitrate as the standard was intended, to fit 40-55 mins of high quality video onto a single cdr. This means avg bitrates of 1.7mbits to 2.4mbits and resolutions of 480x480/576. If you want to test CCE vs TMPGEnc at ultra low bitrates like 500-700k then fine, but this test means absolutely nothing to a whole lot of people.

If you want to test CCE vs TMPGenc at encoding SVCD's then use max 2.5mbits avg around 2mbits and use a more reasonable resolution. Even at 2mbits per sec avg with a resolution of 704x480 your still getting a worse bits per pixel ratio than a standard VCD! With your templates as the starting point there is no way to test the SVCD output of these encoders, only very low bitrate non-standard xsvcds.

If you want to test CCE vs TMPGenc at encoding DVD's then use max 9mbits, avg 6mbits. With your templates as the starting point this is not possible

If you want to test a Ferarri you don't leave it in 2nd gear. How about we run some tests that actually have some real world applications, like how well the encoders work when they can actually take advantage of the max bitrates of each respective format.

"There's no sense in doing any more tests. TMPEG is better for low bit rates. Tried and proven. And at higher bit rates, there's no difference between encoders"

Exactly what tests are you basing these conclusions on? The one where you encoded one sample to mpeg1 and the other to mpeg2, or was it the one where you preferred the quality of one encode while others preferred the quality of the other?

I sorta agree with one of your points though. At reasonable to high bitrates (2mbits and up) there is still a difference between the two encoders but its slight and its subjective. Isn't this what I've been saying all along?

I never admitted TMPGEnc outperforms CCE at low bitrates. I simply implied that this was the general consensus but its still worthy of testing. I was simply making the point that the current tests were too narrow and that they were only testing something which is probably already known.

[Martyn1980]

I have to stick my head in again here after reading a few things, and first ive got to admit ive been impressed by Tmpeg again, not by anyones opinions or propoganda but by my own tests ive conducted.

First thing ive got to say is the best way to get quality is use CCE x pass at a higher maximum bitrate than 2300, increase it to 2600 and there is a difference in blockiness and the filesize does not change and its within standards of SVCD, some people may disagree, but using a max of only 2300 CCE is inferior that is obvious, i personally go higher than 2600 and most people on here either have or could easily get a DVD player that would let them go as high as 5000, so i dont think thats a problem, Kwag your templates do wonders with what they are given, but if your player properly supports SVCD then CCE x Pass VBR at a high maximum is the best quality route, Tmpegs CQ cant predict filesize accurately and its VBR is not the greatest.

AS for a CBR encode at 7000kbps CCE is superior but its not by much and its not $1950 better, the biggest difference is the speed. Tmpeg is a fantastic bit of software for the money and if it had a better VBR and was faster then it would crush CCE but at the moment with only a 2 pass VBR (which is only equivelant to CCE x Pass VBR with 1 pass selected) and the fact that even if it could do a proper 3 or 4 pass like CCE's a 2 hour movie would take over a day even on a top system.

One more thing, what about 1 pass VBR on CCE, i personally havn't used it and dont know how to properly, cant Kwags template be applied to that to try similar tests, i havn't used 1 pass, and i probably never will, but it was just a thought.

Kwag you shouldn't come across as so biased, its part of the reason i was so against Tmpeg at the beginning.

[kwag]

@therick

Ok, because I don't have a "high bandwidth" connection, I downloaded the first 2.5MB of each sample. I can see clearly that sample 2 and 3 are TMPEG's. Right? Then I slowly stepped through a couple of frames in slow motion, and noticed that on the TMPEG samples, there were less artifacts, but on high motion, I could see some ( barely visible ) blocks. So I decided to check why. Can you explain why you used 9,800Kbps MAX on CCE's samples #1 and #4 I hope it's because you forgot to set it at 5,000Kbps . Now let's analyze something here. If you look closely 2 seconds into the sample, and pause right where the scene changes to a view from the interior of the truck. You see the camera behind the couple, when he's holding the cigar ( wonder if her name is Monica ) . Look very closely at samples #1 and #4. You'll see very small artifacts around their noses. But on samples #2 and #3, they're hardly visible! This is NOT a subjective view, this is a REAL VISIBLE view that everyone can see. This part is an excelent example of TMPEG's superior motion estimation algorithm, because their noses are almost still, but the background is moving, as seen through the windshield. Now, even though bit rate viewer can't detect very small spikes of high bit rate, because in an order of milliseconds, bit rate's granularity or sampling just wont display it, there are short bursts over 5,000Kbps in the clips. But in TMPEG, the MAX bit rate was truncated to 5,000Kbps. So, if you drop the bit rate on CCE to 5,000, the samples will look worse. If you increase TMPEG's MAX to 9,800, as in CCE, it will look blockless as in CCE's samples. But CCE's artifacts will be there. So now, what's next?

@Martyn1980

At 7,000Kbps CBR or whatever, the same principle applies. The motion estimation in TMPEG is better than CCE's, no matter what encoding mode you use, be it VBR, CBR, X-Pass, etc.

@All
And I'm not biased. I just picked a point of reference which shows exactly what I mean. A still front with a moving background, which is a very good point of reference for analyzing encoders.

-kwag

[kinneera]

NTSC DVD's are recorded as CCIR601. That is the digital standard television signal, with a scale from 0 to 255. That's why I suggested to change to 0-255, because the sample mentioned for the test, is from a DVD. 16-235 is used for viewing on PC monitors.

Right, except you got that backwards. 0-255 corresponds to the full 8-bit RGB colorspace which is used on computer monitors. The various luminence/chroma formats (ie YUV) used for television are almost always 16-235.

[kinneera]

right where the scene changes to a view from the interior of the truck... Look very closely at samples #1 and #4. You'll see very small artifacts around their noses. But on samples #2 and #3, they're hardly visible!

Right after a scene change huh? I would be willing to bet that the person using CCE did not enable scene change detection and the person using TMPGEnc did - CCE's default is to have it off and vice versa for TMPGEnc.

Also, CCE's 'quality' slider can have a lot to do with the type of noise you're going to see, such as minor details around noses. This does not mean its motion-detection algorithm is poorer, only that it has been given a different bias through encoding settings. I don't recall TMPGEnc even allowing such a thing.

Too many variables, too little experience with encoders. This is why we never get anywhere with these tests. This is why it isn't fair to make biased blanket statements about one encoder always being better than another. This is why people are going to do whatever they subjectively think is giving them the best results. Just give it a rest already...

[therick]

@kwag - unbeleivable. I try to do a real test, with real parameters, blindly, and this is what I am hit with?

"Can you explain why you used 9,800Kbps MAX on CCE's samples #1 and #4 "

The max for all 4 files was 5000. Period. kwag, I consider myself an adult...what would I possibly gain by cheating?...and if I did, explain how I hid it? Ok, kwag, here we go-I have not run bitrate viewer on any of these files, except the one that came out of your CQ template-but I challenge you to show me, in bitrate viewer, where ANY of the samples go above 5 mbits.

You want to go around this board and accuse me of lying?

******* PROVE IT.

[therick]

@kineera - can you tell me where the scene change detection settings are in CCE?

[kwag]

NTSC DVD's are recorded as CCIR601. That is the digital standard television signal, with a scale from 0 to 255. That's why I suggested to change to 0-255, because the sample mentioned for the test, is from a DVD. 16-235 is used for viewing on PC monitors.

Right, except you got that backwards. 0-255 corresponds to the full 8-bit RGB colorspace which is used on computer monitors. The various luminence/chroma formats (ie YUV) used for television are almost always 16-235.

Yes, you're right. I should have said "0-255 is used for viewing on PC monitors". Because for the purpose of this tests, most peple wouldn't burn to CD anyway, just looked at the samples on the monitor.

@therick

Sorry, but you didn't use 5,000 as MAX. The samples I did with CCE, if run through bit rate viewer, do show the MAX. Your's show CLEARLY 9,800Kbps in bit rate viewer. BOTH. And bit rate viewer doesn't lie. If you did a mistake, accept it, that's fine. But if you insist you encoded at MAX 5000, well .....

Did anyone run bit rate viewer on any of the samples?

-kwag

[Martyn1980]

Kwag your talking to me like i didn't actually do the encodes, i ran 7 clips through a variety or bitrates on CBR on both encoders and i prefered the CCE ones, less blocks, nearer to the original source, i usually watch the original first, so i dont forget about the quality of the original, what im expressing is an opinion, other peoples opinion might differ, what im not doing is saying the Tmpeg ones are shit and they dont have this or whatever, earlier on you said at 6000kbps there was no difference, now your saying Tmpeg is superior, make up your mind, a few days ago you were also saying CCE was better at high bitrate Mpeg2, what next.

therick im going to try to download the clips now and i'll compare them in a bitrate meter and in Power DVD, i think kwags is on the blink, it always has been, if ity worked then his CCE encodes would be the same size as his Tmpeg CQ ones, mine always are, give or take a few k.

im having trouble finding them, i dont want to sound dumb here but how do i access your ftp site cause i can only get the web one, with the old clips

[therick]

post them kwag...I want to see them. I did NOT put the max at 9,800, it was at 5,000.

[kwag]

right where the scene changes to a view from the interior of the truck... Look very closely at samples #1 and #4. You'll see very small artifacts around their noses. But on samples #2 and #3, they're hardly visible!

Right after a scene change huh? I would be willing to bet that the person using CCE did not enable scene change detection and the person using TMPGEnc did - CCE's default is to have it off and vice versa for TMPGEnc.

Let's BROADEN the term "scene change". I said "after the scene changes" But I didn't say "THE FRAME AFTER THE SCENE CHANGE". So after that scene change, you can "slow motion" frame after frame after the scene change and you'll see the effect. TMPEG's equivalent would be the "motion search precision".
And for your information, ALL KVCD templates have "detect scene change" OFF in TMPEG.

-kwag

[kwag]

post them kwag...I want to see them. I did NOT put the max at 9,800, it was at 5,000.

[therick]

ok, I see what happened-I have looked at the file in bitrate viewer kwag...and the Nom. bitrate DOES say 9800 kbit/sec...it only says that because I checked the DVD compliant box in CCE. My fault, my fault...I am going to redo the encodes without it checked...

honsestly, though, did you ever actually see any of the stream go above 5 mbits on the graph?

[therick]

@kwag - I know you said you couldn't download the clips, so here is bitrate viewer for the ones in question:

Sample #1



Sample #1 (done w/o DVD compliant checked)



Sample #4



Sample #4 (done w/o DVD compliant checked)



As you can see, although bitrate viewer lists those first files with a Nom. bitrate of 9800 kbit/sec, they never get above 5000...and in the second encodes, you can see the Nom. bitrate is now 5000 kbit/sec. The files are almost identical. I wasn't lying...I was just stupid

[Martyn1980]

Kwag i imagine you have Power DVD, what do the files say in that, it usually gives me an accurate reading.

[vhelp]

therick,

ok, be nice I was looking for forward to another episode of such issues
and already some off words are used.

In any case, I'm happy to be back surfing, after being w/out phone for
approx a week.

Few! I've ben reading all posts, and had many of my opinions, about
this and that, but forgot most of what was on my train-of-thought during
my busy work day(s)

I will be following along - and possibly have my opinions to add too.
But, expect only honest ones, and I'm on know-ones side, he he....
...only the encoders. And, although I still prefer TMPG over CCE, I am
not one too taint/tarnish CCE's performance, as it has it's good points
or features over TMPG, and vis-versa

I've ben playing around some more with CQ, CQ_VBR, 2pass via TMPG,
and CCE's 3pass (4 if you count the *.vaf file)

The issue w/ color space:
I find that the 16-235 is the best setting to use, and all tests TMPG vs. CCE
should be set accordingly
* CCE, just make sure you have the [x] 16-235 checked
* TMPG, just make sure you have the [ ] Output YUV... UN-checked
then, these two will be in the correct color space (16-235)

CCE's Quantom Matrix scale THEORY:
I believe (theorictically) that CCE uses some kind of logorithmic or
quantom scale, that, during encoding, this invisble Matrix is being
scalled in real-time, per encoding algorithem during encoding.
Again, it's just a theory I have! And, it seems to fit since you can't
or have any access to it. Makes sense to me. so, I sticking to it,
till otherwise corrected.

TMPG's Quantom Matrix scale:
I too, have ben working very honestly on a setting or too to incorporate
into a template for TMPG. So far, I have two that seems to yield some
fiar quality. But, the settings really depend on the quality of the
source, if it's from a captured source. I haven't really tested the
two on a DVD source yet. And, if my hunch is correct, as long as its a
DVD source material, then maybe, just maybe the quality will maintain
through out the encode. I'm just not up to it yet, as my MAIN concirn
is with/for AVI captures. So, until I nail it, I won't finalize my
Matrix scale settings' findings.

As far as a sample length to more accuratly judge a given bitrate or
bitrate's performance, a large file would problably yield closer
results but would put most of us viewers who are trying to follow
this thread. But, I do feel a larger file is better unfortunatly.
But, if at all possible, I wouldn't mind a 20MB file. 45MB is pretty
taxing, unless you have a Cable or DSL available. So, remember this
when you continue your Bitrate Battle.

Bitrate AVERAGE rate or number:
Now, as for the Average number retrieved via bitrate viewer.
I agree w/ Adam on this, that you CAN'T get an accurate number.
Ok, for those who are still having trouble with this. Cinsider this:
1 * you encode a 1 minute clip (DVD source)
2 * you then look at it via bitrate viwer
3 * you grab that Average number used (say it's 2.1mb)
4 * you encode the same 1 minute clip (DVD source) with this Average no.
Ok, with me so far.
Take a nother look at the bitrate viewer's graph from that same clip
initial 1 minute clip, and notice the scale and values used throughout
the sequence window.
Ok, here is the catch. and, I'm assuming this is where your eyes will
be opended
Take that same 1 minute clip you encoded, but this time:
1 * you encode a 2 minutes instead (DVD source)
2 * you then look at it via bitrate viwer
3 * you grab that Average number used (say it's 1.8mb)
4 * you encode the same 2 minute clip (DVD source) with this Average no.
Now, do you see why Adam says that you can't use bitrate viewers
Average for the give 1 minute (or two minute) clip above???????????
It's SO NOT accurate! But, maybe, just maybe, if you encoded the
whole DVD movie, you could then get/use the Average bitrate number.
But then again, I understand the even bitrate viewer is NOT so accurate
anyways. So, use the average at your own risk. But, still, it's better
than guessing at an average... ain't it?

I'm not sure I understand this clearly enough. So I'll be giving my
own opinion here.
I agree that comparing CQ vs. 2 or 3pass VBR is like comparing Apples
to Oranges.
But, if CCE had a CQ rate control mode as TMPG does, then it WOULD be
comparing Apples to Apples. But, for the time being, it's Apples to
Oranges. So, expect much friction here or there a little throughout.
As far as who's right or wrong? I hope it doesn't get too personal.
Let's just all get together and combine our knowledge, and use what
we find to help others... as guides towards encoding approaches per
givin capture/rip/encoding projects... or not, he, he...

Oh, and yes, I like the blind taste test - - oh, I mean the sample
viewing test.
Oh, and again, a standard needs to be utilitzed if you're going to
ax it out again. Stop using different source materials for
tests arguments... please! Set a standard now. So, can someone
through out a specific source, and scene, and how long the clip to
encode, etc. so that everyone else here following can do their own
tests

therick
couldn't you shrink (resize) those images? Now all the proceeding
pages will have to be panned RT/LFT very enoying. But, then again,
actually, it's you thread... but would be nice
ok, ok, i'll have to do something for you - - just wait a minute or too.

Well, this is all I could come up with for now...
Expect more from me in the coming pages maybe.

-vhelp

[vhelp]

therick,

here, hope this helps. If you want to use it, just copy it to your website
and point your url to it as I did.

Samples show in the text part of the images per .jpg (.gif) image page.
...so, nothing is lost otherthan size. A good comprimise in future image
posting

[kinneera]

Let's BROADEN the term "scene change". I said "after the scene changes" But I didn't say "THE FRAME AFTER THE SCENE CHANGE". So after that scene change, you can "slow motion" frame after frame after the scene change and you'll see the effect.

I guess I have to spell this out a little more clearly. If one encoder inserts an I-frame immediately once it sees the scene change, there will be far fewer artifacts in all of the subsequent frames compared to an encoder that simply continues to propagate P and B frame errors based on a completely different I-frame. Therefore, it is not just the one frame after the visual scene change that will be affected. It's good to know you don't have scene detection on, but even still there's a substantial discrepancy in the size of the GOPs being used, so it would require some effort to determine if one encoder placed an I-frame at a more opportune position than the other. I hope that this is helping to drive home the point of why making comparisons when variables haven't been equalized is of dubious value at best. Especially when you are nitpicking to the point of stepping through individual frames after a scene change analyzing someone's nose. I'm sure there are probably places in the movie where a scene change occurred and CCE encoded an I-frame sooner, producing the exact opposite result of what you are seeing.

TMPEG's equivalent would be the "motion search precision".

Don't think so. Read the CCE documentation. The quality slider largely affects the tradeoff between two major types of artifacting, it is not a direct correlation to the motion search algorithms. For example, it doesn't affect the encoding speed, whereas that setting clearly does in TMPGEnc.

[therick]

@vhelp - yeah I was just beginning to think that bitrate viewer was crap...but I have heard adam criticize it and I am now in that boat. What you say makes sense, though. Opinions are what I want...I will post some new clips (around the 20 MB size) this weekend. Another scene of the movie that would be good for testing the encoders has just occurred to me

[vhelp]

therick,

If I may throw in a suggestion on a set scene. . .

If you (or anyone else) can find one has the following in it:
* slow (some slow, like talking, so we can see how the bitrate hovers
at lower levels) then...
* FAST (lots of movements, like hands/running/fighting, etc)
* Slow again.

I'm hunching that we can get a better perspective of the bitrate
from when it's at a maderate levels, then jumps up into high gear, and
then back down to a moderate levels
I think that this will give us a better picture of HOW the bitrate is
being applied in a give scenario - - at least that's how I'm visualizing
it.

-vhelp

[therick]

actually it is funny that you say that because that is why I chose the scene I did...only it is little different-it starts out with a lot of motion (Logan flying through the windsheild)-then goes to slow as he gets up, exchanges glances with Rogue and his head heals-goes tight into his face as he smells Sabretooth-and speeds up again at Sabretooth's attack and the appearence of Storm and Cyclops and then the explosion of the van.

but if there is a particular scene from a particular movie that you are thinking of let me know and I will see if I have it.