MPEG2 Encoder test

[lordsmurf]

Lightning is a fun one too.

[Sillyname]

Lightning is a fun one too.

Yeah... Strobes, however usually occur in an enclosed area and so will light up the entire room. When they are not there it is usually darker than any outside environment. Unless, of course, it is "a dark and stormy night"

[BJ_M]

a lot of our films we make have this sort of effect --either in cgi or "real"

[Sillyname]

a lot of our films we make have this sort of effect --either in cgi or "real"

Right! Hmmm... It'd be neat if someone built an actual encoder stress video. Like a camera test pattern or an audio setup CD for home theaters. One with undulating lines and single frame occurences. A DV file that fits on one CD for distribution...

[BJ_M]

well the sample i posted here is a pretty good micro stress test -- as it is an real explosion of fire with massive particle effects -- i have so other good ones like waterfalls shot with 70mm film and lots of other explosions and lighting strikes -- lots of cgi and live shots ... jumpback fire samples are also good -- though they would not like them to be used for re-distribution in whole or part on a public forum...

[BJ_M]

single frame occurrences do not really mimic the content found in most mpeg and wouldn't really be a fair test -- there would be no vectorization data

[Skynet107]

Great test! Very useful. I will have to study the images when I am not so tired. It bears repeating that seeing the result in motion is a whole other ball of wax so view the images with this in mind.

[Sillyname]

single frame occurrences do not really mimic the content found in most mpeg and wouldn't really be a fair test -- there would be no vectorization data

But what about The Exorcist? :P

[NewHomebrew]

hmmmm.... procoder looks the best. there isn't any of that strange swirly noise like in some of the others. i am kinda pissed... i have Vegas and will probably end up using another encoder now.

[FredThompson]

And remember that the flaws are easier spotted in PAL because it's got better everything than NTSC: better resolution, better colors, better contrast ratio, etc., so all the MPEG2 crap will be seen there for sure, while with NTSC various MPEG2 encoders will seem to have less vatiation in quality. At least to your naked eye.

Wrong.

compare actual vertical chroma resolution of NTSC DV with PAL DV. Which is greater? NTSC DV has a true vertical chroma resolution of 480 lines. Compare that with PAL's vertical chroma resolution.

better "everything"? Really? How about frame rate? Strike two for PAL DV.

[Poplar]

Haven't been here for quite a while. Great tests.

I am planning on doing a project and did some test of my own last night before I read this thread. The encoders I used were MC 1.42 and Procoder 1.5. I didn't do frame by frame comparison, but the impression I got was Procoder was a little better. This thread kind of confirmed it. Procoder used to painfully slow, now I have a 3.2 GHz P4-HT, I think I'll let it do the heavy lifting...

[vidnew]

This was a nice MPEG2 Encoder Test, but if I may ask.
I would like to see TMPGEnc 3.0 XPress and QuEnc (NuEnc) in the test too, seeing how these are also popular choices.

Thanx

[BJ_M]

TMPGenc 3 i would have to buy ,and i dont want to as 2.5x works perfectly well.

quEnc i dont use ...

[troyvcd1]

It is all kind of subjective but I thing CCE wins this hands down. It seems closer to the original image and that is what an encoder is supposed to do. If I need to soften the image then I can add some avisynth filters to do this. Plus CCE is a lot faster.

[troyvcd1]

When comparing CCE versions there was a big long thread at doom where the encoder versions were compared. There was a big controversy about how CCE 2.5 was looking better than later versions. It turned out that the people doing the test were using illigit versions of CCE to do the test. When the smoke cleared and legitimate CCE versions were used the later CCE versions won.

[BJ_M]

i am using legal version, including and up to 2.7.01.03

[Alan69]

It is all kind of subjective but I thing CCE wins this hands down. It seems closer to the original image and that is what an encoder is supposed to do. If I need to soften the image then I can add some avisynth filters to do this. Plus CCE is a lot faster.

Hear, hear! I think some are forgetting that that 'added noise' CCE is causing by keeping the detail can also be blurred through TIME not just by blurring a frame. Much of the work of MPEG encoding and decoding are done on the time axis, to judge everything on only frame by frame analysis is to determine which one makes better stills for a certain scene, not necessarily which made a better MPEG.

MPEG is not a series of frames in how it is encoded. It's a solid block of thousands of frames that are slowly pushed through your monitor, you can only see one slice at a time. But much of the compression works on diagonals in the X, Y, and Time axes, to judge total quality on only an X,Y frame is to totally bypass what the encoder is doing on the time axis. Encoders that use sophisticated techniques in the time axis will lose since you're ignoring it, even if their total scene output is actually better.

Black/white transitions through time will flicker. But small, compensated changes are invisible, basically it is blurring through time instead of X or Y. No way to be totally accurate judging by one stopped frame when it's supposed to be temporally blurred with the one ahead and behind, and the encoder used that fact to it's advantage in its encoding.

Since most of the compression occurs through the time axis, how well it is handled can actually be a bigger factor than what happens in X and Y. This isn't just bitrate and motion, many effects that can be done in X, Y can also be done in time when you view the MPEG as a 3D block of pixels. Time is your Z axis, it's as important or more so than the X and Y, so to treat it as less important by judging only by a single frame is not a proper test.

Alan

[BJ_M]

it is a good rough test, but very informative ... at some point i may post complete clips , but last time i did that it here took heavy bandwidth (600 gig/month) ..

examination of a singe frame can tell a lot about mpeg quality if you know what to look for ..

[hrlslcbr]

...including and up to 2.7.01.03

Can you try encoding with that version? I want to know if the new 2.70 engine is better than the old one.

[Alan69]

it is a good rough test, but very informative ... at some point i may post complete clips , but last time i did that it here took heavy bandwidth (600 gig/month) ..

examination of a singe frame can tell a lot about mpeg quality if you know what to look for ..

Does a circle tell you whether something is really just a 2D circle, or a slice of a cylinder, or a slice of a sphere? The end objects are very different in 3D, yet your 2D slice looks exactly the same. It is precisely the same type of 2D snapshot of a 3D medium, and tells you absolutely nothing about what's going on in the third dimension. And MPEG's third dimension can be made more complex than I've seen most people on this board ever give it credit for being. It is not just bitrate and motion, other transforms can be done through that dimension by the encoder.

It only tells you something about how that frame looks stopped, and since part of how a frame really appears to your vision system in playback is how you blend it with the one before and after, that isn't so much. This is well understood, and can be programmed for in a good encoder. You guys don't just think things like "psychovisual enhancements" in DIVX operate only on individual frame pictures do you? Surely some are working predominately in the time axis.

Does that extra noise actually show up as extra noise? Or is it part of the encoding, and it's used to keep the extra detail and then blended out along the time axis? You can't possibly tell since there's no time axis info in a frame. Any 2D slice through a 3D object can be very misleading about that object, this is no exception. If any of these encoders are doing even reasonably easy things in that third dimension then a frame test simply can't be all that.

And I do know what to look for, why did you guys leave out such an important part and talk like these pictures are even relatively definitive by themselves? The conclusions reached from just them can actually be quite wrong if any of these encoders are working on the third axis. Can't say for sure if they are, but these things are discussed in the MPEG specs, so I'd be very surprised if the better ones don't do quite a bit on the time axis frame to frame. I must say I do not see websites with proper pictures of an MPEG as a solid block, showing how things happen on the diagonals for time and motion as I see it, but I haven't looked that hard either really. Advanced ideas are in the specs though, so obviously someone knows what they're doing and sees it that way, it's where I picked it up.

Don't think huge files are needed really. Actually I think 5 frames looped might be the best to see it anyway, blending on time axis has to be done in short order or it'll show up visibly. Two same 5 or 10 frame segments looping side by side in sync would be a far truer comparison and let you see if one is keeping more detail while letting your eye/brain blend the individual frame distortions if they're doing it. Still would have to test across high, medium, and low bitrates, high and low contrast, high and low motion, and high and low detail to start making any type of broad judgements about which are truly better encoders and which are worse. MPEG encoding is complex, so only a reasonably complex test would give any solid indications to say one encoder is better than another in the specific sense. You can say generally by looking at a few movies at similar bitrates, but even then one encoder may still beat out in one aspect while being generally poorer in others, all depends on specifics in the encoders.

Soon as you start seeing some faring worse than expected by general opinion of how good they are, that should throw up immediate red flags that the thing left out in this test, time, may be way more important for some than others. The frames are not encoded as single stop motion pictures, they are looked at as a block with the frames ahead and behind. Stop a mediocre videotape. Frame will look like crap. The video isn't great, but looks much better than a single frame. The random noise blends. In an MPEG, that 'noise' may not even be random. Working with the frames ahead and behind, you can easily keep detail in the X and Y axis, still compress and allow the encoding distortions in the X and Y axis to keep that detail, and blur the extra distortions through the time axis.

If you allow say 5% per frame error for a 5% frame improvement in detail, make sure your 5% detail lines up and is visible, make sure your 5% noise doesn't and blends out to say a distributed 2% visual error over time, then you've made a 3% net improvement in picture quality by working through time. And any individual frame will look up to 3% worse than it actually is in context. How can you say your test is even reasonably indicative when the errors you're judging by are quite possibly less than the error you've intentionally caused by stopping the motion? I can see these ideas just by having glanced through the specs a time or two and being a programmer, I'm quite sure the people writing the specs and better MPEG encoders are well aware of them.

Don't get me wrong it is something easy to look at and judge by, but you still have to cross check it by seeing the same frames in motion to be sure any judgements made are accurate even for those frames, much less other contexts in a video. Without that check the possible margin for error of this poll of individual stopped frames is greater than the margin of victory between the frames, so not conclusive.

[BJ_M]

since i work in the film business, a bad frame is a bad frame is a bad frame .... no mater how you want to justify it ..

the pics above should be treated as well, the same .....

when we produce a commercial dvd (and i have done 1000's) , we check them frame by frame for errors, same as film ... we also play them of course ....

i made no conclusions -- that is up to the reader ...

[BJ_M]

if you don't like the test - don't use it !! :

[chicola]

I completely agree with Alan69, but I would extend the 5-10 frame segments to 1 or 2 seconds of video sequence.

IMHO, better than obtain a "super slo-mo" comparison is to obtain "almost realtime" feeling.

And, one would be stretching the 3rd dimension and, thus, having more different transitions.

[vidnew]

TMPGenc 3 i would have to buy ,and i dont want to as 2.5x works perfectly well.

quEnc i dont use ...

I understand your standing on TMPGEnc 3.0 XPress I wouldnt either pay for something that I was just going to use in a test. But why I was thinking about it was because when new people starts with encoding they know the name TMPGEnc and when they go to buy it they see 3.0 XPress as the newest version and buy that one. And once they buy it they arent likely to go professional and buy expensive software as CCE SP 2.7 for $2000 they will stick to that one because they once payed money for it. Therefore I belive we are going to see more of that encoder in the future. And I dont belive there are going to be many updates on the old TMPGEnc 2.5 version. I know my logic may be flawed, but I belive we should start taking in account TMPGEnc 3.0 XPress. Anyway seeing as you have enough encoders TMPGEnc 3.0 XPress probably isnt something you need

But about the QuEnc, it is the only FREE MPEG2 encoder which is enough to put it on the test list. Seeing how again this is going to be used in many free DVD packs. And many people will use only this one as they see no need to throw money on something they can get for free. And again this is deveoped by people on their free time, not by people doing a job and getting payed, and anyone can paticipate which brings many ideas and solutions. Thats why it would be interesting to see how it performs compared to the other commercial encoders.

[BJ_M]

you make a case for QuEnc , makes sense ...

I have many other encoders also - several hardware high end encoders and several more software ones .. i just stuck to what is popular here ..

[vidnew]

Yes, but we have to remember QuEnc is only beta and looking at NuEnc we can see that many features are implented and will be submitted to be implented in QuEnc when they are finished. So I belive that now feature-wise QuEnc cant measure up against the commercial encoders, but I wonder how it does on quality and speed. Cant wait to see the result after QuEnc goes final against the newest encoders of that time. Its going to be exiting, seeing if some made without gainig any money can beat something commercial

Of course the commercial encoders are going to be used by professionals because they are more steady regaring updates, but for amateures I hope its going to be a good solution.

[Poplar]

Just when I was about to jump on Procoder bandwagon, it did some thing very strange. I encoded a 40-minute analog capture last night and found out that Procoder produced several garbage frames. I say garbage because that's what they are, hardly viewable and everything was screwed up in the frame. I checked the source frames and they looked all right. I have been using CCE and MCE forever and they never did something like this. Can someone explain it for me?

[erratic]

you make a case for QuEnc , makes sense ...

QuEnc 0.56 [Alpha] is availbale.
http://forum.doom9.org/showthread.php?&threadid=84857
Now it has audio encoding support (MP2/AC3). Still requires Avisynth but can convert frameserved VirtualDub files if you change the extension to avi (it can also convert regular avi files now). I tried it with a .vdr file but the resulting mpeg2 video was garbled, probably because it wasn't converted to YV12. Problem was solved when I changed the vdr extension to avi.

[cosmin]

And remember that the flaws are easier spotted in PAL because it's got better everything than NTSC: better resolution, better colors, better contrast ratio, etc., so all the MPEG2 crap will be seen there for sure, while with NTSC various MPEG2 encoders will seem to have less vatiation in quality. At least to your naked eye.

Wrong.

Oh, no, it's quite right. PAL DV is better than NTSC DV.

compare actual vertical chroma resolution of NTSC DV with PAL DV. Which is greater? NTSC DV has a true vertical chroma resolution of 480 lines. Compare that with PAL's vertical chroma resolution.

NTSC DV uses chroma sampling 4:1:1. PAL DV uses chroma sampling 4:2:0, and so does MPEG2 for DVD and HDTV. Chroma in NTSC is very unbalanced, meaning that it's very bad horizontally and very good vertically. Chroma in PAL is well-balanced (same quality) across both dimensions. Downsampling 2 times in PAL is acceptable (although not perfect), but downsampling 4 times shows color bleeding even on a low-end monitor.

Not to mention that conversion from NTSC DV to MPEG2 is doomed, due to the piping of 4:1:1 into 4:2:0. It gives you the worst of both worlds, a horrid "4:1:0" in which chroma is downsampled 4 times horizontally and 2 times vertically.

How about frame rate? Strike two for PAL DV.

That's irrelevant in the current discussion, when comparing MPEG encoders on a frame-by-frame basis.

[Edmund Blackadder]

And remember that the flaws are easier spotted in PAL because it's got better everything than NTSC: better resolution, better colors, better contrast ratio, etc., so all the MPEG2 crap will be seen there for sure, while with NTSC various MPEG2 encoders will seem to have less vatiation in quality. At least to your naked eye.

Wrong.

Oh, no, it's quite right. PAL DV is better than NTSC DV.

compare actual vertical chroma resolution of NTSC DV with PAL DV. Which is greater? NTSC DV has a true vertical chroma resolution of 480 lines. Compare that with PAL's vertical chroma resolution.

NTSC DV uses chroma sampling 4:1:1. PAL DV uses chroma sampling 4:2:0, and so does MPEG2 for DVD and HDTV. Chroma in NTSC is very unbalanced, meaning that it's very bad horizontally and very good vertically. Chroma in PAL is well-balanced (same quality) across both dimensions. Downsampling 2 times in PAL is acceptable (although not perfect), but downsampling 4 times shows color bleeding even on a low-end monitor.

Not to mention that conversion from NTSC DV to MPEG2 is doomed, due to the piping of 4:1:1 into 4:2:0. It gives you the worst of both worlds, a horrid "4:1:0" in which chroma is downsampled 4 times horizontally and 2 times vertically.

Thank you cosmin. Obviously, either FredThompson has never seen a true PAL DV output on high quality CRT monitor, or simply has got some patriotic thing about NTSC. I work with multi-system equipment all the time and the difference between PAL and NTSC is striking, with PAL being oh so much better. And yes, colors, especially reds are superior in PAL DV versus NTSC DV. Just open the DV file in Windows Media Player 6.4 or earlier (not 7 or later) and see how reds are more uniform in PAL DV file than in NTSC DV. There are still chroma sampling squares in PAL DV, but not as ugly as in NTSC DV.

I actually compared the footage taken from NTSC version of Sony PD-150 (theoretically better camera) to PAL version of Sony DCR-TRV900E (theoretically not as good as PD-150). I looked at it on various multi-system CRT televisions and also on our Panasonic TH-42PWD6UY 42inch EDTV plasma display. On every display, be that CRT or plasma, PAL TRV900E image beat the crap out of NTSC PD-150 image. Everything was better in PAL DV - contrast, colors, resolution. Yes, even on 480P plasma display you can see the advantage of PAL resolution. I even made sample DVD's from these footages (using ProCoder naturally), connected to TH-42PWD6UY through component connection - and PAL still beat NTSC. Those were two different cameras, so to be fair I did the same test with NTSC DCR-VX1000 and PAL DCR-VX1000E. Well, PAL footage quality from VX1000E was so much better that I could hardly believe that it was the same camera model. And once I put Optex 16:9 Anamorphic Lens onto my PAL cameras and watched that footage on Panasonic TH-42PWD6UY, that was honestly the best Standard Definition footage I've ever seen.

As to frame rate being 25 frames (or 50 fields), I don't see why so many people complain, especially about the flicker. For some reason nobody seems to complain when they go to the movies and see 24fps flicker. To me if I watch a film in PAL, besides having a higher quality picture, it also gives me an impression of the movie theater because of the similar amount of flicker .

And another note, now about ATSC North American Hi-Def format. How come it seems that everything shot in highly-praised 24P format still has to be shown at 29.97fps with a pulldown? What about displaying the true 24 frames per second? I guess somebody chickened out when they were setting standards. So to me PAL based HDTV showing films will still look better than ATSC, even if the resolution is going to be equal for both camps - 25fps is closer in motion perception to 24fps than 29.97 with pulldown. Audio pitch correction (4%) is easy to do, so that's not an issue.

So, to conclude, FredThompson you're wrong. PAL is better than NTSC, be that DV, DVD or any other Standard Definition format. Get yourself a quality PAL equipment and see for yourself (no PAL->NTSC converters please).
And the last, a lot of North Americans - (I witnessed it myself, while demonstrating PAL recordings) - when they see PAL for the first time in their lives, they think that it's actually a Hi-Definition signal