Any harm in using ConvertToRGB24() by default?

[jagabo]

Seems convincing enough to me! Thanks for going to all that effort.

Yeah, it is pretty convincing. All the behavior is consistent with converting YUV to RGB and back again. But it's possible that it is doing something else that is screwing up the intensities.

In any case, I've been wanting that test pattern for a long time so I didn't mind spending the time to put it together. And I learned a little more about how to get the best results out of TMPGEnc (although I don't use it much anymore).

I didn't mention it earlier but I also verified that AviSynth's ConvertToRGB()/ConvertToYUY2() pair is much better at retaining accuracy between luma 16 and luma 235. Starting with a YV12 or YUY2 video, using ConvertToRGB() followed by ConvertToYUY2() or ConvertToYV12() retains all 240 intensities between luma 16 and 235. Below 15 and above 235 are still clamped of course.

[Jim44]

In any case, I've been wanting that test pattern for a long time so I didn't mind spending the time to put it together. And I learned a little more about how to get the best results out of TMPGEnc (although I don't use it much anymore).

Could you share how you made those test patterns, or the patterns themselves? Nice work, and since TMPGEnc Plus is my main (only) encoder at the moment I'd like to make sure I'm getting the most out of it, particularly for the DV-AVI stuff I encode (usually using AviSynth).

Just when I thought I had a handle on this stuff......

Thanks.

[jagabo]

Could you share how you made those test patterns, or the patterns themselves?

I wrote a program to make a simple 8 bit binary file. I imported that into Ulead PhotoImpact as gray scale data and saved it as an uncompressed 24 bit BMP. The BMP was imported into VirtualDub to make an uncompressed RGB AVI file. That was imported into TMPGEnc and saved as an MPG file (basic YCbCr enabled). RGB and MPG files attached:

blocks720x480.zip

[DRP]

It's good enough for me too! The next question I have is can we determine that TMPGEnc converts to RGB24 as opposed to RGB32? I have read somewhere that it converts internally to RGB24 only. Think I saw it over on doom9 somewhere. If true, then in the name of performing as few colour space conversions as possible, I think it would be best to use the ConvertToRGB24(matrix="PC.601") command in my AviSynth script as opposed to ConvertToRGB(matrix="PC.601"). That way avoiding the possible TMPGEnc internal conversion from RGB32 to RGB24 before it does the encoding.

Comments?

[jagabo]

RGB32 has the same precision as RGB24 -- 8 bits per color component. It just has an extra 8 bits of padding so that each pixel is one long word for easier handling CPUs with 32 bit registers. The extra 8 bits can also be used for an alpha channel.

I don't know if TMPGEnc uses RGB32 or RGB24 internally. Converting to the right one should be a little faster since TMPGEnc won't have to convert again. There shouldn't be any losses from converting one to the other though.

[DRP]

I've read in many places on doom9 that TMPGEnc does convert internally to RGB24 for internal processing. If it gets faced with an RGB32 source, it just strips the alpha channel off it. In any case, I've just tested it and I can't use RGB24 anyway because Donald Graft's MSharpen filter (which I want to use) will only accept the 32 bit flavour of RGB as an input anyway, so that's that.

[Jim44]

Thanks jagabo!

[DRP]

Well, I've done some testing of the new encoding settings now and yes, there is a large improvement. Colours are much brighter with much greater contrast range. Sharpness is vastly better too with using MSharpen. Only downside is a much slower encoding speed. About half as fast as using TMPGEnc's default internal sharpener filter. It does seem that MSharpen is quite slow. Oh well, can't get something for nothing I guess. Even with that downside, the improvement in picture quality is worth it. Thanks for the tutorial jagabo

[manono]

You could always use a better and much faster encoder, HCEnc, and a better and faster sharpener, LimitedSharpen(Faster). No need for that colorspace changing-back-and-forth nonsense either, if the source is DVD or DivX/XviD AVI. The benefits far outweigh your questionable preference for cropping pixels by Mod1. And it's easy to crop Mod1 by combining the crop and resize, anyway.

http://avisynth.org/mediawiki/LimitedSharpen_Modded
http://avisynth.org/mediawiki/LimitedSharpen

[jagabo]

And it's easy to crop Mod1 by combining the crop and resize, anyway.

No, you can't crop YV12 mod1. The minimum is mod2. YUY2 can be cropped mod1 on the vertical axis, but only mod2 on the horizontal axis.

I guess you could double the size, crop mod2, addborders, and resize back down -- that would be the equivalent of cropping mod1. But it's a waste of time and you'll mess up the image with all the resizing.

[manono]

I just tested this with a DVD source:

Lanczos4Resize(720,480,5,41,0,-39)

It opens in VDubMod. Did I do something wrong? Understand that I don't crop Mod1 myself, but was only trying to find a workaround for DRP.

[DRP]

You could always use a better and much faster encoder, HCEnc, and a better and faster sharpener, LimitedSharpen(Faster). No need for that colorspace changing-back-and-forth nonsense either, if the source is DVD or DivX/XviD AVI. The benefits far outweigh your questionable preference for cropping pixels by Mod1. And it's easy to crop Mod1 by combining the crop and resize, anyway.

http://avisynth.org/mediawiki/LimitedSharpen_Modded
http://avisynth.org/mediawiki/LimitedSharpen

You need to know some more things about what I'm doing which I have left out so far (for clarity) before I can consider doing as you suggest.

I am watching on a CRT television.
I letterbox everything I encode to allow for CRT overscan
I always crop down to 16:9 AR (at the widest) for watching regardless of the source AR (this is done because it is the smallest size picture I can tolerate watching on my screen)
I always convert to PAL whenever I possibly can. This is at least 95% of the time.

With those non-negotiable restrictions present in the way I choose to do things, the vertical resolution of my encodes is almost always 403 pixels. An odd number you will notice. I cannot resize to a vertical resolution that is not divisble by 2, so that in itself necessitates all the colour space conversions. Thank you for your concern and recommendations anyway.

[DRP]

You could always use a better and much faster encoder, HCEnc

Thanks, but that isn't an option for me. I still want the option of using VirtualDub as my frameserver and that encoder doesn't appear to include that as one of its options. I need VDub for the logoaway filter I like to use and I'm not willing to learn the intricacies of a second MPEG encoder so I can use one for some tasks and another for different jobs. That's just not how I roll.

[jagabo]

I just tested this with a DVD source:

Lanczos4Resize(720,480,5,41,0,-39)

It opens in VDubMod. Did I do something wrong?

You're right! I usually use Crop() which doesn't allow odd values. I didn't realize the resize filters do.

A note to the OP: cropping YUY2 or YV12 at odd boundaries will lead to significant blurring of colors:

That was done to a 720x480 source using the LanczosResize(716, 480, 2, 0, -2, -0) on the top and LanczosResize(716, 480, 1, 0, -3, -0) on the bottom, on a YV12 source. Using BilinearResize() instead was a little better. Of course, this source has unusually sharp edges which makes it much more noticeable. And note, no resizing was performed, just cropping. Converting to RGB first eliminated this problem.

[DRP]

Cool, thanks for that jagabo. You keep coming up with all the answers and graphics to back them up! I too just assumed that Crop + Lanczos4Resize would be the same as incorporating the cropping into the Lanczos4Resize function call. That's interesting to know, but makes no difference for what I'm doing. Reason being that I want to perform an MSharpen function in between the Crop and the Resizing, so I need to keep them separated.

In testing I have found that the MSharpen takes longer working on a resized image rather than the original source. This is probably because the area involved is larger rather than anything to do with image complexity. I can always get the horizontal resolution very close to original, but converting to PAL usually results in upsizing the vertical resolution by about 50%. MSharpen is quite slow at the best of times anyway, so every little bit helps.

When all's said and done though, the info provided in this thread has made a dramatic improvement in my encodes. I swear I can not tell the difference between an XSVCD made from an XviD source and a retail DVD it is that good. In fact, in some cases I would actually go so far as to say the XSVCD looks better! Richer colour and sharper image. I had no idea it was possible to get this much quality out of plain old-school SVCD. Well, not really "plain" as such. They are 544x576 resolution SVCD, so not exactly standard as such.

Anyway, thanks very much.

One last thing... if anyone has the skillz to recompile the source code for a .vdf VirtualDub filter, then it would really help me out a lot if you could change the default values and recompile the David A Graft MSharpen VDub filter for me.

[poisondeathray]

My head hurts from reading this thread, but there is a lot of good information in here.

@DRP, you can get Msharpen for Vdub from Mr. Graft himself
http://neuron2.net/msharpen/msharpen-vdub.html

EDIT: sorry you wanted it *recompiled* DUH...my head hurts from reading this thread...

[FulciLives]

You could always use a better and much faster encoder, HCEnc, and a better and faster sharpener, LimitedSharpen(Faster). No need for that colorspace changing-back-and-forth nonsense either, if the source is DVD or DivX/XviD AVI. The benefits far outweigh your questionable preference for cropping pixels by Mod1. And it's easy to crop Mod1 by combining the crop and resize, anyway.

http://avisynth.org/mediawiki/LimitedSharpen_Modded
http://avisynth.org/mediawiki/LimitedSharpen

You need to know some more things about what I'm doing which I have left out so far (for clarity) before I can consider doing as you suggest.

I am watching on a CRT television.
I letterbox everything I encode to allow for CRT overscan
I always crop down to 16:9 AR (at the widest) for watching regardless of the source AR (this is done because it is the smallest size picture I can tolerate watching on my screen)
I always convert to PAL whenever I possibly can. This is at least 95% of the time.

With those non-negotiable restrictions present in the way I choose to do things, the vertical resolution of my encodes is almost always 403 pixels. An odd number you will notice. I cannot resize to a vertical resolution that is not divisble by 2, so that in itself necessitates all the colour space conversions. Thank you for your concern and recommendations anyway.

I read this and instantly thought: "insanity"

You hardly make any sense what-so-ever with your 16:9 AR comment and the 403 pixel comment and "need" to do mod1 cropping is all very dubious at best.

I just call them as I see them. I just roll like that.

- John "FulciLives" Coleman

[DRP]

You hardly make any sense what-so-ever with your 16:9 AR comment and the 403 pixel comment and "need" to do mod1 cropping

Really? Horizontal resolutions I variously use are 480, 528, 544, 704 & 720. Respectively I use the following letterboxed image resolutions to allow for CRT overscan: 448, 492, 508, 656 & 672.

Under PAL and using a source aspect ratio of 16:9, the vertical image resolution required to match those horizontal ones is always 403 pixels (rounded to the nearest). Some come out close to 402.5 and others closer to 403.5, but the rounded integer result is always 403. Do the maths yourself, I have.

Granted, in NTSC-land, things become a bit easier, because the vertical resolution works out to be 336 instead which is a much nicer number being divisible by 16, but as stated earlier, I don't like NTSC. Even using an odd number like 403 pixels for the height, the advantage of PAL over NTSC far outweighs any downside to using a 403 vertical image resolution.

[jagabo]

In that situation (at least the wider ones) I'd probably go with 400 lines and letterbox with 80 lines on top and 96 on the bottom to keep the picture aligned on 16 pixel boundaries.

[DRP]

I've just converted a DVD movie using these new settings (the whole ConvertToRGB(matrix="PC.601") + basic YCbCr enabled) bizzo with TMPGEnc. The strange thing is though that when watched on a CRT TV the black is less than absolutely black. During the credits beginning and end you can see that the letterboxed picture area is very dark blue rather than absolutely black. This contrasts with the original DVD when played, which has no visible border between the black of the letterboxed area and the black of the letterboxing created by the DVD player (it is a 16:9 transfer DVD).

Why did this happen? I thought these colour space settings were to maintain full colour space so that jet blacks would remain jet black even when watched on a CRT TV. I don't understand where I've gone wrong here.

Certainly, when converting downloaded AVI files, the contrast is very definitely much greater. Why is it different when encoding a DVD? The reason for encoding the DVD was to change the aspect ratio.

[jagabo]

If you did everything correctly a shift in black level would not occur. Did you select the "Output YUV data as Basic YCbCr not CCIR601" option in TMPGEnc? If you didn't do that TMPGEnc would compress the luma range when it converted back to YV12. That would explain your black level getting lighter.

[edDV]

Here's the basis for all of this:

The darkest darks in digital video should be at luma=16. The brightest whites should be at luma=235. But on computers the blackest blacks are at RGB=0 and brightest whites at RGB=255. So normally, for viewing YUV video on a computer you expand the contrast so that luma 16 and everything below it become RGB=0, luma 235 and everything above it go to RGB=255. The 220 values between 16 and 235 are spread out between 0 and 255.

Conversely when taking RGB images and converting to YUV video you want to compress the RGB range so that the 256 different RGB levels fall between luma 16 and 235.

Both of those conversions can lose accuracy.

But you are recompressing one YUV format to another YUV format. You're just using RGB as an intermediate state, it doesn't matter how the RGB looks on the monitor during the recompression. So you can avoid the contrast expansion/reduction cycle to retain more information and avoid losing accuracy.

ITU-Rec-601 16-235 luminance scaling for YUV video maintains a safety or overshoot range. This is very important for analog capture since NTSC and PAL specify up to 20% overshoot (120 IRE clipping) for analog recording or transmission. The 236-255 digital overshoot range still results in clipping above analog ~108 IRE*. Clipping is to be avoided if possible because it generates out of band harmonics (ringing and digital conversion noise). Likewise low analog black levels are accommodated by analog NTSC or PAL to prevent black clipping thus allowing for downstream black restoration.

While all this is important for recording and transmission of high quality video, typical home video suffers from similar or greater artifacting from high digital compression, poor analog low pass filtering and gamma mismatches.

Also, the cable and satellite distribution industries have avoided use of the overshoot ranges except for levels correcting along the transmission path. "Compressing" luminance excursions to 16-235 allowed FM transponder bandwidth reduction for satellite links. Thus the signal reaching the home usually lacks information in the 0-15 or 236-255 ranges except where levels are off.

Thus levels corrected cable/sat source can be recorded stretched with 16-235 -> 0-255 conversion but any levels errors will clip black or white.

The 0-255 stretch is not appropriate for direct analog recording (without proc amp) or for digital stream transfer from consumer camcorders. Consumer camcorder gray scale covers the full 16-255 range. These sources contain essential information in the 236-255 range that will be clipped off with 16-235 to 0-255 conversion.


* The +/- 20% overshoot range for composite NTSC and PAL includes both luma and chroma. ITU-Rec-601 is a component YCbCr standard. It was determined that 8% overshoot management was adequate for luminance only.

[DRP]

If you did everything correctly a shift in black level would not occur. Did you select the "Output YUV data as Basic YCbCr not CCIR601" option in TMPGEnc? If you didn't do that TMPGEnc would compress the luma range when it converted back to YV12. That would explain your black level getting lighter.

Yes, I did check my settings and the Output YUV as basic YCbCr setting was checked. Anyway, it's not a big drama, just something I noticed is all. I've since seen the same effect on some SVCD I made of downloaded AVI files too, so my claim that it only happened while converting a DVD was false.

I'm sure I've got all the settings correct. I don't know why the blackest of blacks is not quite absolutely black when viewed on a CRT television, but it's only noticeable for a few seconds during the credits at either end, so it isn't a big drama. The rest of the video in between is markedly better than before, so I guess this is just how it's supposed to be.

[jagabo]

I'm sure I've got all the settings correct. I don't know why the blackest of blacks is not quite absolutely black when viewed on a CRT television, but it's only noticeable for a few seconds during the credits at either end, so it isn't a big drama. The rest of the video in between is markedly better than before, so I guess this is just how it's supposed to be.

No, something isn't right. If you convert to RGB without luma expansion and convert back to YV12 without luma compression the black level before and after should be identical (aside from some minor rounding errors which wouldn't be visible). Unless, of course, you did some other processing on the video that changed the luma levels.

[DRP]

These are the only processing lines I perform:

ConvertToRGB(matrix="PC.601")
Crop(95,76,-96,-77)
MSharpen(strength=80)
Lanczos4Resize(508,403)
AddBorders(18,86,18,87)

After that, it's sent to TMPGEnc. I use none of the filters at all within TMPGEnc and I have the 'Output YUV as basic YCbCr' setting checked.

The source file is a DGIndex frameserver project file of the ripped VOBs extracted using the 'PC scale' setting within DGIndex (that's the default setting) for YUV->RGB conversion

[jagabo]

It looks like you're doing things properly. Try using the AviSynth filter VideoScope() to view the levels before and after your conversion.

The YUV->RGB conversion setting in DgIndex only effects its display, not how the MPEG data is processed with Mpeg2Source() in AviSynth.

[DRP]

Okay I tried that but I don't understand what the output is telling me. I've noticed this dark blue effect during credits ona few more encodes I've done recently now. I wish I could take a screen capture of it to show, but I can't because the effect is only visible when viewed on a CRT TV. On the computer, everything looks perfect.

Are you sure I should be checking the 'Output YUV as basic YCbCr' setting in TMPGEnc when my intended viewing device is a conventional CRT TV?

I admit that I don't really understand why this is happening because I create the black borders in my encoding process and they are black. They're not created by the DVD player on playback. The black borders are actually part of the encoded MPEG file. That's why I don't get why I'm seeing a difference between the black of the picture area (when it's supposed to be black) and the black created by the MPEG encoder as intentional letterbox bordering. Surely black is black whether it's created by the encoder or part of the source picture.

I can only guess that all the colour conversions I end up having to do are inducing errors that are showing up and being visible in certain circumstances.

[jagabo]

Are you sure I should be checking the 'Output YUV as basic YCbCr' setting in TMPGEnc when my intended viewing device is a conventional CRT TV?

Yes.

Okay I tried that but I don't understand what the output is telling me. I've noticed this dark blue effect during credits ona few more encodes I've done recently now. I wish I could take a screen capture of it to show, but I can't because the effect is only visible when viewed on a CRT TV. On the computer, everything looks perfect.

I admit that I don't really understand why this is happening because I create the black borders in my encoding process and they are black. They're not created by the DVD player on playback. The black borders are actually part of the encoded MPEG file. That's why I don't get why I'm seeing a difference between the black of the picture area (when it's supposed to be black) and the black created by the MPEG encoder as intentional letterbox bordering. Surely black is black whether it's created by the encoder or part of the source picture.

I can only guess that all the colour conversions I end up having to do are inducing errors that are showing up and being visible in certain circumstances.

I think your TV isn't adjusted correctly.

Working the way you are the blackest blacks in the image (aside from a little overshoot) should be R=G=B=16. If you are adding a black border of R=G=B=0 then you are adding a blacker than black border. You should NOT be able to see a difference between those two colors if your TV is adjusted correctly.

Download the Belle Nuit chart and make yourself a DVD from it:

http://www.belle-nuit.com/testchart.html

It's full range 0-255 RGB so you should use the 'Output YUV as basic YCbCr' option in TMPGEnc Plus. Of the dark areas marked 20, 12, 4, and 0 you should only be able to see a difference be 20 and 12. The areas marked 12, 4, and 0 should all be totally black.

You can use AviSynth's internal ColorBars() command to generate a chart with a black level test. See the AviSynth docs.

Or buy something like tha Avia video calibration DVD and use its tools:

http://www.amazon.com/Avia-II-Guide-Home-Theater/dp/B000X4NJNS/

[DRP]

Okay, I've just figured out what I'm doing wrong. It's the (matrix="PC.601") bit that is causing the lighter than normal blackness. I've done a test encode on a 30 second bit of video where the problem is visible and done it 3 different ways. The first was with my normal AviSynth script using ConvertToRGB(matrix="PC.601"), the second was the same only using just ConvertToRGB and the third was using VDub as the frameserver instead.

The last two look perfect with no visible border between the picture and the black letterboxing border. The first has the picture being brighter than the black letterboxing border which is visible in dar scenes like the credits.

So then using the (matrix="PC.601") is bot the correct thing to do when your source material is already scaled to be suitable for viewing on a CRT TV such as DVD/downloaded AVI? That is my conclusion anyway.

[jagabo]

Once again: ConvertToRGB(Matrix="PC.601") is the right thing to do if you are using TMPGEnc's 'Output YUV as basic YCbCr' option. The problem is your TV (or DVD player) is maladjusted. I explained why you see a difference in the added black border already. If you want to add a black border you should use 16,16,16 as the border color.

VOB from DVD:


After ConvertToRGB(matrix="PC.601") and MPEG compression with TMPGEnc Plus with 'Output YUV as basic YCbCr':


Black and white levels identical.

The black of space in the video frame is about luma 20 (also RGB=20 in the images). It should appear nearly black on a properly adjusted TV. Note that it is probably not black on your computer monitor because the luma levels have not been adjusted for viewing on a computer. The black in the VideoScope background is luma 0. If you are adding black borders with AddBorders(...) while working in RGB that is what you are adding. If you see anything more than a barely perceptible difference between the luma 20 and luma 0 on your TV it (the TV) needs adjusting.

The VideoScope graph at the bottom of the images is a graph of brightness (vertically) vs pixel position (horizontally). It might be most easily understood by looking at a single line of pixels (actually 2 in this picture):



At the top of the image are two scanlines taken from the middle of the earlier images. The body of the graph shows the brightness of the pixels across the width of the image. The higher a point is on the graph the brighter that pixel is within the image above. The red arrows show how the bright spots of the image become peaks on the graph, the dark parts on the image become valleys.