how to fix levels if they exceed rec.601

[codemaster]

I have some mpeg 2 clips that I want to edit in a NLE. In order to edit these clips in Premiere Pro, first I have to decode them to lossless, using DGIndex, AviSynth, VirtualDub and Lagarith codec.

The mpeg 2 clips are generated by a consumer camcorder. I observed that some individuals that are specialists in video compression, or video editing, or other video processing tasks, are informing me (and others that want to edit video footage generated by their consumer camcorder), that SD consumer camcorders can not only exceed Rec.601, but they can also exceed RGB 0-255.

I would like to learn:

Q1. How to check if levels in a YUV clip exceed rec.601 and/or RGB 0-255 ?
Q2. If levels in a YUV clip exceed rec.601 and/or RGB 0-255, how do I correct it's levels so that they do not exceed rec.601 and/or RGB 0-255 ?

I will compress the decoded mpeg 2 clips with Lagarith in YV12 colorspace, to save hdd space. I understand that Premiere Pro works in YUV mode with DV clips, and in RGB mode with YV12 avi clips that were compressed with Lagarith.

I am still confused about what happens with levels when the imported avi clips are converted from YV12 to RGB by Premiere, and when the exported Premiere sequence is converted from RGB to YV12 by AviSynth:

Q3. After importing avi into premiere, levels YUV 16-235 are expanded to RGB 0-255 ?
Q4. After importing avi into premiere, levels YUV 16-235 are expanded to RGB 16-235 ?
Q5. After exporting the premiere sequence to avi, with Lagarith compression in RGB mode, and after converting that avi to YV12 with AviSynth with ConvertToYV12(matrix="Rec601"), levels will be contracted from 0-255 to 16-235, or clipped from 0-255 to 16-235 ?
Q6. After exporting the premiere sequence to avi, with Lagarith compression in RGB mode, and after converting that avi to YV12 with AviSynth with ConvertToYV12(matrix="PC.601"), levels will not be expanded nor contracted ?

And after I encode the avi to mp4 with h264 lossy compression, for delivery, I am confused about what happens with levels when that mp4 file is played back in MPC-HC, VLC, WMP, QuickTime:
Q7. If mp4 is YUV 16-235, levels are expanded to RGB 0-255 by the video player ?
Q8. If mp4 is YUV 16-235, levels are not expanded, but maintained at RGB 16-235 by the video player ?

What I am doing is I am following sanlyn's procedure and recommendations, and there is alot of confusion at first, since I don't have certain knowledge yet, to fully understand those recommendations, but I asked these questions because I don't want to make a mistake and unknowingly delete detail in the image, or cause the image to shift colors and tonality when it is imported in Premiere, and/or when it is played back on Windows and Mac.

Q9. Can you recommend books or documentation that could clear this confusion ?

Do not suggest to import the mpeg 2 clips in Premiere Pro. I already tried that, and found out that although Premiere accepts them, I had problems with performance and artifacts, because Premiere was not designed to work well with such highly compressed files, that are inter-frame/GOP/keyframe based. And on top of that, Premiere adds another generation of compression to those already highly compressed mpeg clips. Also, converting to DV and fixing levels in Premiere seems like an easier alternative, and I had no problems with performance or artifacts with DV, but I want to try Lagarith, because it uses lossless compression, and because the camera already compressed those mpeg 2 files for delivery, and maybe I can avoid another lossy compression to DV.

[themaster1]

You can use this to check illegal values (but is not practical to my eyes):

MPEG2Source("D:\MyVideo.d2v", cpu=0)
converttoyuy2(interlaced=true)
ColorYUV(analyze=true)
ColorYUV(off_y=-35, gain_y=150) # brighten it up to create illegal colors
diff = Subtract(last, ConvertToRGB(last,interlaced=true).ConvertToYUY2(i nterlaced=true)).Levels(112,1,144,0,255)
return(StackHorizontal(last, diff)).VideoScope("bottom")

You need the filter videoscope see below:
http://avisynth.org.ru/docs/english/externalfilters/vscope.htm



Otherwise you can use this simple script:

MPEG2Source("D:\MyVideo.d2v", cpu=0)
ColorYUV(analyze=true)

If you want stay in the safe ballpark for chroma values try to set it around 212 max.
Why 212? try this script and will understand

function int2mode(int index) { return Select(index, "classic", "levels", "color") } #for HISTOGRAM
colorbars(width=720,height=480,pixel_type="YV12"). trim(0,7000) # rec 601 colorbars
changefps(29.970)
ColorYUV(analyze=true)
Histogram(int2mode(1))

You have to keep some room so you can adjust the saturation on tv and hit the magic number "240"

[jagabo]

I use VideoScope() or Histogram() to check levels. To get the customary horizontal graph from Histogram() use TurnRight().Histogram().TurnLeft().

If you want to see the picture without the usual rec.601 contrast stretch use ConvertToRGB(matrix="PC.601"). You can then use CSamp to read RGB values off the screen.

[poisondeathray]

Q1. How to check if levels in a YUV clip exceed rec.601 and/or RGB 0-255 ?

In addition to the other suggestions, you can use premiere's waveform monitor if the asset is treated as YUV (see below) . If it's treated as RGB, it will already be clipped (already incurred that RGB conversion)

Q2. If levels in a YUV clip exceed rec.601 and/or RGB 0-255, how do I correct it's levels so that they do not exceed rec.601 and/or RGB 0-255 ?

many NLE's also have "broadcast safe" filters that will fix things , but it's usually better if you do it manually

I will compress the decoded mpeg 2 clips with Lagarith in YV12 colorspace, to save hdd space. I understand that Premiere Pro works in YUV mode with DV clips, and in RGB mode with YV12 avi clips that were compressed with Lagarith.

Old versions of premiere work in YUV with most native camera formats (e.g. AVCHD, DV, HDV , even MPEG2) . Almost everything else is converted to RGB on the timeline immediately even without adding filters.

Newer versions of premiere handle YUV better (You have a YUV preserved workflow with YUV filters) . Even lossless intermediates like UT video are treated correctly as YUV . Applying an RGB filter (e.g. RGB curves) will cause RGB conversion in only those segments that it's applied to

I am still confused about what happens with levels when the imported avi clips are converted from YV12 to RGB by Premiere, and when the exported Premiere sequence is converted from RGB to YV12 by AviSynth:

Q3. After importing avi into premiere, levels YUV 16-235 are expanded to RGB 0-255 ?

Depends on the type of "AVI" , DV is treated as YUV and it's a YUV timeline even with some filters like fast color corrector (no RGB conversion) . Old versions of premiere treat almost everything as RGB, incurring a Rec601 conversion (this means yuv 16-235 => RGB 0-255), except native camera formats and v210. Even HD formats are incorrectly treated with Rec601 (instead of 709) . The only exception was v210. New versions of premiere handles YUV better, have more YUV filters, correctly does 709 for HD formats

Q4. After importing avi into premiere, levels YUV 16-235 are expanded to RGB 16-235 ?

In old versions of premiere, RGB conversion was always Standard 601. So always Y' 16-235, CbCr 16-240 => RGB 0,0,0 - 255,255,255. This means you always lose Y' 0-15, 235-255 if RGB conversion occurred. So the key was to use fast color corrector to "fix" levels (if it's a YUV format, and it's interpreted as a YUV format like DV) before you incur the RGB conversion. One way you can tell if you have incurred an RGB conversion when there is red render bar on the timeline

Q5. After exporting the premiere sequence to avi, with Lagarith compression in RGB mode, and after converting that avi to YV12 with AviSynth with ConvertToYV12(matrix="Rec601"), levels will be contracted from 0-255 to 16-235, or clipped from 0-255 to 16-235 ?

It won't be clipped

Q6. After exporting the premiere sequence to avi, with Lagarith compression in RGB mode, and after converting that avi to YV12 with AviSynth with ConvertToYV12(matrix="PC.601"), levels will not be expanded nor contracted ?

That will be full range conversion. RGB 0,0,0-255,255,255 will be "mapped" to Y' 0-255, CbCr 0-255 . Usually this will not be good (illegal levels) . Vegas is different when using studio RGB (it maps differently, and you need to expand range "back" to normal)

And after I encode the avi to mp4 with h264 lossy compression, for delivery, I am confused about what happens with levels when that mp4 file is played back in MPC-HC, VLC, WMP, QuickTime:
Q7. If mp4 is YUV 16-235, levels are expanded to RGB 0-255 by the video player ?

Usually yes, but quicktime depends on the os and version - usually some sort of gamma shift as well accompanies this . It's a terrible player

Q8. If mp4 is YUV 16-235, levels are not expanded, but maintained at RGB 16-235 by the video player ?

Normal computer configuration setup is RGB 0-255 . So It will be like Q7, not Q8 . Some people don't have their configuration (player, graphics card, drivers,) setup properly. You have no control over that

[pandy]

Q9. Can you recommend books or documentation that could clear this confusion ?

For example http://www.poynton.com/ - hard to find something better than His papers.

[sanlyn]

Sometimes it's easier to use different histogram styles than to try playing with a ton of numbers.

One source of confusion might be that YUV is a video storage technology, while RGB is a video display technology. The contrast and luma range of most YUV colorspaces is greater than that of most RGB display colorspaces. And again, PC display and TV display have differences as well, as you're probably aware. I use two types of histograms: YUV histograms to view the way video is stored as YUV, and RGB histograms to view the way that same video will display as RGB on either a PC or a TV.