Rookie Help - Getting Rid of Artifacts/General Quality

[bschneider]

It depends on the software or device. [...]

That wrapping sounds awful (probably looks awful too). Thanks for the info!

[...] As a general rule: standard definition video uses rec.601 YUV, high definition uses rec.709 YUV. The correct color model can be specified within the video. But if it's not you have to guess.

Yeah, I can see the difference, put next to each other. Might be hard if not compared directly though. Thank you for the link!

In truth, I hardly ever use it either. If your video includes a standard SMPTE color bars you can use it as a guide to your color adjustments. That is, you can adjust the hue and saturation (or whatever you need to adjust) to get the color bars right, then hope that that correction holds for the entire video.

You mean that before or after the program they would broadcast a test screen? I don't know many programs that do that, the ones that I try to edit definitely don't, unfortunately.

If you see the vectorscope graph has data out at the very edges (UV<16, YV>240) you know the picture is over saturated -- but you probably already know that just from looking at the picture. I use the U and V waveform graphs a lot more.

Ah, so that's how you see it, I figured as much, looking at the Orange Caramel vectorscope.

Yes. You need to ConvertToYUY2(interlaced=[true or false]) before calling VideoScope().

Okay, then I saw that right, thanks!

You might be trying to cover too much detail in too many places at once. I did that at first. I used a lot of automated one-stop-shopping apps that spit out finished videos lickety-split. But I learned nothing about the details. [...]

I know I might seem like I'm trying to rush over things, but that's not my intention. Maybe at one or two points, admittedly, but generally not. The problem for me is that there's a lot of ground to cover for many different topics that seem to apply to the video I chose to work with (thank you Korean TV station!). I'm trying to go for a better picture overall and that includes working with colours, the focus has shifted towards that, but really just because questions seemed to come up for that (since I have a harder time working with colours than the general stuff, I think). It's not that I'm trying to ignore things I don't like or anything, the conversation just naturally steered into that direction. I'm having trouble finding a clear route to take, so I shift from the path left and right to learn new things, even if it might not be relevant at this time.

Depends. Most of the time you could get shifts in saturation or color balance. HD has generally higher saturation and other differences. I use the ColorMatrix plugin and check it out, just to cover my tracks if nothing else.

I see.

If you get involved in CMS calibration systems at a site like AVSforum you get into math and charts that will have you on industrial strength Valium in no time.

Sounds like it has its upsides

Re: colorspaces for histograms and 'scopes: Check jagabo's notes on that. In VDub you can tell gradation curves to work with different color. Switching the plugin to a different colorspace takes time, though, because gradcurves builds a new copy of the video. With a long video you could be sitting there for quite a spell.

Ah yeah, I found that! That's really handy, thank you!

If running VirtualDub in "full processing mode", it'll convert to RGB and outputs RGB by default if you don't specify otherwise. To avoid that, set output compression/color to what you want and use "Video" -> "fast recompress".

I think I read about that somewhere, good to know.

]Working with off color video is a pain. It won't be perfect whatever you do. After letting it rest overnite I saw a few things I'd change, but I let it be. Mind, you can't do it with every video that comes down the pike. Life ain't long enough. I made hundreds of recordings on a DVD recorder off cable TV, and all I did was cut out commercials with a smart-rendering editor and author a menu for it. Never had to touch color or noise.

Oh believe me, I do not intend to do this for all of my videos. I do enjoy learning something new and working to improve quality of my files (subjectively, but still), but I do not have a death wish.

I get the idea of a slightly off-color costume from the first image you posted. It's too green. [...] Maybe the tops were white and the pants different, I don't know. If I could get skin tones that didn't make the girls look like they had liver disease, I used my best guess.

Haha, okay okay, I see what you mean. It definitely looks off.

Tips from other posts and graphics sites: don't work in bright light. Work in subdued, indirect light. Don't put a lamp in front of your monitor. Don't work in total darkness, or your eyes will think the monitor is too bright and will shut down on you. If you stare at something for too long, your brain tries to "normalize" it. Turn away for a while and come back, things will look different. 24 hours makes a big difference, too.

That's good advice, thank you. Incidentally, that's exactly how I like it, I sit in the corner of the room and have a not-so-bright ceiling bulb in the center of the room. So I think that should give okay light for most of the time. Not overdoing it is certainly another valuable point of interest.

Working with an uncalibrated monitor is like shooting yourself in the foot. That's a different Pandora's box too complicated for here and now, but you can run a cheap (free) monitor check here: http://www.lagom.nl/lcd-test/. Like many people here I use an IPS LCD display calibrated with a colorimeter kit. Not cheap. But see what the lagom link has to offer for free. Lots of people use their site.

Thanks for that link, I worked through all of it and only had to adjust contrast a bit, not by much. Unfortunately, I realised that my monitor doesn't allow for gamma adjustment. On the other hand, the gamma test seemed good, but I still would've liked to fiddle around with it to make sure I'm as well-calibrated as possible within my budget of nothing My display has IPS too, no expensive calibration has been done though. I use a Dell U2412M, in case it's relevant.

Noise is difficult to see in bright light. I didn't recognize that background grunge until much later in the process. Sometimes you miss noise problems altogether and see it 2 days later on your TV. Drat!

That is good to know, I will keep that in mind, thanks!

QTGMC: I didn't see a need for it here. You can use it with various parameters, but it's slower than low-power MCTD and doesn't fix the same problems. The two plugins are different. I've seen bad video that required both. In that case, might as well drive around and do some shopping while you wait.

Well, I don't mind long encoding times to be honest. I don't like interlacing if I can avoid it though, so de-interlacing is kind of a must for me, personally. Personal preference I guess, maybe it's entirely psychological, of course in running video, I probably couldn't really tell the difference.

You can copy settings from a .vcf to use VirtualDub filters in an Avisynth script. If you stick with script-only, be sure to convert to RGB for that filter. [Using Vdub filters in avisynth]
It can be more trouble than it's worth. You get the right filter values by first using the filter in VirtualDub, save the .vcf data, then put it in the script. If you want to change or tweak, you have to do it all over again unless you can figure out what those numbers mean. Pain in the whatsit. If you do this with something like gradation curves, you have a text string of over 132 characters to format and the numbers have to be formatted as text, not numbers. Triple pain.

Ouch yeah, that would be very annoying. But thank you for the detailed explanation, it's very good to know, I'm sure it will come in handy. By the way, I downloaded Lagarith and just tried fiddling with ColorMill a bit, then encoded to AVI using Lagarith. Very happy with the result, it was fast and fool-proof, but as you said, the file size is quite baffling Still, thanks a lot for bringing it up, I think I will use that A LOT. Having found that the colours in those two video samples are majorly stunted, I will probably find many applications for ColorMill, GradationCurves and Lagarith going through some of my collection!

First, I'd stick with learning how to get up a script and just run it. Color can come later and you don't always need RGB. You have to denoise and do YUV stuff first anyway. That's a subject in itself.

Oh for sure, and I'm absolutely interested in those fields, the conversation just kind of drifted in this direction and it felt like something I'd have a harder time with, so I guess I tried to focus on the more difficult part and got carried away a bit. Which doesn't mean I'm sick of colours, I want to learn more about that too, but you're right more knowledge on basic scripting would probably be advised. I just kind of hit a dead end and ran out of things to ask. Probably a good sign to go for some investigating of my own.

Good grief, after not posting here for many years, I seem to be in overdrive, LOL! Never thought that would happen. Better get to those earlier links from jagabo and poisondeathray for a while and learn something new.

Input is always appreciated!

[bschneider]

Okay, so it's been kind of quiet. I've been playing around a bit, looked at a few other videos I have and am quite stumped at how broken those are. If you've gone crazy trying to figure out how my first video got past QA for TV broadcasting, you don't want to see the rest, haha.

That's not what I'm here for, I just wanted to "hand in" my so far final result of the video we discussed here for the most part. Just so you see how it came out, I figured you might be interested. I pretty much took a bit of what everybody suggested and maybe fiddled with a variable or two, it's pretty much the things you've posted here. The colours I did "myself", I played with ColorMill and while I think it looks better than before, it's far from perfect. There's a bit too much blue in some scenes, unfortunately, turning down the blue means too much green and red and it's an endless cycle. It's hard to get it exactly right.

The only thing I tried on my own was when I noticed that adjusting the colours with ColorMill left some noticeable banding in some spots. I ran GradFun2DBmod() over that, it still left some banding, but it helped out a bit.

It's a 10 second sample again, I tried to roughly cut out the same 10 second bit I sent as the original sample. It's the finished mp4 (I like mp4 because it's quite light hard drive space-wise), the Lagarith avi was a bit large.

If you have any thoughts on the video, feel free to tell me, but this is really just because I didn't want to leave abruptly without saying anything

On that note, a final huge, gigantic THANK YOU to everybody involved! You guys opened a lot of options to me, some of which I'll have to put on hold due to other priorities at the moment but still, you gave me exactly what I needed, a good intro into the whole matter. I'm really grateful for all your efforts!

[poisondeathray]

Color adjustments are very subjective, but it looks to me like you've reduced the highlights too much .

Have a look with histogram() (it's really a Y' waveform) on the output video, you will see significant amount of unused headroom. When you reduce it that much , you tend to get this "burn" effect where there is no separation in levels, low highlight contrast - ie. the details become reduced to fewer "shades". White doesn't look white anymore, it becomes "grey" (but you've made some adjustments to make a reddish tint to it). Another way of saying this is you're using a limited dynamic range, when you have more available

[jagabo]

I think you reduced the bright levels but not in a way that restored any of the details above Y=235. That is best done in AviSynth with something like ColorYUV(gain_y=A, off_y=B). Be careful if you use Tweak() -- it crops brights and blacks by default. If you do it in VirtualDub you must use a filter that works in YUV. Otherwise VirtualDub will convert the video to RGB, losing the super brights and super blacks before the filter is applied. The Brightness/Contrast filter in VirtualDub can work in YUV. So use that before any others.

The video is also over sharpened. The original video already has some over sharpening halos (my guess it was originally 720p and was upscaled to 1080i with sharpening) but your additional sharpening has really accentuated the halos.

One thing I wanted to mention about color adjustments: These are stage productions. There are colored lights all over the place. It wasn't the intention of the producer to have perfectly natural colors everywhere. This is also true in movies where colors are often intentionally tinged to add a sense of warmness, coolness, night, etc.

[bschneider]

Oh my, that's quite the response. Yeah, I had some problems with it, I couldn't always put my finger on it, but the brights you brought up were another issue I noticed.

Color adjustments are very subjective, but it looks to me like you've reduced the highlights too much .

Have a look with histogram() (it's really a Y' waveform) on the output video, you will see significant amount of unused headroom. When you reduce it that much , you tend to get this "burn" effect where there is no separation in levels, low highlight contrast - ie. the details become reduced to fewer "shades". White doesn't look white anymore, it becomes "grey" (but you've made some adjustments to make a reddish tint to it). Another way of saying this is you're using a limited dynamic range, when you have more available

I agree with you I went a bit heavy on the brights. I will study the histogram again, that's a good idea, the time I chose to get into this whole matter was not optimal, I have a few things on my schedule at the moment.. So I'm grateful for every observation that I don't have the time and experience to make Thank you!

I think you reduced the bright levels but not in a way that restored any of the details above Y=235. That is best done in AviSynth with something like ColorYUV(gain_y=A, off_y=B). Be careful if you use Tweak() -- it crops brights and blacks by default. If you do it in VirtualDub you must use a filter that works in YUV. Otherwise VirtualDub will convert the video to RGB, losing the super brights and super blacks before the filter is applied. The Brightness/Contrast filter in VirtualDub can work in YUV. So use that before any others.

The video is also over sharpened. The original video already has some over sharpening halos (my guess it was originally 720p and was upscaled to 1080i with sharpening) but your additional sharpening has really accentuated the halos.

One thing I wanted to mention about color adjustments: These are stage productions. There are colored lights all over the place. It wasn't the intention of the producer to have perfectly natural colors everywhere. This is also true in movies where colors are often intentionally tinged to add a sense of warmness, coolness, night, etc.

I did use ColorYUV but I didn't know how far I should go with it. Guess it could have used some stronger variables then. Afterwards I did have issues with how the pants still reflected the light so heavily that they pretty much absorbed half the screen. So I should steer clear of any filters that only work in RGB? Or just use them at the very end? LMotlow recommended ColorMill, but I'm actually not sure if he meant it as a way of getting some practice or if he suggested I use it for my final edit, I don't recall.

It's over sharpened? Oh, I guess I didn't look closely enough, I remember you suggesting to sharpen mildly after using Deblock_QED, so I applied some light sharpening at the end. I'll take another look at that, thank you!

Oh definitely, I think I did go a little heavy on one scene or another (especially another one that isn't in the sample I overdid I think). But generally I tried to keep in mind that the stage lights affect the colours a lot. Still, I agree with LMotlow that the colours aren't really right and thought that what he did with the sample was a definite improvement. My attempt at recreating it just didn't work out as well as I hoped, I suppose, hehe. Thanks again that all of you are spending so much time giving me advice!

[LMotlow]

Considering you're just getting your feet wet with this stuff, things are coming along. My first time at this denoising and color stuff, I mostly cursed in 3 languages and invented 2 new ones.

I've edited hundreds of home-made DVD and HD recordings off cable tv. Never had to apply detailed fixups with them, save for a handful of real low-quality movies. I'd edit Castle episodes by removing commercials but never had to adjust for invalid levels or weird colors. Responsible broadcasters usually don't let excessively blown-out video get through.

Your two samples are different. As I said earlier, it looks like the boys in the control room were stoned. Or they did a few b.s. things on purpose that some might think are cool, but I wouldn't be paying money to watch it. Again, each of those 4 shots has a different color balance and levels, even with blue overheads and foots shutting on/off and yellow floods doing floor sweeps. The levels are haywire and the green cast is too uniform. Looks like something out of a video player that had the wrong IRE settings.

Even with videos I had to repair, I seldom had to cut up separate scenes. It's more common with VHS. A tape looks different every time it's played (that's why I make 2 or 3 captures of my old tapes). Mainly I start by setting some dark and bright YUV levels to cover some of the wildest and craziest parts of a video. If the original is a real mess, it usually requires detail work in RGB -- but not always. RGB has filters that let you control very specific areas of an image without affecting other parts. Most of the work I did on that color-graded mkv was with gradation curves, and ColorMill sometimes to tweak it.

A common method is called primary and secondary correction. Apply an initial set of color filters (either YUV or RGB) to handle the worst problems, like overall levels and color. That's the primary setup. Then add a second set of filters to refine the first in more specific areas -- that's the secondary correction. What I did in effect was apply a primary fix in YUV, then secondary tweaks in RGB.

As jagabo and poisondeathray imply, your two samples pose two main problems. One is the apparent distortion of levels, saturation, etc., either deliberately or carelessly. The other is the usual way stage lighting varies. I made choices based on what I figured was going on, which of course means I guessed a lot. That's difficult with many sources nowadays. Effect without content has tremendous popularity these days, along with an obsession with over sharpening so that people can count pimples and nose hairs without being distracted by such matters as whether or not anything is really happening underneath all that. Not my cup of tea, but lots of people ante up for it. Nice thing about learning how to work with filters: you can make video look however you want.

Just some more detail here about how I worked After School. Mind, this is what I did to get my results, not what anybody must do. Here's the script for the later mkv:

MPEG2Source(path to "SAMPLE After School.d2v")
ColorMatrix(mode="Rec.709->Rec.601")
ColorYUV(cont_y=-30,off_y=-10)
SmoothLevels(8, 0.75, 255, 16, 255,chroma=200,limiter=0,tvrange=true,smooth=200,dither=100,protect=4)
ContrastMask(enhance=2.0)
santiag()
MCTemporalDenoise(settings="low",interlaced=true)
ConvertToRGB32(matrix="Rec601",interlaced=true)

In VirtualDub I loaded the results 4 times, tweaking each for only one scene, one version at a time. It's a short clip, so doing this 4 times produced 4 lossless Lagarith RGB clips. I called the results "sc1", "sc2", etc., for Scene 1, 2, etc. I have attached a .zip file that contains the .vcf files for Scenes 1 and 2. Sorry I lost vcf's 3 and 4, must have saved them in another subfolder somewhere(!). The filters were gradation curves and ColorMill.

Here's an RGB histogram and vectorscope of filtered Scene 1. You can see I have brights looking just a little bit hot, though not as hot as the source. Those sharp "peaks" at the right-hand sides of the histogram show clipping in the original:



TMPGenc's TVMW5 encoder gave me these histograms (YUV on the left, RGB at right).


Working without these tools is a pain. They would have revealed that in the Finished mp4, luma was depressed but red was soaring beyond RGB255 at the bright end. Poisondeathray commented on these effects:



The script below loads all 4 versions of AVI's that I made with VirtualDub. I auditioned the output, tweaked again, then joined again. The script cuts the scene I want from each version, then joins the cuts. I also made a copy of the audio track as a lossless .wav file. The script retrieves the audio and makes sure everything is sync'd.

v1=AviSource(path to "AfterSchool_sc1.avi").Trim(0,47)
v2=AviSource(path to "AfterSchool_sc2.avi").Trim(48,159)
v3=AviSource(path to "AfterSchool_sc3.avi").Trim(160,319)
v4=AviSource(path to "AfterSchool_sc4.avi").Trim(320,0)
aud=WavSource(path to "AfterSchool.wav")
v5 = v1 + v2 + v3 +v4
AudioDub(v5,aud)
AssumeFPS("ntsc_video",sync_audio=true)
ConvertToYV12(interlaced=true)
FFT3DFilter(bt=3,plane=3)
GradFun2dbMod(thr=1.5,mask=false)
ColorMatrix(mode="Rec.601->Rec.709")
return last

To echo poisondeathray and jagabo: color and final visual impressions are often a personal matter, but just as often you should consider things like lighting and what the video's creator had in mind. It also helps if the broadcast isn't a mess -- which makes a mystery to me of what the broadcasters had in mind here. Maybe they just wanted the usual in-your-face effect. I would've switched channels.

I still think my mkv's darks are too dark and everything's too red. If I had it to do all over again, I probably would. And here I am again, taking up too much time and space.....so, adios.

[jagabo]

I did use ColorYUV but I didn't know how far I should go with it. Guess it could have used some stronger variables then.

Since most VirtualDub filters work in RGB, and it will crush values below 16 or over 235, you want to be sure that all luma values you intend to keep fall within 16 and 235. Maybe you did this and the loss of details was too strong noise filtering. U and V values need to be within the RGB cube partion of the YUV colorspace, not simply between 16 and 240.

So I should steer clear of any filters that only work in RGB?

No, working in RGB is fine. And some things can really only be fixed in RGB. But make sure the initial YUV to RGB conversion doesn't crush super blacks and superbrights in parts of the picture you care about

It's over sharpened?

Yes. Use a magnifier and look at sharp, high contrast edges. You'll see halos. At the transition from dark to light there is an extra dark band in the dark area, an extra bright band in the bright area. A 4x point resize of a small area in the original source on the left, your sharpened version on the right:



There is a bit of a halo in the original image and your sharpening has strongly increased it. Below I used a quick DeHalo_alpha(rx=2, ry=2) to reduce the halo then sharpened the image a bit to approximately match your sharpness (again, your processed image on the right):

[LMotlow]

Good point from jagabo.

Saw one version (?) of the whole clip several days ago on UTube. http://www.youtube.com/watch?v=piu1QP-TGcw . Looks like there's lots of extra South Korean won to spend on scenery. The original set probably looked fabulous. Looks like the afterburner effects were on purpose. I'd rather see less burnout and more Girl.

[jagabo]

Unless you know this is the actual digital broadcast transport stream I wouldn't necessarily blame the broadcaster for the problems. It could have been captured from the analog component output of a cable/satellite box with something like the Hauppauge HD PVR, with bad proc amp settings.

[LMotlow]

Yep, I realize that. Never know what you'll see on UTube. I think the O.P. said somewhere his vid was recorded off TV (? ?). Whatever. Wouldn't mind having a commercial release of the concert.

[bschneider]

Considering you're just getting your feet wet with this stuff, things are coming along. My first time at this denoising and color stuff, I mostly cursed in 3 languages and invented 2 new ones.

Thanks for saying such nice things, I do feel a bit stupid though for forgetting many of the basic things I learned already. I guess what you said earlier, that happened to you, applies to me as well. Tried to skim at a lot of places and got a sub-par result because of it.

Your two samples are different. As I said earlier, it looks like the boys in the control room were stoned.

Well, looking at other videos I have, from different broadcasters and from different times (ranging from 2007-2014) I would say that either all channels employ the same video guys for 7 years, all QA teams in Korea have very liberal working conditions, or none of them are on mind-altering substances, and just have a very warped perception of quality. Not sure which one of those is the worst, but fact is, most of the recordings I have seem very troublesome. As jagabo suggested later, this is probably not 100% the broadcaster's fault, but also due to bad recording.

A common method is called primary and secondary correction. Apply an initial set of color filters (either YUV or RGB) to handle the worst problems, like overall levels and color. That's the primary setup. Then add a second set of filters to refine the first in more specific areas -- that's the secondary correction. What I did in effect was apply a primary fix in YUV, then secondary tweaks in RGB.

Okay, thanks a lot, now I know for sure!

Effect without content has tremendous popularity these days, along with an obsession with over sharpening so that people can count pimples and nose hairs without being distracted by such matters as whether or not anything is really happening underneath all that. Not my cup of tea, but lots of people ante up for it.

Tell me about it... I've also repeatedly used those invented languages for swearing that you mentioned earlier when I look at the camera work. Some cameramen literally just shake the camera left and right very quickly during bass-heavy parts of a performance. Personally, I'd rather have 3 different cameras on tripods, a close-up, one for the group as a whole, one that might be able to do some novelty shots. So sick of cameramen trying to prove their worth by "inventing" new techniques that look absolutely idiotic. But that's another matter entirely.

Just some more detail here about how I worked After School. Mind, this is what I did to get my results, not what anybody must do. Here's the script for the later mkv:
[...]
I called the results "sc1", "sc2", etc., for Scene 1, 2, etc. I have attached a .zip file that contains the .vcf files for Scenes 1 and 2. Sorry I lost vcf's 3 and 4, must have saved them in another subfolder somewhere(!). The filters were gradation curves and ColorMill.

Thanks a lot, exact code is always nice to see which filters have been used! There are so many of them for so many different things, sometimes it's hard to gauge which are appropriate. Some seem like they haven't gotten updates in a long time, but some of them seem to stay relevant regardless.

[Histograms]

Yeah, I see. The clipping is maybe the most unfortunate thing, as I mentioned earlier with the pants shining brighter than the sun.

And here I am again, taking up too much time and space.....so, adios.

Oh my, I don't want you to invest time that you don't have As I said somewhere in this thread, you've all done a lot and if you have other things to do, then by all means, please don't let yourself be distracted by all the questions I ask, it's mere curiosity. Sorry that you took so much of your time for this video.

Maybe you did this and the loss of details was too strong noise filtering. U and V values need to be within the RGB cube partion of the YUV colorspace, not simply between 16 and 240.

Okay, so how to check for correct luma values is obvious, how do I check this for U and V? Histogram("color") won't help me here, right?

No, working in RGB is fine. And some things can really only be fixed in RGB. But make sure the initial YUV to RGB conversion doesn't crush super blacks and superbrights in parts of the picture you care about

Right, thanks for clearing that up!

Yes. Use a magnifier and look at sharp, high contrast edges. You'll see halos.

Oh my, I messed up big time there. You're definitely right, that was terrible. Thanks for the suggestion with the dehaloing function. LMotlow suggested FFT3Dfilter earlier and I noticed it does quite a lot of things, among others it has a dehaloing feature, too. Is that any good or is it better to use dedicated filters like the one you used?

I'd rather see less burnout and more Girl.

Preaching to the choir

Unless you know this is the actual digital broadcast transport stream I wouldn't necessarily blame the broadcaster for the problems. It could have been captured from the analog component output of a cable/satellite box with something like the Hauppauge HD PVR, with bad proc amp settings.

Yep, I realize that. Never know what you'll see on UTube. I think the O.P. said somewhere his vid was recorded off TV (? ?). Whatever.

Since some performances don't appear on YouTube or online streams, I'm pretty sure these have to be recorded off HD TV channels. They usually come in .ts or .tp format as well, which would seem odd for YouTube rips for example.

Quality between performances varies a lot, so it's safe to assume that some of the blame should be taken by the uploaders. Unfortunately, I'm dependent on them, since I can't get Korean channels here.

Wouldn't mind having a commercial release of the concert.

My idea exactly, I tried to contact the broadcaster beforehand, I wouldn't want to bother all of you if there was a way to easier get reliable, good quality. But they don't seem to keen on customer service, especially not "potential customer" service. I'd love to get a decent quality BluRay of some performances and would be more than willing to pay good money for it. That option is apparently not appealing to that particular TV broadcaster.

[jagabo]

Some notes on hue and saturation and the UV vectorscope:

Hue is the angle of rotation about the center of the UV vectorscope. A straight line from the center of the UV vectorscope to the periphery is of constant hue, increasing in saturation from the center (grey) to the edge (fully saturated color). The attached AviSynth animations show hue and saturation changes via Tweak().

This first script uses standard SMPTE color bars to show the effect of hue changes with Tweak(). As you can see, changing the hue rotates the UV coordinates about the center of the UV plot. This is similar to the "tint" control on your TV.

Colorbars(width=640,height=480, pixel_type="YUY2")
Trim(0,360) # only 361 frames
Crop(0,0,-0,320) # just the colorbars
AddGrainC(0.0, 1.0) # add noise to the chroma channels to make the dots in the UV vectorscope bigger
Animate(0, 360, "Tweak", 0.0,1.0,0,1.0,false,false,  360.0,1.0,0,1.0,false,false)
VideoScope("both", true, "Y", "UV", "UV")

This second script uses SMPTE color bars to show saturation changes with Tweak(). Increasing the saturation increases the distance from the center of the graph to the UV coordinate -- but they move in straight lines from the center to the edges. This is similar to the "color" control on your TV.

Colorbars(width=640,height=480, pixel_type="YUY2")
Trim(0,360) # only 361 frames
Crop(0,0,-0,320) # just the colorbars
AddGrainC(0.0, 1.0) # add noise to the chroma channels to make the dots in the UV vectorscope bigger
Animate(0, 360, "Tweak", 0.0,0.0,0,1.0,false,false,  0.0,1.0,0,1.0,false,false)
VideoScope("both", true, "Y", "UV", "UV")

I used AddGrainC(0.0, 1.0) to make the dots in the vectorscope bigger. You can leave that out but you will see single pixel dots. In both scripts Y never changes. You can see that in the Y waveform display at the top right. At the bottom left you can see the U and V waveforms.

[jagabo]

U and V values need to be within the RGB cube partion of the YUV colorspace, not simply between 16 and 240.

Okay, so how to check for correct luma values is obvious, how do I check this for U and V? Histogram("color") won't help me here, right?

That's correct. Histogram("color") is of limited usefulness for this. You can use the HightlightBadRGB() script I linked to earlier.

Here's a video that uses HighlightBadRGB() to display cross sections of the RGB cube in the YUV colorspace. It starts out with all pixels set to Y=0. At frame 1 Y is increased to 1, at frame 2 it's increased to 2... up to Y=255 at frame 255. In every frame U and V are gradients cover the full range of possible U and V values, 0 to 255, and all ~65000 possible combinations. HighlightBadRGB(0) is used to blacken out all pixels where U and V fall outside the RGB cube. So each frame of the video shows all valid RGB colors available at that Y value. This is based on the rec.601 RGB/YUV standard.

Sample of frame 80 (Y=80):

[bschneider]

Sorry for the delay, I only now found time to reply.

That's correct. Histogram("color") is of limited usefulness for this. You can use the HightlightBadRGB() script I linked to earlier.

Ah right, I totally forgot. I'm going to save that script. If I do find out that I have bad colours in there, what can I do to "fix" the colours so they appear relatively normal after conversion?

Thanks a lot for the animations too! I saved them in a file, just for safety, it's really useful to be able to take another look to see how Hue and Saturation work.

[jagabo]

Exactly what you want to to adjust illegal colors depends on why they're illegal. Sometimes you need to reduce the saturation or shift the U/V planes. Other times brightening or darkening the luma is necessary. Sometimes RGB adjustments are necessary.

Even the best produced video may have a little illegal coloring around small details. Just watch out for large areas with large errors.

As noted earlier, my HighlightBadRGB() is crude. Some borderline good colors may look bad, some borderline bad colors may look good. Gavino's ShowGood() script in that same thread is more accurate but slow. It takes my computer a minute or two to load the script. And his script shows good colors as black (or the optional color you specify), bad colors with their colors. I added a mod to his script, ShowGoodRGB() that leaves good colors as they are, and marks bad areas with black or a user specified color -- the same thing my HighlightBadRGB() does.

function ShowGoodRGB(clip c, bool "pcRange", int "color") {
  mask = c.RGBMask(pcRange).Invert()
  mt_merge(c, c.BlankClip(color=color), mask, U=3, V=3, luma=true)
}

Add that to the end of his ShowBadRGB() script if you want to use it.

[LMotlow]

Again, that's good stuff from jagabo. If this keeps up, I'll run out of disc space for archiving copies of his posts!

You're at a disadvantage with Windows7 in some respects. Many video tools that work in XP don't work there. I use Win7 for encoding and authoring big HD files and other things, but even my old Photoshop and Corel photo apps have problems with Win7. Like jagabo said, his script for Invalid RGB and even some other scripts for RGB that run in Avisynth can give useful info. But readjusting and then running those scripts over and over is a pain.

Look at it this way: budget NLE's from SONY, Cyberlink, and free stuff and whatnot don't have histograms, vectorscopes, curves, fancy color wheels, and all that for working RGB -- or YUV, for that matter. Better tools are in pricey Pro apps like Premiere Pro, Vegas Pro, Photoshop Pro, AfterEffects. Even pro apps can mess things up when not working with lossless input. It's possible to work in YUV and RGB by eyeball alone. But it ain't easy, even with pixel readers. Then there's the learning part -- even with a still camera it's tough to learn about corrections without those 'scopes to tell you what's happening. Also, if you want to work with loosless HD you'll get mighty big files.

That's one reason why many folks still use XP. I know a guy using Win98!! It's not just a matter of invalid values. There's more to it. Thanks to a college-age nephew, I could afford a student copy of Photoshop CS5 for my Win7, but I can't use CS5 in XP. For XP I still have a 2001 Photoshop v6 and old Corel stuff. That's where I learned to work with levels and color a while back, with still photos. Relatives would bring me godawful prints made with those throw-away film cameras. I'd scan 'em into Windows for fixing. Later, digital cameras made quicker work of it. For many quickie videos I use the color controls in TMPGenc's encoders, which work for YUV and RGB. Earlier I posted images of their histograms. I corrected a ton of ugly VHS captures using TMPGenc's tools alone, never went thru VirtualDub for it except to denoise. I always use Avisynth to make basic primary corrections first.

Here's a pic of TMPGenc TVMW5's color filter interface. At the bottom of the pic I've loaded some yuv filters, but made no adjustment here. The video loaded was the one done in VirtualDub. Upper right, the histogram shows RGB results. You can see it in YUV, too. Just a suggestion, but I use this app all the time for quick work in SD and HD. Has pretty good encoders, MPEG2 and x264. Also has a simple but useful timeline mode. Like many folks I've used TMPGenc stuff for years, doesn't cost a fortune. Click on the image in your browser to see it full size.



There are other workarounds. Some guys at doom9 use VirtualDub filters in avs scripts and AvsPMod. Personally I think that's a nightmare, but some people just have morbid phobias about virtualDub. Kind of like using a tank to move your furniture around.

[bschneider]

Exactly what you want to to adjust illegal colors depends on why they're illegal. Sometimes you need to reduce the saturation or shift the U/V planes. Other times brightening or darkening the luma is necessary. Sometimes RGB adjustments are necessary.

Even the best produced video may have a little illegal coloring around small details. Just watch out for large areas with large errors.

As noted earlier, my HighlightBadRGB() is crude. Some borderline good colors may look bad, some borderline bad colors may look good. Gavino's ShowGood() script in that same thread is more accurate but slow. It takes my computer a minute or two to load the script. And his script shows good colors as black (or the optional color you specify), bad colors with their colors. I added a mod to his script, ShowGoodRGB() that leaves good colors as they are, and marks bad areas with black or a user specified color -- the same thing my HighlightBadRGB() does.

function ShowGoodRGB(clip c, bool "pcRange", int "color") {
  mask = c.RGBMask(pcRange).Invert()
  mt_merge(c, c.BlankClip(color=color), mask, U=3, V=3, luma=true)
}

Add that to the end of his ShowBadRGB() script if you want to use it.

Okay, great, thanks a lot! I think that doesn't leave any questions for now. You've been quite thorough

Like jagabo said, his script for Invalid RGB and even some other scripts for RGB that run in Avisynth can give useful info. But readjusting and then running those scripts over and over is a pain.

Look at it this way: budget NLE's from SONY, Cyberlink, and free stuff and whatnot don't have histograms, vectorscopes, curves, fancy color wheels, and all that for working RGB -- or YUV, for that matter. Better tools are in pricey Pro apps like Premiere Pro, Vegas Pro, Photoshop Pro, AfterEffects. Even pro apps can mess things up when not working with lossless input. It's possible to work in YUV and RGB by eyeball alone. But it ain't easy, even with pixel readers. Then there's the learning part -- even with a still camera it's tough to learn about corrections without those 'scopes to tell you what's happening. Also, if you want to work with loosless HD you'll get mighty big files.

I saw that the Histograms and such were a bit different, but this type of software is quite expensive, gets updated all the time (by that I don't mean free updates, but completely new versions that cost a fortune again) and gets ever fancier with copy protection, internet activation and physical copies are getting a lot less popular. I don't want to pay a hefty price and then get locked out of the program I paid good money for because I changed my hardware. In the case of TMPGEnc Mastering Works, the site tells me two things I don't appreciate, firstly, that it gives me the German version only (which isn't terrible, but in all honesty, I prefer English, translations to German are often flawed and most of the terminology is in English anyway, so I don't see the point. But I could probably get the English version with a little fiddling). The other thing is that it requires "recurring internet activation". In the end, I don't know how reliable and quick the customer support is in case there'll be a problem and I don't want to constantly be at the mercy of the producer even though I already paid for their product. Which isn't to say I don't understand why they do it, I feel for them and their losses due to piracy, I just don't want to be punished for buying it.

This is not to shoot you down or anything, I hope you can convince me to buy it (although I don't think "doesn't cost a fortune" as you said applies, 100 dollars isn't pocket change for me ) and you're right, it's a name I read all the time, it seems to be really popular with people who know their stuff and that is definitely a plus. I just want to feel reassured before making such a purchase, in general, the program looked nice.

That's one reason why many folks still use XP. I know a guy using Win98!

Oh wow, yeah, I can imagine that a lot of this stuff isn't built for Windows 7 anymore. So far I fortunately didn't run into any problems though. There are times where I wish I had XP back, but Win7 is solid for normal use. Especially since I hear that XP was quite bad in its 64-bit form, never tried it myself though. I might make a small Win XP partition at some point, I do run into problems rarely, but not enough to justify making such a partition for yet.

Here's a pic of TMPGenc TVMW5's color filter interface. At the bottom of the pic I've loaded some yuv filters, but made no adjustment here. The video loaded was the one done in VirtualDub. Upper right, the histogram shows RGB results. You can see it in YUV, too. Just a suggestion, but I use this app all the time for quick work in SD and HD. Has pretty good encoders, MPEG2 and x264. Also has a simple but useful timeline mode. Like many folks I've used TMPGenc stuff for years, doesn't cost a fortune. Click on the image in your browser to see it full size.

I definitely like how it looks and hearing that it has some YUV colour correction filters is certainly a relief, but as I said earlier, I'm a bit uneasy with digital-only DRM-heavy purchases... Especially in that price range. If you could say something about your experience with it, that'd be much appreciated. Things like how many computers you installed it on, if there ever were any problems, if you had to contact customer support once, if yes, were they friendly/fast/helpful? I hope I'm not being too paranoid, but I'm not exactly rich. Not poor either, just at the point where I want to think hard about where to spend my money while still having food on the table lol.

[LMotlow]

Yep, you do have legitimate concerns. TMPGenc was just a suggestion, and no sense spending $$$ just to get some image controls. I bought v5 for the x264 encoder/HD decoder and interface. I don't know about the German you mention. Pegasus/TMPGEnc is a Japanese outfit with headquarters in Tokyo and a few big cities around the globe. I guess their installers can do German, but I always specified English. Internet validation at first install is one time for about 10 seconds. They seem to check up at random when they run, but I've run them plenty of times with the 'net turned off. Also transferred them to other machines. Updates are free, not needed unless what you have causes problems. I still use older versions from 2004 and later. I hear no complaints about their customer service, but I never needed it. Other software also uses that validation business, but Adobe gets worse with selling their top stuff by subscription. You don't own it, you just rent it. No, thanks! None of those apps can replace a lot of the repair and restore ability of Avisynth and VirtualDub -- which are definitely cost-effective, I'd say.

Anyway, there's still work to do in YUV. Stuff I edit from cable TV doesn't need all that work. It's just quick-cut, re-author, and done. It's those internet downloads and tapes and home movies that drive you up the wall.

BTW, 64-bit is OK but there are very few 64-bit filters for Avisynth or VirtualDub. Doesn't seem to be a big rush to get there.

[bschneider]

TMPGenc was just a suggestion, and no sense spending $$$ just to get some image controls.

Well, I read the name often, the interface looks nice enough, so that gives some reassurance. I wouldn't spend the money for the image controls though, actually, I'd probably use it like you, as a reliable encoder, since I work almost exclusively with MPEG transport streams and mostly encode to MP4 which TMPGEnc can handle. I don't question the usefulness of the program, that part I'm not worried about at all.

Internet validation at first install is one time for about 10 seconds. They seem to check up at random when they run, but I've run them plenty of times with the 'net turned off. Also transferred them to other machines.

Okay, that's a relief. The website said that only one installation on one machine is allowed, and to install it on another one, you have to fully uninstall it from the original computer. Which leaves me unclear what happens in the case of a system crash? What if I can't get back into my system to uninstall properly? What if my hard drive just dies?

Updates are free, not needed unless what you have causes problems. I still use older versions from 2004 and later.

That's very good!

Other software also uses that validation business, but Adobe gets worse with selling their top stuff by subscription. You don't own it, you just rent it. No, thanks!

I noticed that not too long ago, it's ridiculous. But unfortunately, they have kind of a monopoly on editing programs that are, in their basics, comprehensive and work without much investigation (and if you have to search for something, there's tutorials everywhere). So I guess they found many people just think Adobe programs are too expensive (which they are, sweet lord) and implemented that subscription system. Which is brilliant to get the people who think they'll save money, but probably did quite some damage to the old user base. I hate when such business practices become the norm, if it goes on much longer, we might see more and more of that kind of stuff pop up.

It's those internet downloads and tapes and home movies that drive you up the wall.

I can definitely confirm the bother that is internet downloads

BTW, 64-bit is OK but there are very few 64-bit filters for Avisynth or VirtualDub. Doesn't seem to be a big rush to get there.

Yeah, I don't use any 64-bit Avisynth/Virtualdub versions for that reason, I read everywhere that it's not supported well. Still, RAM is dirt cheap, having lots of RAM but not being able to use it would be a waste and I don't only use my computer for video editing, especially for video games, I personally need some RAM. It's weird how prevalent x86 still is, seeing as RAM is so readily available nowadays.

[LMotlow]

The website said that only one installation on one machine is allowed, and to install it on another one, you have to fully uninstall it from the original computer. Which leaves me unclear what happens in the case of a system crash? What if I can't get back into my system to uninstall properly? What if my hard drive just dies?

I've replaced PC's or moved the software to other computers without uninstalling from the old one. It didn't even occur to me to do it that way, LOL!. I just didn't use it on the old computer any more. I did read in another forum a while back that someone told TMPGenc they couldn't uninstall because a drive died. They were told to install the program elsewhere and just don't run the old drive any more. Come to think of it, I believe it was the digitalfaq forum a while back. But Adobe's a lot tougher to deal with IMO.

Anyway, good luck learning to work with Avisynth and VDub. RGB color tweaks usually come last anyway, if needed. There must be a ton of scripts and ideas to learn from in old threads here.

[bschneider]

I've replaced PC's or moved the software to other computers without uninstalling from the old one. It didn't even occur to me to do it that way, LOL!. I just didn't use it on the old computer any more. I did read in another forum a while back that someone told TMPGenc they couldn't uninstall because a drive died. They were told to install the program elsewhere and just don't run the old drive any more. Come to think of it, I believe it was the digitalfaq forum a while back. But Adobe's a lot tougher to deal with IMO.

That's a big relief, the site said that uninstalling is necessary or it won't work anymore. I guess I'll go for it then, thanks for the recommendation and the answers, and of course for all the help that came before!

Anyway, good luck learning to work with Avisynth and VDub. RGB color tweaks usually come last anyway, if needed. There must be a ton of scripts and ideas to learn from in old threads here.

Thank you! I have some busy days ahead of me, and after that, I'm sure I will find lots of interesting things on here

[manono]

There must be a ton of scripts and ideas to learn from in old threads here.

There'd be a whole lot more if that fellow sanlyn hadn't removed his. Maybe you know him?

[LMotlow]

Say, that's weird. Someone mentioned this before. I don't know why it keeps coming up, but all I know is that a long time ago in a photo chat forum I asked sanlyn about working with color in video. Result was, it got me into VirtualDub and started my browsing here. For all I know, "him" could be a "her". But I saw a wife mentioned in some posts, so I guess we're talking about a "him". Nowadays, though, you never know.

Why? Some of those scripts? Someone in another thread asked about that, and I said I copied dozens of scripts. Next thing I know I get a flood of PM's about scripts, vcf files, plugins, whatnot. Had to turn PM off for a while. Haven't had time to look. I believe in give-back to forums I learn from, but I don't imagine I'm the only reader who copies posts.

Otherwise all I recall from that old chat forum is that "he" hailed from west Tennessee, like myself.

I even got a PM asking about FulciLives. No idea where THAT came from!

[jagabo]

Okay, I understood how masks work, your links with the illustrations were very helpful! I wonder how you would create such a mask in avisynth though, it seems complicated.

Here's an example of how you would use a mask to apply color changes to a video based on the brightness of each pixel:

bmask=last.ColorYUV(cont_y=100, off_y=-15, cont_u=-256, cont_v=-256)
Overlay(last, ColorYUV(off_v=6), mask=bmask)

bmask is a a greyscale version of the video with a little contrast and brightness adjustment. If you add Return(bmask) between those two lines you can view that greyscale image (I did that to get bmask the way I wanted it). Overlay() here uses bmask to control which pixels get the ColorYUV() adjustment. Black parts of the image are unchanged, bright parts of the picture get the ColorYUV() adjustment, greys in between get in between adjustments depending on their brightness:

Before:


After:


You can see that the piano player hardly changes colors. But the bright white outfits lose most of their cyan tint and the women's faces look a little more natural. Getting the brightness mask just right is the tricky part.

[bschneider]

Here's an example of how you would use a mask to apply color changes to a video based on the brightness of each pixel:

bmask=last.ColorYUV(cont_y=100, off_y=-15, cont_u=-256, cont_v=-256)
Overlay(last, ColorYUV(off_v=6), mask=bmask)

Okay, so the first line describes the mask itself? As in, it tells the mask which pixels to take into account? I noticed that if I lower off_y for example, everything gets a lot darker when returning bmask (obviously), as you suggested, but what this means exactly I'm not so sure about. And Overlay takes all the conditions set by bmask and raises off_v for pixels that aren't completely dark, right?

You can see that the piano player hardly changes colors. But the bright white outfits lose most of their cyan tint and the women's faces look a little more natural. Getting the brightness mask just right is the tricky part.

Oh yes, I can see a very clear difference! It looks way better! Masks still seem kind of complicated to me. Maybe I should be more specific. What exactly do the variables set in the first line? I know what they do (contrast, offset), but what does it mean? Do you set the mask so that the parts you do not want to edit appear black when you Return(bmask)? As in, if I were to set off_y really low, almost everything would be black, meaning the parts of the video affected by the mask would be much smaller?

@LMotlow: I just got to use some YUV adjustments in TMPGEnc for the first time. Very intuitive and very helpful! Made a few small adjustments to another video that I think really improved the colouring. Thanks for the recommendation, so far, it's really fun playing around with it, even though, technically, I don't have time for it lol

[bschneider]

So Google didn't turn up anything and I don't know well how to describe this issue (language barriers maybe), so I hope I can throw another quick question in here. I attached a picture, as you can see, the background is all purple and green bars. What do you call that and what can you do against it?

[jagabo]

What exactly do the variables set in the first line? I know what they do (contrast, offset), but what does it mean? Do you set the mask so that the parts you do not want to edit appear black when you Return(bmask)? As in, if I were to set off_y really low, almost everything would be black, meaning the parts of the video affected by the mask would be much smaller?

Yes. In the general case Overlay() deals with four frames:

output_frame = Overlay(input_frame1, input_frame2, input_mask)

For each pixel of the output frame the corresponding pixels from the three input frames are examined. If the mask is full black the output is the same as input_frame1. If the mask is full white the output is the same as input_frame2. If the mask pixel is a shade of grey the output pixel is a blend of the two source pixels, weighted by the brightness of the mask pixel. If the mask pixel is 50 percent grey the output is a 50:50 mix of the two source pixels. If the mask is 25 percent grey the output is a 75:25 mix of the two source pixels.

In our specific case we have the original video (or whatever state of processing the original video is in at this point of the script) as the first source, a color modified version of the video as the second source, and a greyscale version of the source to use as as the mask. Since the mask is built from the brightness of the original video each output pixel is controlled by the brightness of that pixel. Dark pixels come predominantly from the original source video, bright pixels come predominantly from the color modified video.

Why did I contrast enhance the mask? For two reasons. First, legal Y values in the source range from 16 to 235 (before reaching this point of the script I had already adjusted the source to conform to this). But the overlay mask uses the full range from 0 to 255. So to make sure blacks were not effected, and full brights were fully effected I had to stretch the Y range from 16-235 to 0-255.. Second, I wanted the contrast stretched even further so not only full blacks were untouched by the overlay but other dark shades as well; and not only full brights got the modified colors but some lesser brights as well. I came up with the numbers I did (cont_y=100, off_y=-15) by looking at the resulting mask and looking at the result of the overlay. cont_u=-256 and cont_v=-256 were used to remove all color from the mask. Since I was already using ColorYUV() I made the image grayscale this way rather than a separate call to GreyScale().

How did I decide the blacks didn't need a color change and that brights needed off_v=6? First I isolated some blacks with a Crop() and examined the U and V channels (I used the dark heads of the audience in the foreground later in the video). They were pretty close to 128, perfect grey/black, so I thought blacks didn't need to change. Then I isolated something I thought should be white (part of one of the white outfits) and examined the U and V channels. The U channel was very close to 128 so it didn't really need changing. But the V channel was about 122, adding a cyan tinge. Increasing V by 6 would bring it up to 128, removing that tinge. If I applied that V adjustment to the entire image the darks would be tinged with red. Hence the Overlay() to limit the color change to bright parts of the picture.

[jagabo]

So Google didn't turn up anything and I don't know well how to describe this issue (language barriers maybe), so I hope I can throw another quick question in here. I attached a picture, as you can see, the background is all purple and green bars. What do you call that and what can you do against it?

I'm pretty sure those purple and green bars are supposed to be there. Ie, the dark background was replaced with purple and green bars with an overlay. But if you really wanted to remove them you might be able to reduce saturation in the dark areas. Use a greyscale image and a mask to apply that greyscale only to the dark areas.

[poisondeathray]

What is the pattern of bars accross time ? Does it only occur under certain lighting conditions ? Are they continuous throughout the whole performance?

I don't think those bars are supposed to be there : they are primarily only present in V plane

So you might attempt to apply filter to V selectively

[bschneider]

I'm pretty sure those purple and green bars are supposed to be there. Ie, the dark background was replaced with purple and green bars with an overlay.

Hmm, okay. The pattern seemed weird to me, it doesn't look very natural. But I don't see why somebody would use an overlay like that? I also attached another screenshot where the pattern is less noticeable, but also more irregular. If I'm seeing it correctly, that should be neutral white light with some green and purple blocks in there, but not consistently everywhere. So yes, poisondeathray, it seems to have to do with the lighting to some degree, but I don't think it's natural, same as you.

Aside from that of course there are still very noticeable blocks in general, even though deblocking is cranked to insane amounts, and it looks like strong ringing, especially noticeable in the frame I attached this time. Not quite sure what to do with that but the weird colour issues I noticed first. And since you probably wonder: it's the same in the untouched source video, I simply applied some deblocking and deinterlaced.

But if you really wanted to remove them you might be able to reduce saturation in the dark areas. Use a greyscale image and a mask to apply that greyscale only to the dark areas.

Don't the bars look relatively bright?

[Mask Explanation]

Woah, I think I understood it. Thank you!!

So you might attempt to apply filter to V selectively

I see. I'll try playing with that a bit, thank you!