How do I use ColorMill with Avisynth and GUIs such as VirtualDub?

[VideoFanatic]

I have some very weird colour issues that Tweak won't fix. I know you can manually adjust the sliders in colormill in VirtualDub to see the results but you can't do that in AvsPmod. Is there any GUI apart from VirtualDub that will let me adjust the sliders so I can see the results? And then I can copy the values into my Avisynth script?

Also can someone please give me step by step instructions as to how I can get ColorMill working with VirtualDub so I can adjust the sliders and see the results and then I would like to copy the volues in my Avisynth script? I have VirtualDub but I don't really know how to use it. I don't really like VirtualDub and I usually just enter my scipts manually in an AVS file then open it in AvsPmod to see changes then I encode with Simple x264 Launcher.

[jagabo]

VirtualDub:
1) File -> Open Video File
2) Video -> Filters... -> Add -> Colormill... adjust setting
3) back at the main VirtualDub window, File -> Save Processing Settings
4) Open the VCF file with Notepad. You will see the colormill settings:

VirtualDub.video.filters.Add("Color Mill(2.1)");
VirtualDub.video.filters.instance[0].Config(25700, 28516, 25660, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 5);
VirtualDub.audio.filters.Clear();

The config values are the ones you'll need in AviSynth.

In AviSynth, use the LoadVirtualDubPlugin and call it with the above seeings within the body of your script:

LoadVirtualDubPlugin("g:\Program files\VirtualDub\plugins\ColorMill.vdf","ColorMill") 

AviSource("filename.avi")
ConvertToRGB32() # RGB32 required for VirtualDub filters
ColorMill(25700, 28516, 25660, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 5)

[themaster1]

Don't forget to use separatefields after the rgb32 convertion, cmill don't support interlaced sources

[sanlyn]

Don't forget to use separatefields after the rgb32 convertion, cmill don't support interlaced sources

Not correct. ColorMill doesn't care whether the source is interlaced or not.

[jagabo]

If the source is interlaced use ConvertToRGB32(interlaced=true).

[VideoFanatic]

Any advice about what settings I would typically need to change? I see the following - what do they do and is there any documentation that explains what each setting does?:

Middle Point - Settings, Middle Point, Booster, Base Shift
Similar or Real HSV - sHue or Real, sSat or Real, sValue or Real
Gamma - Red, Green, Blue
Levels - Dark, Middle, Light
Red Green Blue - Dark, Middle Light
Saturation - Dark, Middle, Light
Sharp Smooth - Preprocess, Darm H & low, Postprocess (what do those things mean and do?). When I adjust the preprocess slider the program crashes.

There's also a Gamma is "Preprocess or Postprocess" button for every setting. What is the difference in these and what should I choose?

What does HSV stand for? Are there any need to use Tweak before or after using ColorMill?

What does the keep brightness and bad source options do? I ticked and unticked keep brightness but it made no change to the picture.

[themaster1]

middle point -> no idea what it is, never used and sharp/smooth i never use aswell

the best thing to do is use histograms in avisynth , the ones i use most of the time are:
Histogram(int2mode(1))
HistogramRGBParade()
HistogramRGBLevels
HistogramCMYLevels(range=true)

[jagabo]

You're basically asking somebody to write a book on color processing for you.

http://processing.org/learning/
http://www.amazon.com/Introduction-Computer-Graphics-James-Foley/dp/0201609215/

Gamma pre/post process: perform gamma adjustments before or after the other adjustments. It makes a difference.

HSV = Hue Saturation Value (brightness), a colorspace similar to YUV. Used typically in print color processing.

Keep Brightness does exactly what it says. If you simply pump up, say, the Green values, an image would get brighter. Keep Brightness will scale the image to retain overall brightness.

[VideoFanatic]

OK I've got my script but colormill is causing Simple x264 Launcher to crash with a popup saying avs2yuv_x86.exe has stopped working. Back in the main windows the error code is: PROCESS EXITED WITH ERROR CODE: -1073741819


What have I done wrong? I can adjust the colormill settings in VirtualDub but when I try to save the video I get the above problem

setmtmode(5,9)
Mpeg2Source("J:\2 = New\z = Encode\Raw January 05 1998.d2v", CPU=6)
setmtmode(2,0)

McTemporalDenoise(settings="medium", interlaced=true) # Sharpening strength is 150

turnleft().nnedi3(dh=true).turnright()

AssumeTFF() # or AssumeBFF() depending on the source
QTGMC(Preset="Super Fast") # Removes interlacing artefacts. Strangely this is faster than Ultra Fast
Vinverse()
RemoveSpots()
AddGrainC(var=1.0,uvar=1.0)
SeparateFields()
SelectEvery(4,0,3)
Weave()

Tweak(Bright=0, Sat=1.0, Cont=1.0, Hue=0.0, Coring=False)
ConvertToRGB32(interlaced=true)
LoadVirtualdubPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ColorMill.vdf","ColorMill",1)
colormill(25700, 25700, 25700, 25700, 21604, 25716, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 1)
converttoyv12(interlaced=true)

Crop(6,0,-8,0)
AddBorders(16,0,14,0, $000000)

[jagabo]

Isolate portions of your script to figure out where the problem occurs.

[VideoFanatic]

It happens on this line:
colormill(25700, 25700, 25700, 25700, 21604, 25716, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 1)

Presumable just importing a plugin and not using a function of it does nothing as just having the import plugin line doesn't cause a crash.

All I did in ColorMill was change the Levels to -16 Dark and middle to 16

[themaster1]

You should have written colormill with a capital C & M because ->
"ColorMill",1)

[jagabo]

Get rid of SetMtMode(). Run single threaded. AviSynth isn't case sensitive. ColorMill() is the same as colormill().

[VideoFanatic]

It's working in single threaded mode. I assume that's why RemoveSpotsMC won't work either and the other variants. What a shame. I really needed ColorMill but I need McTemporalDenoise more which takes over 24 hours to use on 1 video single threaded. Multi-threaded, it takes 3 hours. So it looks like I can't use ColorMill. Are there any alternatives to ColorMill I could use?

[jagabo]

Leave the first two SetMtMode() in place, call SetMtMode(5) just before the call to colormill.

[VideoFanatic]

OK I added the line you said but it's still crashing.

setmtmode(5,9)
Mpeg2Source("J:\2 = New\z = Encode\Raw January 05 1998.d2v", CPU=6)
setmtmode(2,0)

McTemporalDenoise(settings="medium", interlaced=true) # Sharpening strength is 150

turnleft().nnedi3(dh=true).turnright()

AssumeTFF() # or AssumeBFF() depending on the source
QTGMC(Preset="Super Fast") # Removes interlacing artefacts. Strangely this is faster than Ultra Fast
Vinverse()
RemoveSpots()
AddGrainC(var=1.0,uvar=1.0)
SeparateFields()
SelectEvery(4,0,3)
Weave()

Tweak(Bright=0, Sat=1.0, Cont=1.0, Hue=0.0, Coring=False)
ConvertToRGB32(interlaced=true)
SetMtMode(5)
LoadVirtualdubPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ColorMill.vdf","ColorMill",1)
ColorMill(25700, 25700, 25700, 25700, 21604, 25716, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 1)
converttoyv12(interlaced=true)

Crop(6,0,-8,0)
AddBorders(16,0,14,0, $000000)

[jagabo]

Adding the SetMtMode(5) worked for me. I guess you'll have to work in sections: perform ColorMill() as a separate script.

[sanlyn]

Any advice about what settings I would typically need to change? I see the following - what do they do and is there any documentation that explains what each setting does?:

Like any RGB color/contrast control, you learn to use it by just using it and observing what happens in VirtualDub's output panel. Once you hit a slider and the image turns totally dark or entirely blue, you'll get the idea. Even if you're using advanced image controls in Photoshop, After Effects, etc., there's no user manual or indicator telling you when your image is too red or too dark: you have to judge that for yourself. The plugin has a home page with a few examples (scroll down the page to see a few pics): http://fdump.narod.ru/rgb.htm . The control's download .zip comes with a brief .txt file; put that .txt file in your plugins folder to see it when you click the control's "help" button (although it's rather basic help, and you can always just read the .txt file by itself).

Middle Point - Settings, Middle Point, Booster, Base Shift
Similar or Real HSV - sHue or Real, sSat or Real, sValue or Real

The MiddlePoint is an additional gamma setting. If you don't know what gamma is, then imagine that all your colors and luma from 0 to 255 have an RGB value of 128 at the middle of the scale. In RGB or with ColorYUV, you can expand luma or colors toward the bright end by "stretching" the image values from the middle upwards and downwards. without too much effect on the very darkest or very brightest values. Raising or lowering the midpoint moves the midpoint from 128 to a higher or lower position on the scale. With ColorYUV you can increase or decrease the amount of stretching of values, but you can't move the middle point. In RGB, you adjust the middle point with this slider. Adjusting it makes the midtones brighter or darker (more or less intense) but leaves the brightest and darkest colors as-is. "Booster" will increase/decrease the amount of stretch above the midpoint; "Base Shift" will increase/decrease the amount of stretch below the midpoint. If you don't adjust the midpoint to begin with (set at "0"), Booster and Base Shift do nothing.

Unfortunately if you don't know the difference between the common terms dark, midpoint, and bright, you're at a disadvantage with any kind of image control whether you're in RGB or not. To vastly simply: you know that colors have an RGB range of 0 to 255. Most users would consider anything "dark" to have RGB values from 0 to 64 or so. Midtones include most of the skin tone range and extend from RGB 64 up to about 180. Anything brighter than 180 is considered the bright range. RGB 128 is the ultimate midpoint, or a neutral middle-gray if all 3 colors are involved. Also remember that adjusting any RGB color also adjusts luma; chroma and luma values are stored together for each pixel in RGB.

One advantage to working in YUV is that you can adjust luma with little effect chroma balance, or adjust chroma with little effect on luma. On the other hand, if you adjust one chroma channel in YUV you affect the other, and there's no direct way to control the perception of Green without adjusting U and/or V. In RGB you can correct any color without affecting the others. You can also adjust a specific range of colors, which is really limited in YUV. An RGB gradation curve can let you do things such as making a "notch" filter: for example, you can change only the RGB values of a color between RGB 10 and RGB 30 without changing anything else.

"Real HSV": Frankly, I don't know what the designer meant by "similar" or "real", but since you can't change between them it doesn't seem to matter. This control is similar to Tweak's hue and saturation settings, while "V" is really the same thing as "intensity" used by Donald Graft. The latter is basically an overall bright-and-dark "Contrast" setting. Simulates working in YUV.

Gamma - Red, Green, Blue
Levels - Dark, Middle, Light
Red Green Blue - Dark, Middle Light
Saturation - Dark, Middle, Light

As the labels indicate, these adjust specific values for Dark, Middle, and Light ranges. Adjust one range without much affect on the others. If your blacks look too blue, you can lower dark Blue to correct it without affecting the other two colors or the other color ranges. If lowering that dark color makes your blacks look "too dark", raise the Dark Levels a bit. Gamma is somewhat different: like ColorYUV's gamma_y, this is another control that works a "stretch" or "contraction" of values on either side of the RGB 128 midpoint.

You can think of the Gamma and Levels controls as kinds of Brightness and Contrast controls that are more specific in their action. By now you probably know that "Brightness" really adjusts black levels, and "Contrast" really adjusts highlight levels.

Sharp Smooth - Preprocess, Darm H & low, Postprocess (what do those things mean and do?). When I adjust the preprocess slider the program crashes.

Yes, it does. Don't use it. This plugin was designed for earlier versions of VDub. The other controls are OK. I used to have a couple of other VDub sharpeners that crashed as well.

There's also a Gamma is "Preprocess or Postprocess" button for every setting. What is the difference in these and what should I choose?

PreProcess, PostProcess etc. work only on the Sharp/Smooth control; they do nothing with the other controls. The setting tells the plugin to smooth, sharpen, or do some noise reduction before working with the other color stuff (preprocess) or after (post porocess).

What does HSV stand for? Are there any need to use Tweak before or after using ColorMill?

Hue, Saturation, and Value (or "Intensity" in some apps). You can use Tweak whenever you want, as long as the colorspace is set to YUV. You would use Tweak in YUV to especially adjust dark and bright levels before going to RGB, to prevent crushing and clipping. In fact you'd likely have to correct those levels with Tweak and/or ColorYUV whether you work in RGB or not.

What does the keep brightness and bad source options do? I ticked and unticked keep brightness but it made no change to the picture.

Yes they do. But if you haven't changed anything else, they do nothing. Let's say you want to reduce or increase the amount of green or some other color. In RGB, changing chroma values also changes luma values. It might or might not be necessary to restore the brightness ranges after making that change, so the "Keep Brightness" control attempts to maintain the original brightness values by (a) increasing the other two colors if you decrease any one color, or (b) decreasing the brightness of two colors if you increase a third color. I find that this feature just gets in the way and is unpredictable. Don't use it unless it helps. The "Bad Source" button seems to make a little pump in saturation. Don't use that one, either.

In any graphics application and in any colorspace, you should be working with pixel value samplers and with histograms. You can still make adjustments without those tools, but it's a bit frustrating because you're really guessing why colors and levels don't look "right" -- and because the human eye "knows" if something isn't correct, but doesn't always know "why".

You're basically asking somebody to write a book on color processing for you.

Yep.

[VideoFanatic]

OK thanks. Is there no way to set a setmtmode for each filter instead of having it for the whole file? I tried Jagabo's method of having Setmtmode(5) before ColorMill but it didn't work and crashed my encoder.

[themaster1]

read my post again:
https://forum.videohelp.com/threads/351041-How-do-I-use-ColorMill-with-Avisynth-and-GUI...=1#post2202713

[sanlyn]

I'm not proficient with MT modes, don't use them because of the multi-stage workflow I've stayed with. But I do know that not everything is compatible. If I have to do some touchy color correction I do the denosing/repair in one step, save it, then work other refinements later. Trying to do everyhting in one step, especially with many changes in complex scripts, is too painful. Work one step, save it, then bring up the saved work and add new steps without running and re-running everything over and over again.

If everything else works in MT modes, I'd just save that file and bring in another script or process separately. I've had to do that quite often. If you're going to use an RGB import in Avisynth, you have to go to RGB anyway and revert back to another colorspace for further work or encoding. IMHO that's too much of a hassle for tweaking a one-step script.

[VideoFanatic]

read my post again:
https://forum.videohelp.com/threads/351041-How-do-I-use-ColorMill-with-Avisynth-and-GUI...=1#post2202713

Jagabo already said after that post that Avisynth isn't case sensitive. I tried it before he said that and it made no difference.

I'm not proficient with MT modes, don't use them because of the multi-stage workflow I've stayed with. But I do know that not everything is compatible. If I have to do some touchy color correction I do the denosing/repair in one step, save it, then work other refinements later. Trying to do everyhting in one step, especially with many changes in complex scripts, is too painful. Work one step, save it, then bring up the saved work and add new steps without running and re-running everything over and over again.

If everything else works in MT modes, I'd just save that file and bring in another script or process separately. I've had to do that quite often. If you're going to use an RGB import in Avisynth, you have to go to RGB anyway and revert back to another colorspace for further work or encoding. IMHO that's too much of a hassle for tweaking a one-step script.

Are you talking about saving to lossless video with the 1st scrip and then running the 2nd script on that to save the final version as a compressed file?

[sanlyn]

Yes. Color balancing or grading and such refinements are a separate step anyway and usually the last step, and requires some trial and error. Also, many first-stage denoise and repair steps will often make changes in color balance/levels. Why keep re-running the first steps? Get Step 1 in shape and working, then save it as YV12 or whatever with lagarith or huffyuv, etc. Then open that job, convert to RGB, do what has to be done, and save the new output however you want. Re-running stuff like MCTemporalDenosie or QTGMC over and ov er again when it's already working correctly just takes a lot of extra time and effort.

[VideoFanatic]

middle point -> no idea what it is, never used and sharp/smooth i never use aswell

the best thing to do is use histograms in avisynth , the ones i use most of the time are:
Histogram(int2mode(1))
HistogramRGBParade()
HistogramRGBLevels
HistogramCMYLevels(range=true)

I don't understand how a histogram would help me when it's me who needs to decide what setting to use to change the colours and brightness, etc?

[VideoFanatic]

Are there no other plugins that I can use apart from ColorMill that will allow me to fine tune the brightness and colours, etc? I looked at the Avisynth wiki page here and I couldn't see anything apart from Tweak and RGBAdjust. RGBAdjust allows me to adjust the green red and blue levels and Tweak allows me to adjust the brightness, colour saturation, contrast and hue. However occasionally I get a problem where my video is too bright but lowering the contrast/brightness makes a part of the video too dark while the other parts are fine. I need something that will allow me to adjust the brightness and contrast more finely.

[themaster1]

Get yourself a copy of sony vegas and use its vectorscopes, that should get you started
tuto: http://www.youtube.com/watch?v=R6jO3ui_FbY&feature=my_favorites&list=FLkdfLPEpmp_VRaJBLnn8iFw

You can decide whatever color tweaks you want on a given monitor they may not render the same on another because of 2 things:
- your eyes fool you (human eyes aren't very sensitive to colors contrary to luma/luminance)
- monitor must be calibrated


Hence the use of special tools like vectorscopes / histograms

[jagabo]

AviSynth: Levels(), ColorYUV()
VirtualDub: Levels, Gradation Curves

[sanlyn]

I don't understand how a histogram would help me when it's me who needs to decide what setting to use to change the colours and brightness, etc?

As Master1 stated (and I'm repeating myself):

Human eyeballs aren't that accurate. Your brain can sense when colors or levels are not correct one way or another, but even a well-trained professional eye isn't precise enough to know exactly what correction is needed. Is something too red because there's too much red, or because there's not enough cyan ? (And, yes, there's a difference). You look at a histogram, you read pixel values. Every advanced hobbyist works with these tools. It's not rocket science, and it doesn't take long to get the hang of it. But if you don't take a closer look into what can help, you're just guessing about the visual clues in images.

Also, an uncalibrated display can fool you. A display that's too bright can make you think your image needs more saturation or should be darker, when in fact the opposite is likely the case. A monitor whose gamma is out of whack can make you think your colors are "off".

At this point no one knows what you're trying to correct. If you're referring to colors we see in the DVD sample you posted earlier, there's nothing wrong with the color in that clip.