Where can I see the Y values? Right now moving my mouse around, and use the Cursor position (so if y position is 35, I just 255-35 to know whites are at 225).The blacks are at Y=45 and whites at Y=220.
Also, I played around with the Histogram filter in VirtualDub yesterday. You can sample all the video. Not the specific frame. It's hard to do smart adjusts based on specific frame, I have to look in couple of frames until I found the one where whites are above 220. Can I do something similar with Avisynth?
OK. So now it makes much more sense. So the 4 values are pretty easy - because they are direct outputs based on the graph. The gamma will require experiment and seeing what works best for your eye?you use Levels(45, 1.0, 220, 16, 235)
So Sharc, I wondered why you picked 12/255? Did you perhaps caught a frame where higher whites/lower blacks? Also - it seems like Sharc is adjust based on 0-255 and jagabo on 16-235? last time I was reading about that, 0-255 was better for RGB and PC Monitor/Video Cards and 16-235 was more standard in the TV/Film industry?levels(12,1.0,255,0,255,coring=false)
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 31 to 57 of 57
Thread
-
Last edited by Okiba; 3rd Oct 2020 at 01:21.
-
levels(12,1.0,255,0,255,coring=false)So Sharc, I wondered why you picked 12/255? Did you perhaps caught a frame where higher whites/lower blacks? Also - it seems like Sharc is adjust based on 0-255 and jagabo on 16-235? last time I was reading about that, 0-255 was better for RGB and PC Monitor/Video Cards and 16-235 was more standard in the TV/Film industry?
I may have misunderstood that you want to watch it on PC monitor and keep it at 4:2:2 (which is not commonly supported by TVs) , so I went for the PC range 0 ..... 255. The resulting histogram looked ok, but maybe I should better have used the oscilloscope view.
Anyway, main purpose was to draw your attention to the 'levels' filter for further experimentation, rather than recommending specific values.Last edited by Sharc; 3rd Oct 2020 at 03:45.
-
I had a quick look at your results-italy.mp4 and found that the minimum Y was 12 using 'colorYUV(analyze=true)'.
'colorYUV(analyze=true)
I may have misunderstood that you want to watch it on PC monitor and keep it at 4:2:2 (which is not commonly supported by TVs)
Anyway, main purpose was to draw your attention to the 'levels' filter for further experimentation, rather than recommending specific values. -
I don't really know. There are some hints how VDub and Avisynth 'levels' correspond with each other in the Avisynth wiki. Or someone else may help.
'colorYUV(analyze=true)
I didn't know that trick. If its precise - would it be easier compared to check the black/white levels manually on the graph?
Also keep in mind that captions like date and time will influence these values. So one may decide to crop these off for analysis .....
I'm not really. I plan to watch it on both PC monitor, and TV. So 16-235 not to clip things on the TV. Was that a wrong though?
If you find some time you may want to study 'ColorYUV' in more detail .....Last edited by Sharc; 3rd Oct 2020 at 06:24. Reason: typos
-
What you guys aren't understanding is that those very low black, and very high white values aren't really parts of the image. They are noise and oversharpening halos. You don't need to retain those. And RGB values read off the screen in VirtualDub are not Y values. The default in VirtualDub is are rec.601 conversion (Y=16-235 to RGB=0-255 contrast expansion). What you need to look at are large areas of black and white. Note that the waverform monitor from TurnRight().Histogram().TurnLeft() is 256 pixels tall -- one pixel for each possible Y value. So you can count the pixels to get Y values:
[Attachment 55231 - Click to enlarge]
All the little peaks below 45 and over 220 are oversharpening halos and noise.
Not every frame will have full black and full white elements. So you also have to use your judgement about whether the brightest and darkest parts of the image/shot/video (even in very large patches) are reflective of the black and white levels. -
^^^
Thank you for these explanations, jagabo. I wasn't aware of the y/pixel calibration of the waveform monitor, using a pixel ruler. Now the "Loose" values of colorYUV(analyze=true) attempting to discard noise and halos make sense. -
Here's an example that shows oversharpening halos somewhat like those created by VHS decks' sharpening filters:
Code:# create black/white vertical bars, 16 pixel thick bars, 256 pixels total BlankClip(width=32, height=256) StackHorizontal(last, last.Invert()) StackHorizontal(last, last, last, last) ConvertToYV12() ColorYUV(off_y=-45).ColorYUV(off_y=45) # force blacks to 45 ColorYUV(off_y=35).ColorYUV(off_y=-35) # force whites to 220 # show unfiltered bars on the left, sharpened on the right StackHorizontal(last, UnsharpMask(radius=3, strength=50, threshold=0)) HistogramOnBottom() ConvertToRGB(matrix="pc.601") # so RGB values on screen reflect Y values from the video
[Attachment 55232 - Click to enlarge]
The left half of the image is simply vertical bars of brightness Y=45 and Y=220. The right half is the same bars after a sharpening filter. After sharpening the middle of the dark bars remains at 45 but as you get close to the white bars the values drop to 35, then 25, the 15. The middle of the white bars remains at 220, but as you approach a black bar the values rise to 229, 239, and 249. Those values aren't part of the original image, they were created by the sharpening filter. -
Your final encode should be 4:2:0 (YV12) though for best compliance with various playback scenarios/devices.
The default in VirtualDub is are rec.601 conversion (Y=16-235 to RGB=0-255 contrast expansion)
All the little peaks below 45 and over 220 are oversharpening halos and noise.
So you also have to use your judgement about whether the brightest and darkest parts of the image/shot/video (even in very large patches) are reflective of the black and white levels.
So that's makes me wonder. First of all. Does "Levels" means brightness/contrast settings?
And also, I will try to check a very dark scene, and a very bright scene. If the values are still 45-220, is it safe to say Levels is actually a global option (for specific setup) and I can use on the generic QTGMC script? Up until now I was sure brightness/contrast are per scene option. -
Chroma subsampling 4:2:2 is basically better (double vertical color resolution) than 4:2:0, but at the cost of storage space. 4:2:2 is the right format for capturing interlaced VHS with Huffyuv. Leave it at 4:2:2 as long as possible along your workflow. Some (avisynth-)filters may however still support 4:2:0 only. You would have to check the individual filters.
SW players will normally play 4:2:2 without issues.
4:2:0 is standardized for DVD, AVCHD, Blu-ray, PAL-DV to name a few. HW players may reject 4:2:2 sources as "unsupported format" or similar. My TV also doesn't play 4:2:2 mp4 files for example, only 4:2:0. Disc authoring SW (if this should matter) may als reject 4:2:2 footage.
So I wouldn't prefer 4:2:0 over 4:2:2, with the exception of broader playback compatibility and lower file size - if this should matter.
The x264 encoder supports both 4:2:2 and 4:2:0 subsampling formats, default is 4:2:0.Last edited by Sharc; 3rd Oct 2020 at 10:19.
-
Pretty much all commercial distribution is 4:2:0, DVD, BD, online streaming. Computers are about the only thing that will play 4:2:2 h.264. Most other devices will not play 4:2:2 h.264. I believe more modern devices will play 4:2:2 h.265. You typically get a black screen (only audio) or an error message (no audio or video) when 4:2:2 isn't supported.
Yes that histogram is of Y values and the red areas are Y<16 and Y>235.
That is why histograms aren't very useful. You don't know what parts of the picture are outside the range. AlsoVirtualDub's capture histogram is not linear, I believe it's logarithmic. So infrequent values are exaggerated.
For example, with VHS caps the left/right black borders are typically at full black but the active picture may have elevated black levels. If you look only at a histogram you don't know for sure if the red ends are because of the black borders or the active picture.
This is where you have to use your own judgement. If one shot out of a hundred has blacks at 43 and the rest are all at 49 you'll probably want to use 49. Unless that one shot is critical for some reason. If half the shots are 43 and the other half are at 49 you may want to go with 43. Or maybe an average of 46 -- depending on how important the dark detail is.
Also, since you are doing this as a batch pre-processing step, you can let some blacks fall a little below Y=16, and some whites a little above Y=235. Because you can always fix them with your later processing. Or just delay all the levels/colors adjustments for later.
Yes.
Yes, you can achieve the same adjustments with the cont and bright settings in Tweak() or gain_y and off_y settings in ColorYUV(). Though AviSynth's Levels() filter also does the same to the chroma (increasing or decreasing color saturation) so you also have to use sat in Tweak() and cont_u, cont_v in ColorYUV().
It's both. You want to make sure your caps fall within a reasonable range. Then, assuming it's needed and you care, adjust shot by shot for the final output. -
Pretty much all commercial distribution is 4:2:0, DVD, BD, online streaming.4:2:0 is standardized for DVD, AVCHD, Blu-ray, PAL-DV to name a few.
For example, with VHS caps the left/right black borders are typically at full black but the active picture may have elevated black levels.
It's both.
So update batched script:
Code:SetFilterMTMODE("QTGMC", 2) AviSource("Loseless.avi") ConvertToYV16(interlaced=true) AssumeTFF() QTGMC(Preset="Slower", EdiThreads=3) crop(20, 6, -20, -6) Levels(X, 1.0, X, 16, 235) ChromaShiftSP(Y=3) MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp(20).Sharpen(1.0).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height)) Prefetch(3)
-
No.
If you are going to make levels changes later it may make more sense to do it all later rather than a rough change in you pre-processing and finer changes later. Each time you adjust levels/saturation you will get some quantization errors. On the other hand making the rough changes before QTGMC() will help mask the quantization errors at that step. -
Are the quantization errors permanent? so changing levels multiple times will make the video looks worse over time?
Due to the huge amount of videos, I probably not going to tweak all of them, Just couple of Key videos later (for example, my old folks wedding). Probably when I also going to tweak colors.
By the way, using 45/220 makes everything a bit "too" dark. That's because I also need to fix other stuff like saturation?
And sure, thanks for the tip. I now doing every tweaking before QTGMC:
Code:SetFilterMTMODE("QTGMC", 2) AviSource("Loseless.avi") ConvertToYV16(interlaced=true) crop(20, 6, -20, -6) Levels(X, 1.0, X, 16, 235) ChromaShiftSP(Y=3) MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp(20).Sharpen(1.0).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height)) AssumeTFF() QTGMC(Preset="Slower", EdiThreads=3) Prefetch(3)
-
You may want to add add 'dither=true', like
Code:Levels(X, 1.0, X, 16, 235,coring=false,dither=true)
Maybe I can write a script that auto select the cropping value (up until end of the masking bar).
http://avisynth.nl/index.php/AutoCrop
http://avisynth.nl/index.php/RoboCrop
Edit: Hmmm, I probably misunderstood. You mean finding the 'X' values for the levels, right?Last edited by Sharc; 4th Oct 2020 at 02:58.
-
For automatic borders cropping you could try AutoCrop or RoboCrop
dither=true
coring=false
coring was created for VirtualDub compatibility, and it remains true by default for compatibility with older scripts. In the opinion of some, you should always use coring=false if you are working directly with luma values (whether or not your input is 16d-235d). -
I don't have the straight answer to your question, but you could visualize the effect of the 'coring' side-by-side with a script like
Code:v1=AVISource("your.avi") v1=v1.converttoyv16(interlaced=true).crop(20,6,-20,-6) v2=v1.levels(42,1.0,220,16,235,coring=true).histogram() v3=v1.levels(42,1.0,220,16,235,coring=false).histogram() out=stackhorizontal(v2,v3) return out
Added:
You may also run jagabo's script of post #30 and change it to 'coring=true' in his 'function levels_gamma' to see what happens.Last edited by Sharc; 4th Oct 2020 at 05:49. Reason: Added
-
Generally you want to use coring=false to make sure illegal values are handled the same as legal values rather than being crushed. Note that the original levels filter in AviSynth worked in RGB after a rec.601 conversion. So all illegal levels (Y<16, Y>235) were crushed before the levels adjustments were applied (and crushed again after the adjustments are applied). The defaults in the AviSynth filter were to emulate that.
Regarding quantization errors here's an extreme example:
Code:Levels(45,1.0,220,127,129,coring=false) Levels(127,1.0,129,16,235,coring=false) GreyScale()
[Attachment 55243 - Click to enlarge]
With the first line the entire video is converted to only three Y values, 127, 128, and 129. With the second line those three values are spread out to 16, 126, 235. GreyScale() is used to remove the chroma to make the luma changes more obvious. All the in-between values that were lost in the first step cannot be recovered in the second step.
The dither option basically adds ordered noise to reduce this banding/posterization. It works ok for small levels adjustments. If you enable the option with the above sequence you'll see what it does.
If you think the black level is too low after the 45/220 adjustment you can change the values to give you what you want.
Code:Levels(45, 1.0, 220, 25, 235, coring=false)
Code:Levels(45, 1.5, 220, 16, 235, coring=false)
-
For automatic borders cropping you could try AutoCrop or RoboCropNiceeee. No need to reinvent the wheel! Will test those soon. I wonder why I works logically though. How does it know a line/row is perfectly black and is a black bar - or a completely dark video. I guess not many videos are 0 dark for a whole line/row though.
If you prefer automatisms you may also want to give 'AutoAdjust and 'AutoLevel' a try. Again, check the result carefully before you start the final encoding.
http://avisynth.nl/index.php/AutoAdjust
http://avisynth.nl/index.php/Autolevels -
For cropping I often open a script in VirtualDub2, use it's Crop filter to visually get the settings, then use those values in AviSynth's Crop() filter.
-
Thank you for explaining that guys. I added "coring=false" to the Levels section on the script.
All the in-between values that were lost in the first step cannot be recovered in the second step.
If you think the black level is too low after the 45/220 adjustment you can change the values to give you what you want.
AutoLevels seems to be early on development (2 versions, one of which is beta)
But AutoAdjust looks a bit more matured. Will be interesting to check it.
For cropping I often open a script in VirtualDub2, use it's Crop filter to visually get the settings, then use those values in AviSynth's Crop() filter. -
[QUOTE=Okiba;2597231]
AutoLevels seems to be early on development (2 versions, one of which is beta)
But AutoAdjust looks a bit more matured. Will be interesting to check it.
Code:autolevels(ignore_low=0.005,ignore_high=0.0003,autolevel=true,autogamma=false)
(Note: AutoLevels has its own parameters to ignore borders and other crud btw.)
Anyway, one would have to do more thorough tests, including fades, changing day/night scenes and so on....
It may not be possible to find 'one for all' settings for your videos.Last edited by Sharc; 4th Oct 2020 at 13:18.
-
I was simulating what would happen if you adjust levels in your QTGMC pre-filtering step, then load those pre-filtered clips into an editor and apply a a final brightness and contrast adjustment. I just skipped the save/reload step. And used outrageous adjustments to make problem visually obvious.
Say for example you apply a Y' = Y * 1.4 adjustment in one step, then later apply a Y' = Y * 1.3 adjustment in a second step. After the first step a pixel with a Y value of 1 becomes 1.4. But the video output is limited to integers. So that 1.4 is rounded down to 1. Then in the second adjustment that 1 becomes 1.3. Which rounds down to 1 again. But if you had applied Y' = Y * (1.4*1.3) in a single step you get 1.82 which rounds up to 2. So doing the two adjustments in one step is more accurate than doing it in two smaller steps. -
Had some time to play with all the toys you shared
RoboCrop is based on AutoCrop, and aim to be simpler (so for example you don't have to tell it explicitly to take a range of frames, it does that by default). However, I had a strange issue with it. I cropped the video manually, and then with RoboCrop and the results in the AvsPMod preview looked differently. I used AvsPmod to save a screenshot - and it seems identical, so it might just be a preview thingi. But I will do some more verification before using it by default.
I did noticed something else though. While cropping manually, I noticed that the masking I did wasn't perfect. So I crop two more pixels, so I indeed up with un-even cropping (18 to the left and 20 to the right, instead of 20/20 like I mostly have). This broke chroma sharping:
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp(20).Sharpen(1.0).nnedi3_rpow2(2 , cshift="Spline36Resize", fwidth=width, fheight=height))
But if you had applied Y' = Y * (1.4*1.3) in a single step you get 1.82 which rounds up to 2. So doing the two adjustments in one step is more accurate than doing it in two smaller steps.
I tried AutoLevels, and I was indeed unable to identify difference between the results of the plugin to the manual Level we set (45,230). -
-
-
-
I think I am good. As I mentioned, I have two type of Content. Personal family footage, of which I apply the generic script AND keep the loseless format. So in case It will end up just OK and need more tweaking, I can always do it from scratch on the HuffYuv file (so it will be a fresh Level modification). The other content is couple of old Cartoons/Talk shows my father would like to keep. I probably not going to keep Loseless version of those (because they are not that important and the space needed for this all project is already quite big). But because it's not the same setup - I'm going to manually adjust Levels. So they will be precise (as I will be checking the Histogram for each).
Phew. That was a very educational post for me. I learned a lot. Thank you Sharc, Lordsmurf, johnmeyer, poisondeathray and especially to jagabo who answered professional and with patience to every stupid question I had
Similar Threads
-
Qtgmc-GUI. A simple encoder for your Qtgmc scripts.
By ProWo in forum Video ConversionReplies: 17Last Post: 4th Mar 2023, 02:01 -
Help with QTGMC
By Xyena in forum Newbie / General discussionsReplies: 6Last Post: 22nd Sep 2020, 12:25 -
I think this just needs QTGMC...
By pooksahib in forum Video ConversionReplies: 2Last Post: 30th May 2019, 03:12 -
Help with QTGMC?
By attackworld in forum EditingReplies: 3Last Post: 26th Nov 2018, 21:57 -
A video edition software recommendation, basic but no so basic
By helpcito in forum EditingReplies: 11Last Post: 17th Feb 2018, 05:15