Stripes appear after Levels filter in avisynth script

[taigi]

After applying Levels through AviSynth on VirtualDub2, I’m seeing vertical stripes in my video. (These are not the same stripes I posted about on this forum five years ago.)

Sometimes, in other processing videos, those lines do not appear. I’m also running several filters in a single AviSynth script: QTGMC, DeHalo, and Santiag. After Levels, I change the frame using SplineResize

1 image: Captured with the camera lens cap closed.
2 image: Enhanced so the vertical stripes are more visible.

Any ideas on what could be causing this? Thanks in advance.

[Selur]

Could be caused by:
(Capture/GraphicCard) Driver issues.
Improper luma-plane handling during resizing/sharpening/color conversion.

You probably can get rid of them by lowering the luma resolution and then restore it.
I'm pretty sure I saw this before somewhere on the forum.

General idea:

clip = last
# Extract luma
Y = clip.ConvertToY8()
scaleFactor = 2 # if two isn't enough try 4 or 8
# Lowering horizontal resolution; you might want to try different resizers
Y_half = Y.BilinearResize(Y.Width() / scaleFactor, Y.Height())
#  Restoring original resolution; you might want to try different resizers
Y_restored = Y_half.BilinearResize(clip.Width(), clip.Height()) 

# Convert luma back to YV12 by adding dummy chroma  (if you source isn't YV12 adjust this)
Y_restored_yv12 = ConvertToYV12(Y_restored)

# Replace chroma from original clip
MergeChroma(Y_restored_yv12, clip)

Cu Selur

Ps.: Here's also a Vapoursynth version:

# Extract luma
luma = core.std.ShufflePlanes(clip, planes=0, colorfamily=vs.GRAY)

# Lower horizontal resolution
width = luma.width
scalingFactor = 2
luma_half = core.resize.Bilinear(luma, width // scalingFactor, luma.height)

# Scale back to original width
luma_restored = core.resize.Bilinear(luma_half, width, luma.height)

# Merge modified luma back with original chroma
clip = core.std.ShufflePlanes([luma_restored, clip], planes=[0, 1, 2], colorfamily=vs.YUV)

PPs.: rotation the image by 90°, and applying some smoothing combined with a combing mask and then rotating the image back might retail more details.

[VoodooFX]

Stripes appear after Levels filter in avisynth script

Make sure you use the latest Avisnyth version (currently 3.7.5).

Provide a demo to reproduce the issue.

[taigi]

I'm using this script:

video = FFMPEGSource2("file.avi", atrack=1)
video = video.Levels(12, 1.0, 255, 16, 235, coring=false, dither=true)
return video
Prefetch(2)

Anything wrong with it?

[VoodooFX]

I'm using this script:

video = FFMPEGSource2("file.avi", atrack=1)
video = video.Levels(12, 1.0, 255, 16, 235, coring=false, dither=true)
return video
Prefetch(2)

Anything wrong with it?

Try LWLibAvVideoSource("file.avi"), that's LSMASH.
Remove "return".

[Selur]

Remove "return".

or just switch the two last lines for Prefetch to actually do something

SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
video = FFMPEGSource2("file.avi", atrack=1)
video = video.Levels(12, 1.0, 255, 16, 235, coring=false, dither=true)
Prefetch(2)
return video

I doubt it's a problem of the source filter => best share a short sample of your source.

Cu Selur

[taigi]

Source file is typical. I'm having stripes on other similar files too. I think problem is in the whole script. Yes two dehalos:

video = FFMPEGSource2("1.avi", atrack=1)

video = video.QTGMC(TR0=2, TR1=1, TR2=0, Rep0=1, Rep1=0, Rep2=2, DCT=5, ThSCD1=400, ThSCD2=150, SourceMatch=2, Lossless=1, Sharpness=0, Sbb=0, MatchPreset="slow", NoiseProcess=1, NoiseRestore=0, NoisePreset="slow", StabilizeNoise=true, NoiseTR=1, NoiseDeint="true", Sigma=2.0)

video = video.FineDehalo(rx=1.5, ry=1.5, thmi=80, thma=128, thlimi=50, thlima=100, darkstr=1.0, brightstr=1.0, showmask=0, contra=0.0, excl=true)

video = video.santiag(strv=3, type="nnedi3", nns=4, threads=0, nsize=4, halfres=false, scaler_post="LanczosResize")

video = video.FineDehalo(rx=1.0, ry=1.0, thmi=80, thma=128, thlimi=50, thlima=100, darkstr=1.0, brightstr=1.0, showmask=0, contra=0.0, excl=true)

video = video.Levels(12, 1.0, 255, 16, 235, coring=false, dither=true)

video = video.Spline64Resize(1024,576)

Prefetch(2)

return video

[VoodooFX]

Prefetch(2)

return video

I doubt that this works, use it like I wrote you.

Source file is typical.

That doesn't mean to us anything.
Provide a sample to reproduce the issue.

[Selur]

I doubt that this works, use it like I wrote you.

Works for me for years,..

your suggestion

video
Prefetch(2)

should be the same as:

last=video 
Prefetch(2)
return last # since there is no return

Cu Selur

[VoodooFX]

Works for me for years,..
...
should be the same as:

There is no "last" in his script, so I doubt that it works for you too.

EDIT:

Oh, I see, it's you who suggested him that, I guess you are not that secret/unknown avs expert[s] then.

I doubt it's a problem of the source filter

Read there [starts from that post I think] -> https://forum.doom9.org/showthread.php?p=1975646#post1975646

[Selur]

There is no "last" in his script, so I doubt that it works for you too.

Last when I read the Avisynth code it always used "last" unless only direct variables are used and returned.

Read there [starts from that post I think] -> https://forum.doom9.org/showthread.php?p=1975646#post1975646

I don't really see the connection to from InpaintDelogo to this thread.

=> totally lost, but the vertical line problem is useless to discuss without a sample

Cu Selur

[VoodooFX]

Last when I read the Avisynth code it always used "last" unless only direct variables are used and returned.

Show me where there are no direct variables in his or your script.

I don't really see the connection to from InpaintDelogo to this thread.

If you follow the conversation there then you'll see that it boils down that there is some bug in FFMS2 + VirtualDub/StaxRip combo or FFMS2 is not frame accurate.

[Selur]

Show me where there are no direct variables in his or your script.

You are missing the '.. and returned' part.

With:

video
Prefetch(2)

like you suggested, the 'video' line would be the same as 'last = video'
(same as "Levels()" actually is "last = Levels(last)")
and since at the end of the script nothing got returned, last will be returned.
so

video
Prefetch(2)

is actually

last = video
Prefetch(2)
return last

If you follow the conversation there then you'll see that it boils down that there is some bug in FFMS2 + VirtualDub/StaxRip combo or FFMS2 is not frame accurate.

But frame accuracy should not matter with Levels() since that it only works on single frames independently. So unless the source filter has another bug which messes up the luma plane it should not be the cause of the vertical stipes.

Cu Selur

[VoodooFX]

Read it again: "...his or your script."

I know that my example has "last".
Why are you even trying to prove that my example has "last", when I wrote that his and your examples doesn't have it?

But frame accuracy should not matter with Levels() since that it only works on single frames independently. So unless the source filter has another bug which messes up the luma plane it should not be the cause of the vertical stipes.

Yes, but OP wrote that he's using QTGMC.

[Selur]

like I wrote without a sample of the source this is moot,...

[jagabo]

The script isn't causing the vertical stripes. It's just making existing stripes visible. Your VCR, cables, or capture device is causing them.

[taigi]

The script isn't causing the vertical stripes. It's just making existing stripes visible. Your VCR, cables, or capture device is causing them.

Yes, I had stronger vertical stripes in my orig. footage before, and applied Santiag.
But after Levels stripes seem to reappear, in a different form.
Should I apply the same Santiag again?
Those stripes are from my broken DV camera. I captured it directly from the DV camera using a FireWire cable with Windows Vista. I tried capturing with different DV cameras, but got the same result.

[taigi]

Came across the VirtualDub2 “Levels” filter.

Is it reliable for production work, or mainly a legacy/utility tool?

Context: Windows 7, mostly DV sources.

Thanks.

[poisondeathray]

Came across the VirtualDub2 “Levels” filter.

Is it reliable for production work, or mainly a legacy/utility tool?

Context: Windows 7, mostly DV sources.

Thanks.

vdub2 levels filter is not ideal because: does not work at higher bit depths , has no dithering options, and most importantly coring is permanently set to true - which can causing clipping of useful data

For example, many DV recordings have usable overbrights in the 235-255 range. Typically they record 16-255. Clipping 0-15, 236-255 before you even do anything is a shame

(A workaround for the clipping if you must use vdub2 levels filter is to interpret as full range in the decoding options ... but then the black point, gamma change more than they might need to. So you make counter adjustments and manipulations and that can cause more 8bit rounding errors and banding)

[rgr]

After applying Levels through AviSynth on VirtualDub2, I’m seeing vertical stripes in my video. (These are not the same stripes I posted about on this forum five years ago.)

Because these lines are in the source file, you're only brightening the ones that slightly exceed the limit (luma=12). Use this instead of Levels:
ApplyGradationCurves(lumapoints="0,0,12,16,255,235 ")
http://avisynth.nl/index.php/GradationCurve

Maybe it would be a good idea to denoise the source file as well.

And once saved as a limited range file and played correctly, these lines wouldn't be visible. It's hard to judge without the source file.

Edit: Well, this probably won't work, because you only want to lighten part of the "black".

[rgr]

I tried capturing with different DV cameras, but got the same result.

That won't work -- those lines are already in the compressed file you have on the miniDV cassette. It would probably be better to copy it through the AV output, but that would result in lower overall quality.

[rgr]

...

Wouldn't it be better to just throw away the junk data below luma 16?

ApplyGradationCurves(lumapoints="16,16,255,235") # I prefer ApplyGradationCurve(lumapoints="16,16,64,64,128,128,192,192,255,235")
Limiter(16,235,16,240)

Maybe the camera wasn't broken, but just bad (poor compression) and was messing with the data that was out of range? You are trying to recover video data below luma 16, which theoretically does not exist. And anything could be there.

Edit: delete all Santiag filters etc. They are unnecessary.