floating chroma shift issue?

[Selur]

Second parameter is float "varC" = variance chroma, which should be sigma_vU, sigma_vV in vsTCanny. Not sigmaU.
more something like:

v2= vsTCanny(v2, sigmaY=50.0, sigma_vU=50.0 + gblur, sigma_vV=50.0 + gblur, mode=-1)

(using 50 instead of 7)

[cinephil]

Well, "=50" - rises the highlights (unlike "=7"), but either way not resolve "border" issue.

Not sure what "variance chroma" mean exactly, but I guess it somewhat different from "Standard deviation of vertical gaussian blur" - which in vsTCanny is sigma_vY, sigma_vU, sigma_vV (and sigmaY, sigmaU, sigmaV - Standard deviation of horizontal gaussian blur.)

[Selur]

It should be the deviation for U and V as I understood it. )
I assume you did simplify your script to stop after the ContrastMask calls to be sure this is not caused by something else, right?

[cinephil]

I assume you did simplify your script to stop after the ContrastMask calls to be sure this is not caused by something else, right?

Well, if I will not tweak after ContrastMask - the picture will be wash out and I won't see any difference.
Right now it look like this:

old ContrastMask

SeparateFields()
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
Weave()

ConvertToYV12(matrix="rec601", interlaced=true)

SeparateFields() 
even = SelectEven(last).ContrastMask(enhance=3.0)
odd = SelectOdd(last).ContrastMask(enhance=3.0)
Interleave(even, odd)
Weave()

ConvertBits(16) 

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0)
Tweak(bright=3, coring=false)
Tweak(cont=0.9)
Tweak(Hue=-16)

FFT3DFilter(bt=-1, sharpen=0.3, interlaced=true)

ConvertBits(bits=8, dither=1)

new ContrastMask

ConvertBits(16) 

SeparateFields()
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
Weave()

SeparateFields()
even = SelectEven(last).ContrastMask(enhance=3.0)
odd = SelectOdd(last).ContrastMask(enhance=3.0)
Interleave(even, odd)
Weave()

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0)
Tweak(bright=3, coring=false)
Tweak(cont=0.9)
Tweak(Hue=-16)

FFT3DFilter(bt=-1, sharpen=0.3, interlaced=true)

ConvertBits(bits=8, dither=1)
ConvertToYV12(matrix="rec601", interlaced=true)

[Selur]

With 'old ContrastMask' you used MergeChroma on 8bit, not you use it on 16bit.

As a general side node:
Your field handling seems odd / wrong to me, can you explain why you do it that way?

• Convert to 16bit (okay)
  Unsure why since afaik. none of the filters you use here does benefit from it, so it's just a slow-down)
• You separate the fields. (okay)
• You do some chroma adjustments on the separated fields. (okay)
• You weave the fields and then separate them again => this seems wrong. Can you explain why you do that?
• You apply ContrastMask on the fields. (okay)
• You re-interlace the fields using Interleave(even, odd) and Weave(). (okay)
• You change the colors of the interlaced content. (okay)
• You apply FFT3DFilter it the interlaced content. (okay)
• You convert to 8bit, while dithering (okay)
• You convert to YV12 (okay)

Personally I would, deinterlace the content, apply the filters and at the end re-interlace the file, instead of working on the separated fields.
The first step for me would be deinterlacing and the last step would be the re-interlacing.

Cu Selur

[cinephil]

With 'old ContrastMask' you used MergeChroma on 8bit, not you use it on 16bit.

MergeChroma part handling some chroma ringing issue, I can't lose that step.

As a general side node:
Your field handling seems odd / wrong to me, can you explain why you do it that way?
[*]Convert to 16bit (okay)
Unsure why since afaik. none of the filters you use here does benefit from it, so it's just a slow-down)

It's for dithering while converting to 8bit, - to remove the banding.

[*]You weave the fields and then separate them again => this seems wrong. Can you explain why you do that?

I think, there was something between this, but not anymore.

Personally I would, deinterlace the content, apply the filters and at the end re-interlace the file, instead of working on the separated fields.
The first step for me would be deinterlacing and the last step would be the re-interlacing.

There is mixed content of film and video (true 29.97 fps) [but not in this 03.avi clip]. So to avoid complication/problems (and not to lose video frames) I stay interlaced.

[Selur]

MergeChroma part handling some chroma ringing issue, I can't lose that step.

You should do that in the original bit depth,...

It's for dithering while converting to 8bit, - to remove the banding.

use a debanding filter instead

There is mixed content of film and video (true 29.97 fps) [but not in this 03.avi clip]. So to avoid complication/problems (and not to lose video frames) I stay interlaced.

bob deinterlace then. You are not staying interlaced, why you do, atm. isn't keeping the content interlaced,..

[cinephil]

So it will be something like this?

AssumeTFF()
Bob(0.0, 1.0)

MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))

ContrastMask(enhance=3.0)

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0) #cont_y=46
Tweak(bright=3, coring=false)
Tweak(cont=0.9)
Tweak(Hue=-16)

FFT3DFilter(bt=-1, sharpen=0.3)

neo_f3kdb(range=31, keep_tv_range=true, grainY=0, grainC=0, output_depth=8)

ConvertToYV12(matrix="rec601")

AssumeTFF.SeparateFields().SelectEvery(4,0,3).Weave()

I will do some testing tomorrow. But debanding filter, as I see on the Histogram - not doing good enough to replace ->16bit->8bit(dithering).

[Selur]

I would probably use QTGMC in fast mode, but yeah, something like that.

[cinephil]

MergeChroma part handling some chroma ringing issue, I can't lose that step.

You should do that in the original bit depth,...

That definitely fix the "border" issue!

There is mixed content of film and video (true 29.97 fps) [but not in this 03.avi clip]. So to avoid complication/problems (and not to lose video frames) I stay interlaced.

bob deinterlace then. You are not staying interlaced, why you do, atm. isn't keeping the content interlaced

With BOB the picture looks softer. I thought it would be the other way around.
I bumped sharpen from 0.3 to 0.5 - to get similar to the old script look.
* Although sharpen- 0.5 is a bit too much (bring more noise), - 0.4 is fine, I guess...

AssumeTFF()
Bob(0.0, 1.0)

MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))

ConvertBits(16)

ContrastMask(enhance=3.0)

ColorYUV(gamma_y=-35, cont_y=46, gain_y=-10, off_y=0) #cont_y=46
Tweak(bright=3, coring=false)
Tweak(cont=0.9)
Tweak(Hue=-16)

FFT3DFilter(bt=-1, sharpen=0.5)

AssumeTFF.SeparateFields().SelectEvery(4,0,3).Weave()

ConvertBits(bits=8, dither=1)
ConvertToYV12(matrix="rec601")

[Selur]

That definitely fix the "border" issue!

so, that was not caused by ContrastMask

With BOB the picture looks softer.

]
Like I said, I would not use the simple default Bob filter here. Also for light sharpening, try CAS.

[cinephil]

Turns out that BOB is not that bad. But sharpen with FFT3DFilter work better after reinterlacing.
------------------
Now getting back to the clip from my first post - after taken care of the chroma bleeding, I'm still trying to fix halo/ghost issue.
With FineDehalo it's look better, but I still have one bright halo/ghost.

[Attachment 82903 - Click to enlarge]

Is it possible to get rid of that line without degrade picture too much furter?

AssumeTFF()
Bob(0.0, 1.0)

FineDehalo2(hconv = "-1 -1 1 1 40 1 1 -1 -1", vconv = "-1 -1 1 1 40 1 1 -1 -1", showmask=0)
FineDehalo(rx=3.0, ry=2.0, thmi=30, thma=158, thlimi=50, thlima=100, darkstr=1.2, brightstr=1.5, showmask=0, contra=0.0, excl=true)

ConvertBits(32)
ConvertToPlanarRGB(matrix="Rec601")
mlrt_ncnn(network_path="1x_BleedOut_Compact_300k_net_g.onnx")

ConvertToYUV444(matrix="Rec601")

ColorYUV(gamma_y=-35, cont_y=48, gain_y=-10, off_y=0)
Tweak(bright=12)

AssumeTFF.SeparateFields().SelectEvery(4,0,3).Weave()

FFT3DFilter(bt=-1, sharpen=1.0, interlaced=true)

ConvertBits(8)
ConverttoYv12(matrix="Rec601", interlaced=true)

[jagabo]

FineDehalo doesn't work well with wide halos. Try something like dehalo_alpha(rx=7, ry=1, BrightStr=0.8, DarkStr=0.8). Use an edge mask to limit it to the strongest edges (thus protecting the rest of the image).

Even dehalo_alpha works better with smaller halos. So try downscaling horizontally, dehalo_alpha, upscale horizontally and an edges mask.

[cinephil]

Thanks, jagabo!

Maybe I do something wrong with dehalo_alpha (or with edge mask), since don't got much improvement using it?

Spline36Resize(width/2, height)
emask=mt_edge(mode="prewitt", thy1=0, thy2=255, U=2, V=2)
Overlay(last, dehalo_alpha(rx=2.0, ry=1, BrightStr=0.8, DarkStr=1.2), mask=emask)
Spline36Resize(width*2,height)

But I got much better results from FineDehalo (although don't eliminated this bright line completely).

Spline36Resize(width/2, height)
FineDehalo2(hconv = "-1 -1 1 1 40 1 1 -1 -1", vconv = "-1 -1 1 1 40 1 1 -1 -1", showmask=0)
FineDehalo(rx=2.0, ry=1.0, thmi=60, thma=128, thlimi=50, thlima=100, darkstr=1.2, brightstr=0.8, showmask=0, contra=0.0, excl=true)
Spline36Resize(width*2,height)

[Attachment 82954 - Click to enlarge]

[Selur]

You used a different rx value than jagabo suggested.

[cinephil]

You used a different rx value than jagabo suggested.

Well, rx=7 was suggested for no downscale-upscale involving. And it not look good.

emask=mt_edge(mode="prewitt", thy1=0, thy2=255, U=2, V=2)
Overlay(last, dehalo_alpha(rx=7, ry=1, BrightStr=0.8, DarkStr=0.8), mask=emask)

[Attachment 82956 - Click to enlarge]

With downscale-upscale

Spline36Resize(width/2, height)
emask=mt_edge(mode="prewitt", thy1=0, thy2=255, U=2, V=2)
Overlay(last, dehalo_alpha(rx=7, ry=1, BrightStr=0.8, DarkStr=0.8), mask=emask)
Spline36Resize(width*2,height)

[Attachment 82959 - Click to enlarge]

I tried rx=3.5 too

Spline36Resize(width/2, height)
emask=mt_edge(mode="prewitt", thy1=0, thy2=255, U=2, V=2)
Overlay(last, dehalo_alpha(rx=3.5, ry=1, BrightStr=0.8, DarkStr=0.8), mask=emask)
Spline36Resize(width*2,height)

[Attachment 82960 - Click to enlarge]

[Selur]

Have you tried applying YAHR two times?

YAHR()
YAHR()

[jagabo]

Here's what I used:

LWLibavVideoSource("FineDeHalo-before-after.png", cache=false, prefer_hw=2) 
ConvertToYV12(interlaced=true)
Crop(0,0,width/2,height)
src = last

emask = mt_edge(thy1=8, thy2=8).mt_expand().mt_expand().mt_expand().mt_expand().mt_expand().Blur(1.5).Blur(1.5)
dh = dehalo_alpha(rx=7, ry=1, BrightStr=0.8, DarkStr=0.8)
Overlay(src, dh, mask=emask)

[Attachment 82962 - Click to enlarge]

I didn't supply the script earlier because I didn't have the original video to work with.

[Selur]

@jagabo: afaik. the video should be 05-5D[JP]-01.avi from the first post

DeHalo_alpha()
YAHR()
YAHR()
seesm fine too. since YAHR with larger depth and stronger blur is also possible

[cinephil]

Here's what I used:

LWLibavVideoSource("FineDeHalo-before-after.png", cache=false, prefer_hw=2) 
ConvertToYV12(interlaced=true)
Crop(0,0,width/2,height)
src = last

emask = mt_edge(thy1=8, thy2=8).mt_expand().mt_expand().mt_expand().mt_expand().mt_expand().Blur(1.5).Blur(1.5)
dh = dehalo_alpha(rx=7, ry=1, BrightStr=0.8, DarkStr=0.8)
Overlay(src, dh, mask=emask)

I didn't supply the script earlier because I didn't have the original video to work with.

@jagabo: afaik. the video should be 05-5D[JP]-01.avi from the first post

Sorry for inconvenience. Yes the video source is in #1 post 05-5D[JP]-01.avi

[jagabo]

With that source, something like this:

LWLibavVideoSource("05-5D[JP]-01.avi", cache=false, prefer_hw=2) 
TFM()
TDecimate()
vInverse()
MergeChroma(last, Spline36Resize(width/2, height).aWarpSharp2(depth=30).nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=width, fheight=height))
SMDegrain(tr=2, thsad=200, PreFilter=6)
Tweak(sat=2.0, bright=-15, cont=1.15)

# dehalo with edge mask
emask = Spline36Resize(400, height).mt_edge(thy1=10, thy2=10).mt_expand().mt_expand(mode="horizontal").Blur(1.0).GreyScale().nnedi3_rpow2(2, cshift="spline36Resize", fwidth=720, fheight=height)
dh = Spline36Resize(400, height).dehalo_alpha(rx=3.5, ry=1, BrightStr=0.0, DarkStr=0.8).Spline36Resize(width, height).Sharpen(0.5, 0.0)
Overlay(last, dh, mask=emask)

[jagabo]

An even more selective edge mask would be:

emask = Spline36Resize(400, height).mt_edge(mode="-1 2 -1    -1 2 -1   -1 2 -1   6").mt_expand().mt_expand(mode="horizontal").Blur(1.0).GreyScale().nnedi3_rpow2(2, cshift="spline36Resize", fwidth=720, fheight=height)

It only detects vertical edges.

[cinephil]

I mostly done with filters and want to see how it looks if I go VFR route instead of keep 29.97i.
I tried made a script with 2 pass TIVTC, that was suggested by hello_hello in here - https://forum.videohelp.com/threads/409300-3-2-pulldown-flags-and-29-97PsF-in-the-same...de#post2687420

FFVideoSource("E:\SAMPLE\video\video-film.avi")

tfm(output="matches.txt")
tdecimate(mode=4,output="metrics.txt")

tfm(input="matches.txt")
tdecimate(mode=5,hybrid=2,vfrDec=0,input="metrics.txt",tfmIn="matches.txt",mkvOut="mkv-timecodesfile.txt")

ConvertToYV12(matrix="rec601")

https://forum.videohelp.com/attachments/82089-1725883704/video-film.avi

But since my source is not MPEG2 - I don't have d2v index file (and don't know how to use ffindex instead) and got an error message like this:

TDecimate: input error (mode 5 and 6, all frames must have entries)!

What am I missing? Or this metod not suitable for avi source?

[Selur]

I might be wrong, but iirc 2pass TIVTC requires two separate passes, not 'two in one'. (I agree, the later would be nicer, but that is not how it works.)
So first call the script with

tfm(output="matches.txt")
tdecimate(mode=4,output="metrics.txt")

to create the matches.txt and metrics.txt, then run the script with:

tfm(input="matches.txt")
tdecimate(mode=5,hybrid=2,vfrDec=0,input="metrics.txt",tfmIn="matches.txt",mkvOut="mkv-timecodesfile.txt")

to use the created files.

[cinephil]

I might be wrong, but iirc 2pass TIVTC requires two separate passes, not 'two in one'. (I agree, the later would be nicer, but that is not how it works.)

I see...
But it needs some extra step (at least in my setup). After "first part" script with analysis pass I need to copy matches.txt and metrics.txt (without closing MeGUI) to different folder and run "second part" script from there.

So I run "second part" script adding this line to x264 encoder -

--tcfile-in "M:\edit\VFR-test\2\mkv-timecodesfile.txt"

and resulting mkv file is with VFR, but it play at 119.947 fps. How to get normal speed playback

mkv-timecodesfile.txt looks like this -

# timecode format v1
Assume 29.970030
# TDecimate v1.0.11 by tritical
# Mode 5 - Auto-generated mkv timecodes file
65,176,23.976024
# vfr stats:  66.99% film  33.01% video
# vfr stats:  140 - film  69 - video  209 - total
# vfr stats:  longest vid section - 65 frames
# vfr stats:  # of detected vid sections - 1

[jagabo]

After running the second pass you have to mux the timecodes with the video.

[cinephil]

After running the second pass you have to mux the timecodes with the video.

Could it be done by adding custom command line in encoder settings? Or it only works through mkvtoolnix?

And one more question.
After doing that -

tfm(output="matches.txt")
tdecimate(mode=4,output="metrics.txt")

tfm(input="matches.txt")
tdecimate(mode=5,hybrid=2,vfrDec=0,input="metrics.txt",tfmIn="matches.txt",mkvOut="mkv-timecodesfile.txt")

video is stay interlaced?

[jagabo]

No, after TFM/TDecimate the video is progressive. That's the whole point.

[cinephil]

No, after TFM/TDecimate the video is progressive. That's the whole point.

Hmmm... Then MediaInfo not seeing this correctly -

Frame rate mode                          : Variable
Frame rate                               : 25.957 FPS
Original frame rate                      : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First

Or, most likely, I need change my encoder settings (since video is no longer interlaced)

------------------------

The second TFM/TDecimate pass with filters runs ultra slow - below 1 fps
And there could be sync problems with sound - in some test clips I got 16ms time difference (from 30sec clip).

[cinephil]

Apparently Auto-generated mkv timecodes are incorrect

# timecode format v1
Assume 29.970030
# TDecimate v1.0.11 by tritical
# Mode 5 - Auto-generated mkv timecodes file
65,176,23.976024
# vfr stats:  66.99% film  33.01% video
# vfr stats:  140 - film  69 - video  209 - total
# vfr stats:  longest vid section - 65 frames
# vfr stats:  # of detected vid sections - 1

Film frames set from 65 to 176, but in fact they go to the end frame - 209. So 177-209 frames played at the wrong speed

But set this frame intervals manually is not reliable either, since I can miss start/end point by several frames.