How to get rid of DVD (PAL) wide horizontal bars

[OoPee]

Hi, I have tried dozens of different deinterlace methods to get rid of these "wide" horizontal bars, but I'm starting to believe that they cannot be removed. They are not on every shot or consistent. As far as I know, those are not typical stripes from interlaced source, but probably something else? Anyone have a solution?

Source (taken from DVD with MakeMKV and direct cut with Avidemux without re-encoding)
https://drive.google.com/file/d/1aaYt4EimryiCOb9v9l2ib8KJ1ZbDHg6Z/view?usp=sharing

[DB83]

firstly, just a little tip. When you are simply sharing a short video < 500mb upload it direct to your topic as an attachment. In Google, the clip appears to be de-interlaced (so, yes, it is possible) and it messes with the aspect ratio. So one still has to download it regardless.But I, others may disagree, simply see normal interlaced video mostly previlent when the white lights hit the singer - you would not notice them in the darker passages. Play the clip back in VLC and the interlacing vanishes when, for example ydif, is chosen as the de-interlacer.What I do notice is that you sample is bottom-field first whereas, typically, dvd-mpeg2 is top-field first (both are valid) and that may be fooling you.How does this look to you now ?

[ProWo]

Another one (deinterlaced with bwdif and corrected DAR to 1.33, using clever FFmpeg-GUI).

[OoPee]

Thank you for proofing I was wrong Latter one still had them thou, but the first one was already very good. My goal is to clear this as much as possible, and then upscale it to 1080p with TVAI. Any good advices about the working chain, like should I only do this deinterlace first, then put it through Topaz - Proteus + double the framerate, and then on third encoding Proteus + upscale?

[OoPee]

But it still seems, that I can't get this one as clean, no matter what I choose. This is weird. So how did you do it DB83?

[DB83]

Firstly, upscaling is not my forte and, TBH, all these softwares that do it rely on the user not having any knowledge on how to do it themselves >> their work is neither Artificial or Inteligent.

So I, as an example, just build on my original sample without doubling the frame-rate (which will help but you will also have to increase the bitrate a fair bit)


There is a topic on here about Topaz and maybe some alternative suggestions using avisynth which I do not touch.

[DB83]

And to answer your Q. you already have the tool I used.

Avidemux >> video output mpeg2 (ff) changed to AVC h264 for the latest > filter selected = deinterlace ydif + software resize subsequent source PAR 16:15 to 1440*1080 square

[jagabo]

1080p50, 10 bit hevc, AviSynth

[DB83]

^^ Maybe the OP would appreciate sight of the avisynth script.

[OoPee]

^^ I would. That actually looked pretty amazing.

[jagabo]

I wasn't sure you would like it. Some details are gone and some halos are left. Here's what I did:

LWLibavVideoSource("B1_t01_cut.mkv", cache=false, prefer_hw=2) 
Crop(8,0,-8,-0)
ColorMatrix(mode="rec.601->rec.709")
ConvertBits(10)
QTGMC(sharpness=0.5)

Spline36Resize(480, 384)
SMDegrain(thsad=100, tr=2, PreFilter=4)
FineDehalo(rx=2.5, ry=1)
Sharpen(0.3)
nnedi3_rpow2(2, cshift="Spline36Resize")
aWarpSharp2(depth=3)
Sharpen(0.3)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=1440, fheight=1080)
aWarpSharp2(depth=3)
CAS(0.6)

prefetch(12)

In retrospect, I'd probably reduce or eliminate the Sharpen(0.3) after FineDehalo()

[OoPee]

Thanks jagabo. How about if I just want to deinterlace and leave the rest for Topaz?

[jagabo]

Thanks jagabo. How about if I just want to deinterlace and leave the rest for Topaz?

This will only deinterlace:

LWLibavVideoSource("B1_t01_cut.mkv", cache=false, prefer_hw=2) 
QTGMC(sharpness=0.5)

[DB83]

May I inquire what you thought of my own humble attempt at upscaling which, itself, has room for improvement ?

But, honestly, unless you already own the program, do you think that spending $300 is going to be better than avisynth? And it appears that Topaz also does deinterlacing.

[OoPee]

I own TVAI already, but it's deinterlacing didn't quite worked out for this one. For my eye it seems, that Topaz still might have some advantage with upscaling. But you are right, if I only would have known how to get that kind of result from avisynth (and that Topaz can't even handle deinterlacing on some cases), I would have probably never bought it.

[hello_hello]

I've nothing to contribute to the upscaling method, so I thought I'd be the "why not crop all the black" guy instead.
Well... I'm here already, so I might as well be the mpeg4 aspect ratio guy too.

QTGMC()
nnedi3_rpow2(2, cshift="Spline36Resize")
LSFMod()
CropResize(0, 1080, 18,20,-18,-16, InDAR=15.0/11.0, ColorConvert="FMTConv", ColorMode="601-709")
GradFun3()

And while I'm asking.... if you crop all the black, why add more?
If you need to though....

QTGMC()
nnedi3_rpow2(2, cshift="Spline36Resize")
LSFMod()
CropResize(1920, 1080, 18,20,-18,-16, InDAR=15.0/11.0, ColorConvert="FMTConv", ColorMode="601-709", Borders=true)
GradFun3()

OoPee,
Are you aware the aspect ratio of your upscaled sample is incorrect? The picture should be wider.
The upscaling doesn't look too bad though, aside from the de-interlacing problem.

[OoPee]

No reason for the black bars, other than I just quickly tested things. Your samples looked great! This is how my AviSynth+ code looks atm:

FFMPEGSource2("B1_t01.mkv", atrack=1)
Crop(8, 8, -8, -8)
QTGMC()
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=720, fheight=540)
LSFMod()
GradFun3()
Prefetch(10)

And after that I upscale with Topaz. I think this could be it, if you accept my avs? I have no idea what I'm doing, but it looks better than without it. I would have tried the code hello_hello provided, but couldn't understand how that CropResize() work.

[hello_hello]

The link for CropResize is in my signature and there's pictures in the second post demonstrating it's usage.

CropResize(0,1080, 18,20,-18,-16, InDAR=15.0/11.0, ColorConvert="FMTConv", ColorMode="601-709")

The first two numbers are the output resolution. If you only specify one of them (use zero for the other), CropResize will resize to the width or height you specify and calculate the other one for you. The above resized the height to 1080 and the width was resized to 1484.

The next four numbers are the cropping. Because I upscaled to twice the resolution first, it meant the amount of black to crop also doubled, hence the cropping being twice what you cropped (plus a tad more because there were a couple of places with more black than the rest).

The InDAR is the display aspect of the source if it's anamorphic (as DVDs are). 4:3 DVDs tend to use an mpeg4 DAR, which is actually a little wider than 4:3, but you'd use either InDAR=15.0/11.0 or InDAR=4.0/3.0 (the difference isn't huge) and the function uses the specified InDAR to calculate the resizing.

ColorConvert and ColorMode convert from SD to HD colorimetry, for my example above, using the "FMTConv" plugin.

The main purpose of the function is to crop and resize without any aspect error. If you're anal about it and assume a 4:3 InDAR for the DVD....

576 * 4 / 3 = 768
768 / 720 = 1.06667 which is the pixel aspect ratio.
After cropping, the resolution becomes (cropping 8 pixels all round) 704 x 560
If you apply the pixel aspect ratio and resize the width, you'll have 704 * 1.06667 = 750.9333
The nearest correct resizing, without resizing the height, would be 752 x 560 and you'd upscale from there.
750.9333 / 560 * 540 = 724.11, so 720 x 540 isn't far off.

CropResize does all the math for you and it'll crop a bit extra if need be (generally only a pixel or two) so it can resize exactly.
If you were to resize to "Some Width" x 540, so all that's left to do is double the width and height with Topaz (I don't know how much control Topaz gives you), you'd do this. Setting Info=True will show you exactly what the script is doing. I'd use 15:11 for the DAR, but it's up to you. 4:3 will just be a little narrower.

QTGMC()
CropResize(0, 540, 9,10,-9,-8, InDAR=15.0/11.0, Info=true)
# The output is 740 x 540


[Attachment 73146 - Click to enlarge]

QTGMC()
CropResize(0, 540, 9,10,-9,-8, InDAR=4.0/3.0, Info=true)
# The output is 724 x 540


[Attachment 73147 - Click to enlarge]

Adjust the cropping to whatever you prefer.
Don't worry that the picture looks a bit washed out in the screenshots. It's not displaying with the correct luminance levels, but that's a display issue on my PC I forgot to rectify. The script's output will be okay.

nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=720, fheight=540)
doesn't help you all that much because nnedi3_rpow2 doubles the image size when fwidth and fheight aren't used, so it's doubling the width and height and then you're telling it to resize back down to 720x540.
GradFun3 needs to be the final filter. It dithers to fix and prevent color banding, but it probably won't do much good if you're upscaling later. I wouldn't even worry about LSFMod. I just added it for some sharpening but your Topaz sample looked sharp enough to me. It's better not to over-do it.

PS. You can also preview the cropping in CropResize by adding CPreview=1 (or 2 or 3).
The picture isn't resized for the cropping preview so the aspect ratio might look a little off.
Don't use Crop before CropResize. If you end up using CropResize, let it crop too. Cropping changes the DAR so the original DAR won't be correct if you crop first. CropResize can take the cropping into account when it calculates the resizing though, as long as it's doing the cropping.

At least the color is what it should be for this screenshot.
QTGMC()
CropResize(0, 540, 9,10,-9,-8, InDAR=4.0/3.0, CPreview=1, Info=True)


[Attachment 73148 - Click to enlarge]

[OoPee]

Thank you so much. So basically I only need:

FFMPEGSource2("B1_t01.mkv", atrack=1)
QTGMC()
CropResize(0, 540, 9,10,-9,-8, InDAR=15.0/11.0, ColorConvert="FMTConv", ColorMode="601-709")

And leave the rest for Topaz. Or at least see if it's any better compared to your upscaled clip.

I feel stupid not giving Avisynth a chance before, but I remember how all those color and aspect ratio conversions and "you need this filter to use that filter/script" and "you need to specify how many physical cores your processor has + what instruction set it uses" felt just too much. And that you need to download everything separately. Or maybe it is just, that this QTGMC does so much by alone, so it now feels relatively easy.

[DB83]

I already stated that I do not use Topaz. Neither do I intend even to install a trial to see if my 'theory' hold any value.

As I stated in my initial response, if a program assumes top-field first whereas a clip appears to be bottom-field first and that said program has no means to over-ride that basic assumption it thows the whole credibility of the program out of the window. Maybe there is an over-ride but it is up to the user and not me to establish that.


Even my 'humble' attempt at upscaling allows me select the correct field-order upon the de-interlace part. And if you wonder why the correct selection of field-order is essential any interlace artifacts are 'burnt-in' to the video if the wrong field-order is applied.


Of course avisynth will always, with the correct filters, over-ride any issues. But there are many users, like myself, who would rather use a more simplistic approach. And if the choice is between a $300 program against a 0$ program you will know which one I would choose any day.

[hello_hello]

OoPee,
Yeah I'd just go with that first and see how it looks. Judging from the sample you uploaded earlier, it'll probably be fine. Apart from the de-interlacing issue I think it looked better than the one I uploaded, although that wasn't supposed to be an awesome up-scaling example by any means because I'm no up-scaling expert.

If you've only just started using Avisynth you're getting the hang of it pretty quickly. Even getting QTGMC to work for the first time can be a bit of a challenge because it has dependencies and those dependencies can require runtimes or have dependencies....

I've just begun playing around with Vapoursynth and it's doing my head in a bit. The documentation isn't anywhere as detailed as Avisynth's wiki and I seem to be spending a lot more time with Google than I thought I'd need to, trying to understand how to do stuff. If I hadn't already been using Avisynth and had no experience creating scripts or functions, I'm not sure I wouldn't have given up fairly quickly. I'm certainly developing an appreciation for Avisynth's general lack of case sensitivity in scripts. If I have to debug a function one more time because I typed false instead of False somewhere, or something similar, I think my head will explode....

[OoPee]

Any ideas how to make the most out of this one?

[jagabo]

LWLibavVideoSource("nw_romania2004_cut.mkv") 
DeBilinearResizeMT(640, 480)
QTGMC(preset="fast")
TDecimate(Cycle=10, CycleR=6)

DeBilinearResizeMT: http://avisynth.nl/index.php/ResampleMT

[OoPee]

It was fast! Thank you so much