VideoHelp Forum
+ Reply to Thread
Results 1 to 12 of 12
Thread
  1. I am trying to reverse an anime upscale on blu-ray with a filter called debilinear and then rescale it with nnedi3 to get a much more clear and sharp image.

    Code:
    Debilinear(1280, 720)
    nnedi3_rpow2(2, cshift="LanczosResize", fwidth=1920, fheight=1080)
    This appears to work really well, but I am having a slight problem with it. It is adding a black bar all down the left side of the video and the entire outer rim looks like it is having a halo added over the picture.

    Here are 2 PNG screenshots in a zip folder to see what I am talking about.

    It is indeed debilinear causing this, because other resizers like spline or lanczos do not add this in the video. I would just use those other resizers, but they do not look nearly as good or as clear as debilinear does. Downscaling with one of those other resizers and rescaling makes it appear softer or has detail loss. Debicubic didn't seem to be the solution either.

    Here is Spline36Resize vs Debilinear
    http://screenshotcomparison.com/comparison/1038/
    http://screenshotcomparison.com/comparison/1027

    I don't see any extra parameters for debilinear in the avsi file, and it does not appear to be worked on anymore. Is there a reason I am having this problem with it?
    Last edited by killerteengohan; 6th Jun 2019 at 16:41.
    Quote Quote  
  2. It's not uncommon for resizing filters to have issues at the edges of the frame. Just overlay the original edges to get rid of the ringing artifacts. Or overlay an alternate resize method at the edges of the frame before upscaling.
    Quote Quote  
  3. Would you mind giving an example use of overlaying the original image?

    this is the script as it is now

    LoadPlugin("C:\Program Files (x86)\MeGui\tools\ffms\ffms2.dll")
    FFVideoSource("C:\Users\K\Desktop\Encoding\Highsch ool Of The Dead\Jp\Episode 1-2\T2_Video - .mkv", threads=1)
    Debilinear(1280, 720)
    nnedi3_rpow2(2, cshift="LanczosResize", fwidth=1920, fheight=1080)
    smoothtweak(saturation=1.02, dither=-1, interp=0, limiter=false)

    I guess assume I want to overlay the original a width of 4 pixels on all 4 sides for the example.
    Last edited by killerteengohan; 5th Jun 2019 at 23:14.
    Quote Quote  
  4. The edge problems seemed to be 7 or 8 pixels at the left and right, 3 or 4 at the top/bottom. Keeping the original full scale edges:

    Code:
    LoadPlugin("C:\Program Files (x86)\MeGui\tools\ffms\ffms2.dll")
    FFVideoSource("C:\Users\K\Desktop\Encoding\Highsch ool Of The Dead\Jp\Episode 1-2\T2_Video - .mkv", threads=1)
    source = last
    edgemask = BlankClip(last).Crop(8,4,-8,-4).AddBorders(8,4,8,4,color=$ffffff)
    Debilinear(1280, 720)
    nnedi3_rpow2(2, cshift="LanczosResize", fwidth=1920, fheight=1080)
    Overlay(last, source, mask=edgemask)
    smoothtweak(saturation=1.02, dither=-1, interp=0, limiter=false)
    Keeping the downscaled edges:

    Code:
    LoadPlugin("C:\Program Files (x86)\MeGui\tools\ffms\ffms2.dll")
    FFVideoSource("C:\Users\K\Desktop\Encoding\Highsch ool Of The Dead\Jp\Episode 1-2\T2_Video - .mkv", threads=1)
    source = last
    Debilinear(1280, 720)
    edgemask = BlankClip(last).Crop(6,4,-6,-4).AddBorders(6,4,6,4,color=$ffffff)
    Overlay(last, source.Spline36Resize(last.width, last.height), mask=edgemask)
    nnedi3_rpow2(2, cshift="LanczosResize", fwidth=1920, fheight=1080)
    smoothtweak(saturation=1.02, dither=-1, interp=0, limiter=false)
    Adjust the crop/addborders as necessary. If you want to soften the border add Blur(1.5) the edgemask.
    Quote Quote  
  5. I think I will go with keeping the original full scale edges, because its the downscaling with debilinear or other resizers that causes that black on the left and all around edge lines. You can see it in the 720p version, even if its quite a bit smaller. So I assume using those edges would have it still.

    Thanks, I will give that a try in a couple hours when I can, and get back to you if need be.
    Quote Quote  
  6. Originally Posted by killerteengohan View Post
    I think I will go with keeping the original full scale edges, because its the downscaling with debilinear or other resizers that causes that black on the left and all around edge lines. You can see it in the 720p version, even if its quite a bit smaller. So I assume using those edges would have it still.
    No because those edges are replaced with an alternate scaled version (Spline36 in my example) while the video is downscaled.
    Quote Quote  
  7. I just tried both of the above suggestions. The first one works pretty good. The overall image is slightly softer looking, but it still looks better than before and I can always add a little sharpening if need be. It looks like I got the result I was hoping to get for the most part. Thanks a lot Jagabo!

    The second one, I still got the black bar and outer rim, it was just slightly thinner in appearance, and really soft looking instead of sharp and clear, like when no edge mask is used. Even when crop was adjusted it still had it for me.



    I am also trying to play around with this and learn it inside and out. Would you mind helping me with something?
    This is purely for learning and future use should I ever need it.

    This is what I have so far

    Code:
    LoadPlugin("C:\Program Files (x86)\MeGui\tools\ffms\ffms2.dll")
    FFVideoSource("C:\Users\K\Desktop\Encoding\Highschool Of The Dead\Jp\Episode 1-2\T2_Video - .mkv", threads=1)
    
    source = last
    edgemask = BlankClip(last).Crop(128,124,-128,-124).AddBorders(128,124,128,124,color=$ffffff)
    
    Debilinear(1280, 720)
    nnedi3_rpow2(2, cshift="LanczosResize", fwidth=1920, fheight=1080)
    tweak(sat=1.50, bright=150.00)
    
    Overlay(last, source, mask=edgemask)
    smoothtweak(saturation=1.02, dither=-1, interp=0, limiter=false)

    Any filter that goes after the overlay will be done to the entire thing.
    Any filter that goes after the source=last, but before overlay, will be done to the part of the video that's not specified in the rim crop amount. (The center image that the rim mask is supposed to be going around)

    What if I would want to filter just that masked outer rim and leave the rest alone? Say I want to sharpen it, or adjust saturation or something to help it blend in, How would I do that?
    I tried placing tweak(sat=5.00) before source=last and it affects the entire thing. (This is expected)
    I tried placing tweak(sat=5.00) in the mask itself right after addborders with .tweak(sat=5.00), but it did nothing at all or was not visible to my eyes. (Thought this would do it, but I guess not)

    I cannot seem to find out how to get only that outer rim to be filtered and leave the rest alone.


    Its almost as if the overlay in the center is not 100% solid and is kind of transparent. If it is not, can it be made 100% solid and cover up whatever is done underneath 100%?
    Kind of like filtering 2 separate versions of the same source, and then placing one centered over top of the other with no transparency.
    Quote Quote  
  8. So to simplify, I'd like to achieve a script that does something like this. Can be with anything, doesn't have to be with what I asked about earlier. Just a generalized script will help.

    bottom = source filtered how I want specified size outer rim to appear
    top = source filtered how I want the inside of the rim to appear

    Stack one on top of the other, then make specified size rim around the one on top 100% transparent
    Quote Quote  
  9. Originally Posted by killerteengohan View Post
    Its almost as if the overlay in the center is not 100% solid and is kind of transparent. If it is not, can it be made 100% solid and cover up whatever is done underneath 100%?
    I think you're right. When I used AddBorders with color=0xffffff, I forgot that AviSynth would use a limited range RGB to YUV conversion. That leaves the mask with black at Y=16 and white at Y=235. So you need to convert the mask to full range. You can do that with ColorYUV(levels="TV->PC") or Levels(16,1,235,0,255).

    For different filtering of the rim just replace source=last with source=last.WhateverFiltering().
    Quote Quote  
  10. Originally Posted by jagabo View Post
    I think you're right. When I used AddBorders with color=0xffffff, I forgot that AviSynth would use a limited range RGB to YUV conversion. That leaves the mask with black at Y=16 and white at Y=235. So you need to convert the mask to full range. You can do that with ColorYUV(levels="TV->PC") or Levels(16,1,235,0,255).
    Oh, that makes sense. I will try the full range out and see what happens.

    Originally Posted by jagabo View Post
    For different filtering of the rim just replace source=last with source=last.WhateverFiltering().
    I don't know how I let that slip by me, I feel so stupid for not thinking of that when I should have known better, and have done similar before lol.
    Last edited by killerteengohan; 6th Jun 2019 at 13:21.
    Quote Quote  
  11. I tried adding in .ColorYUV(levels="TV->PC") , but its adjusting the levels of the entire things colors, not just the mask. It does indeed fix the center image inside of the basked border, but whats under it (aka the outer rim) is also being converted.

    Try this and see

    Before

    Code:
    source = last.tweak(sat=25.00)
    edgemask = BlankClip(last).Crop(128,124,-128,-124).AddBorders(128,124,128,124,color=$ffffff).ColorYUV(levels="TV->PC")
    Debilinear(1280, 720)
    nnedi3_rpow2(2, cshift="LanczosResize", fwidth=1920, fheight=1080)
    tweak(sat=0.0)
    Overlay(last, source, mask=edgemask)
    smoothtweak(saturation=1.02, dither=-1, interp=0, limiter=false)

    After

    Code:
    source = last.tweak(sat=25.00)
    edgemask = BlankClip(last).Crop(128,124,-128,-124).AddBorders(128,124,128,124,color=$ffffff)
    Debilinear(1280, 720)
    nnedi3_rpow2(2, cshift="LanczosResize", fwidth=1920, fheight=1080)
    tweak(sat=0.0)
    Overlay(last, source, mask=edgemask)
    smoothtweak(saturation=1.02, dither=-1, interp=0, limiter=false)
    Quote Quote  
  12. nvm Jagabo, my eyes were playing tricks on me. I verified colors in a paint software and adjusting the mask levels is not changing the outer rim. It is changing it's colors, but it's changing it back to source levels it had when there was no mask at all. Probably because the mask that covers the entire thing was corrected when levels got changed.

    Thanks so much for taking all this time to help me out! I got the results I was hoping for thanks to you.
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!