Intelligent Down-Scale for Video

[Sandfly]

Hi,

What is the best software, paid or free for intelligently reducing resolution.

I have used Boris FX BCC UpRez in After Effects.
Fits with other processes in AE, but best quality Down-Scale is the aim.

Edit: I should add that options to save are open using Adobe.
For any program it needs to have Lagarith or Tiff Lossless save.

Any other programs for Intelligent Down-Scale?

Thanks

[butterw]

What resolutions, what content ? Also what does Intelligent mean here ?

[Sandfly]

Any, any, and in the same context as the use of "AI" BS.

[Sandfly]

Up-scale gets more complex with size multiplication.

Down-Scaling has increasingly fewer pixel options.

There do not appear to be many options for programs that Down-Scale video.
Boris FX BCC UpRez is the only such program I have found that seems to be designed to meet task.

Anything else out there?

[davexnet]

What about Spline and Lanczos options in Avisynth/Vapoursynth
I've never heard of intelligent resize smaller, only when upscaling

[Sandfly]

I've never heard of intelligent resize smaller either, just thought it worth asking.

From discussion I found by accident between Topaz creator and others, Lanczos is as good as any.
The Topaz forum had un-linked pages!

May be all such software is based on Lanczos or similar.
When upscaling I use whole numbers, reducing the final result to specific dimensions.
Also when CD content has metadata DAR, standard upscale, then reduce one dimension to match DAR.

What about Spline and Lanczos options in Avisynth/Vapoursynth
I've never heard of intelligent resize smaller, only when upscaling

[davexnet]

Lanczos sometime oversharpens a little, you'll have to try it.
I see Spline36resize mentioned often and would make a good choice for a comparison
http://avisynth.nl/index.php/Resize

[Sandfly]

Used Boris FX BCC UpRez on videos for Down-scale.

I have used Lanczos before on tiff images but not with Avisynth.
It required another command to force size if not matching original dimension ratio!
Not seeing this in Avisynth commands?

Not used Spline before.
Could Spline be used on video and save as Lagarith avi?






Copied Avisynth Spline/Lanczos commands for reference.

[hello_hello]

Used Boris FX BCC UpRez on videos for Down-scale.

I have used Lanczos before on tiff images but not with Avisynth.
It required another command to force size if not matching original dimension ratio!
Not seeing this in Avisynth commands?

Using Avisynth's resizers on their own, you need to calculate the appropriate dimensions for resizing yourself, based on the source's dimensions and aspect ratio. If you're using Avisynth though, try the CropResize function in my signature. If you follow the link there's pictures demonstrating it's usage in the second post. You can use any compatible resizer (Avisynth native or via a plugin). The default is Spline36Resize.

If you're really anal, downscaling is technically better when applied in linear light (converting the source to linear light RGB, downscaling, then converting back to the original format). Or there's a gamma-aware resizer for Avisynth here. You can use it with the CropResize function.

For most downscaling though (ie not extreme downscaling), gamma-aware resizing often makes very little visual difference. There's links to extreme downscaling comparison shots about half way down the ResampleHQ page.

Not used Spline before.
Could Spline be used on video and save as Lagarith avi?

Avisynth outputs raw uncompressed video. You can compress it with whatever encoder you like. VirtualDub2 will open Avisynth scripts and supports Lagarith. Avisynth will come with something of a learning curve though.

[Sandfly]

That all sounds interesting, I will need to come back and study in a bit of detail.
Thanks

[jagabo]

http://avisynth.nl/index.php/Resize
http://www.avisynth.nl/index.php/DPID

[Sandfly]

[Selur]

First thing that came to my mind in regard to 'intelligent' resizing were Seam Carving (https://en.wikipedia.org/wiki/Seam_carving) based stuff like ReTarget (http://www.avisynth.nl/users/vcmohan/ReTarget/ReTarget.html),...

[Skiller]

When it comes to downscaling, doing it in linear light – "gamma aware" – can make some subtle differences, especially with very small, bright details. For example stars in a dark sky.
I do all of my Full HD -> DVD downscales in linear light and with Spline16 (36 and 64 appear too ringy for me for this scenario).

[Selur]

Right, when using Avisnyth ResampleHQ might be interessting for that.

[Sandfly]

Lanczos sometime oversharpens a little, you'll have to try it.
I see Spline36resize mentioned often and would make a good choice for a comparison
http://avisynth.nl/index.php/Resize

downscales in linear light and with Spline16 (36 and 64 appear too ringy for me for this scenario).

ResampleHQ might be interessting for that.

Seam Carving (https://en.wikipedia.org/wiki/Seam_carving) based stuff like ReTarget (http://www.avisynth.nl/users/vcmohan/ReTarget/ReTarget.html)

http://avisynth.nl/index.php/Resize
http://www.avisynth.nl/index.php/DPID

downscaling is technically better when applied in linear light (converting the source to linear light RGB, downscaling, then converting back to the original format. Or there's a gamma-aware resizer for Avisynth here. You can use it with the CropResize function.

I think converting the source to linear light RGB for DPID Down-Scale could be interesting.
Programs like Gigapixel can generate intermittent bright areas, which go unnoticed in a single image, but ruin video.
This is where moderation could be key.

Video Enhance AI generates curved artifacts that seem to follow something like Seam carving?
And another pattern best described as looking through fine white nylon curtain used in kitchens.

Would you add any of the above for comparison?

I would like to hear from anybody that has done comparisons!

[hello_hello]

I think converting the source to linear light RGB for DPID Down-Scale could be interesting.
Programs like Gigapixel can generate intermittent bright areas, which go unnoticed in a single image, but ruin video.
This is where moderation could be key.

DPID is probably designed for video/images with standard gamma compression, not linear light, but I've never used it so I don't speak from experience.

As linear light downscaling tends to involve converting YV12 to linear RGB, downscaling, then converting back to YV12, it's probably best done with high quality sources. Standard YV12 video has subsampled chroma, so the conversion to RGB and back isn't lossless (ignoring any other filtering or resizing).

[Sandfly]

DPID is probably designed for video/images with standard gamma compression, not linear light, but I've never used it so I don't speak from experience.

As linear light downscaling tends to involve converting YV12 to linear RGB, downscaling, then converting back to YV12, it's probably best done with high quality sources. Standard YV12 video has subsampled chroma, so the conversion to RGB and back isn't lossless (ignoring any other filtering or resizing).

Don't normally work with quality footage, so you raise a good point.
Prefer to improve the whole image a little, not highlighting the closer objects and sacrificing background.
We are not talking Steven Spielberg directors. Needed more time to consider these things.

Checked your signature link.

I am looking for something that will reduce an image to new dimensions regardless of the aspect ratio.
No crop or pad.

In most cases the smaller size would be chosen to match original ratio, the exception being files with Metadata DAR.

[Skiller]

I don't convert to RGB for my linear light downscales.
I use Dither tools by Cretindesalpes which can convert to and from linear light in YCbCr color spaces.

Part of the script looks like this for example:

#YV12 8 Bit source

Dither_convert_8_to_16()


Dither_y_gamma_to_linear(curve="709")

Dither_resize16(704, 576, kernel="Spline16")

Dither_y_linear_to_gamma(curve="709")


DitherPost()

I'm still on AVS 2.6, and with AVS+ that whole 16 Bits stacked format workflow may be considered deprecated but it gets the job done.

[hello_hello]

Don't normally work with quality footage, so you raise a good point.
Prefer to improve the whole image a little, not highlighting the closer objects and sacrificing background.
We are not talking Steven Spielberg directors. Needed more time to consider these things.

I don't like video that looks sharpened, but sometimes some subtle sharpening can help after downscaling. The sharpeners I use come in the form of Avisynth functions. I wouldn't sharpen video with lots of noise or encoding artefacts though.
The other problem with downscaling, is when you watch the downscaled version full screen, you're somewhat at the mercy of the player's upscaler in respect to how it'll look.

Checked your signature link.

I am looking for something that will reduce an image to new dimensions regardless of the aspect ratio.
No crop or pad.

In most cases the smaller size would be chosen to match original ratio, the exception being files with Metadata DAR.

Then you could just calculate the appropriate dimensions and use the resizer you prefer.
My script will do much the same thing, except it will crop a little to prevent aspect error when the new dimensions don't match the original aspect ratio exactly. Often it crops less than a whole pixel to prevent aspect error.

As a made-up example, if you had a 1920x808 source and wanted to resize to 720p, the exact dimensions for resizing would be 1280x538.66. It's not much different but often that'd be resized to 1280x540 instead with a little aspect error. 1920/808 = 2.376 and 1280/540 = 2.37

The alternative would be to crop a couple of pixels each side before downscaling so there's less distortion. 1916/808 = 2.371
Left to it's own devices, that's what CropResize does, only it crops the exact amount required. The main motivation for creating the function was so I'd never have to calculate that sort of thing again myself when resizing.

For a 1920x808 source, CropResize(1280) would output this.


[Attachment 61894 - Click to enlarge]

Of course if you're only going from one standard aspect ratio/resolution to another (ie 1920x1080 to 1280x720 etc), then you don't need to worry about any of that. How you do it though, is of course up to you.

[hello_hello]

I don't convert to RGB for my linear light downscales.
I use Dither tools by Cretindesalpes which can convert to and from linear light in YCbCr color spaces.

I'm still on AVS 2.6, and with AVS+ that whole 16 Bits stacked format workflow may be considered deprecated but it gets the job done.

Cretindesalpes latest toy is FMTConv. I'll have to look closer to see if it'll do the same.
Mind you I hardly encode a script that doesn't have GradFun3() at the end of it, so I'm with you when it comes to DitherTools.

Edit: I was bored, so I made a thing. Are there other essentials the function should include? I'll go through the full argument list sometime.

function DitherToolsLinear(clip Source, int Width, int Height, \
float "src_left", float "src_top", float "src_width", float "src_height", \
string "Kernel", string "CurveS", string "CurveD", bool "LSBIn", bool "LSBOut")  {

Kernel = default(Kernel, "Spline16")
CurveS = default(CurveS, "709")
CurveD = default(CurveD, CurveS)
LSBIn = default(LSBIn, false)
LSBOut = default(LSBOut, false)

Input = !LSBIn ? Source.Dither_convert_8_to_16() : Source

Resized = Input\
.Dither_y_gamma_to_linear(curve=CurveS)\
.Dither_resize16(Width, Height, src_left, src_top, src_width, src_height, kernel=Kernel)\
.Dither_y_linear_to_gamma(curve=CurveD)

Output = !LSBOut ? Resized.DitherPost() : Resized

return Output  }

[Attachment 61895 - Click to enlarge]

CropResize(720,576, 242,0,-242,0, OutDAR=15.0/11.0, Resizer="DitherToolsLinear", Borders=true, Info=true)

It should work fine with Avisynth 2.6.
I forgot to convert the colors so they've changed a little..


[Attachment 61917 - Click to enlarge]

[hello_hello]

I was bored again so I thought I'd amuse myself with a quick comparison.
That was before I realised FMTConv should come with a warning that trying to use it might make you suicidal.
I couldn't work out how to do linear light resizing with FMTConv without a conversion to RGB and back. I don't think it can.

The screenshots are from the same frame I used in my previous post. The source is quite noisy. I downscaled as much as I could without the picture turning to total mush, then resized it back to 1080p with Spline36Resize. I was mainly looking for differences in the white smoke rings and small specs of white amongst the grey in the top left corner. Each resizing method used Spline36 for downscaling as it's the default for each.

Spline36 in non-linear light obviously loses the most brightness in those small white areas.
Despite DitherTools resizing in linear YUV and FMTConv converting to RGB and back, they look visually the same to me.
ResampleHQ retained a tad more brightness for some reason.

PNGs this time.

Original


[Attachment 61901 - Click to enlarge]

Spline36Resize
CropResize(480,360, 242,0,-242,0, Resizer="Spline36")
Spline36Resize(1440,1080)


[Attachment 61902 - Click to enlarge]

DitherTools
CropResize(480,360, 242,0,-242,0, Resizer="DitherToolsLinear")
Spline36Resize(1440,1080)


[Attachment 61903 - Click to enlarge]

FMTConv
CropResize(480,360, 242,0,-242,0, Resizer="FMTConvLinear")
Spline36Resize(1440,1080)


[Attachment 61904 - Click to enlarge]

ResampleHQ
CropResize(480,360, 242,0,-242,0, Resizer="ResampleHQ")
Spline36Resize(1440,1080)


[Attachment 61907 - Click to enlarge]

function FMTConvLinear(clip Source, int Width, int Height, \
float "src_left", float "src_top", float "src_width", float "src_height", \
string "Kernel", string "CurveS", string "CurveD")  {

Kernel = default(Kernel, "Spline36")
CurveS = default(CurveS, "709")
CurveD = default(CurveD, CurveS)

Resized = Source\
.fmtc_bitdepth(bits=16)\
.fmtc_resample(css="444")\
.fmtc_matrix(mat=CurveS)\
.fmtc_transfer(transs=CurveS, transd="Linear")\
.fmtc_resample(Width, Height, sx=src_left, sy=src_top, sw=src_width, sh=src_height, kernel=Kernel)\
.fmtc_transfer(transs="Linear", transd=CurveD)\
.fmtc_matrix(mat=CurveD)\
.fmtc_resample(css="420")\
.fmtc_bitdepth(bits=8)

return Resized  }

[Skiller]

Yes, barely any difference at all.

That's a neat function you made there.

I do notice some dark halos. In the Dither Tools doc it says that applying a sigmoid curve in addition when converting between gamma and linear shall reduce dark halos.

"This helps reducing the dark halo artefacts around sharp edges caused by resizing in linear luminance."

Like so

Dither_y_gamma_to_linear(curve="709", sigmoid=true)

Dither_resize16(.........)

Dither_y_linear_to_gamma(curve="709", sigmoid=true)

Could you please try this with the same frame, I'd love to see what difference it makes.

[hello_hello]

You must be more sensitive to the dark halos than I am. I had to really look to see what you were referring to.

sigmoid=true seems to reduce the effect of resizing in linear light. At least for this example. While playing around though, I noticed something else. The updated version of DitherToolsLinear is below if you're interested in using it. It has Spline36 as the default, but obviously you can change that.

For sigmoid=true the jaggedness or aliasing, however you'd describe it, is reduced in the dark lines, but for the rec.709 curve it also reduces the effect of linear light downscaling. I don't really understand when you're supposed to use 1886 rather than 709, but in this particular instance, the 1886 curve with sigmoid=true seems to be the best of both worlds. How does it look to you?
I'm having a hard time spotting much difference in aliasing at the downscaled resolution, but it does seem to effect the upscaling. For each one, I've included a screenshot before upscaling.

CropResize(480,360, 242,0,-242,0, Resizer="DitherToolsLinear", RStr="SigmoidS=false")
Spline36Resize(1440,1080)


[Attachment 61936 - Click to enlarge]
[Attachment 61937 - Click to enlarge]

CropResize(480,360, 242,0,-242,0, Resizer="DitherToolsLinear", RStr="SigmoidS=true")
Spline36Resize(1440,1080)


[Attachment 61938 - Click to enlarge]
[Attachment 61939 - Click to enlarge]

CropResize(480,360, 242,0,-242,0, Resizer="DitherToolsLinear", RStr=""" SigmoidS=false, CurveS="1886" """)
Spline36Resize(1440,1080)


[Attachment 61940 - Click to enlarge]
[Attachment 61941 - Click to enlarge]

CropResize(480,360, 242,0,-242,0, Resizer="DitherToolsLinear", RStr=""" SigmoidS=true, CurveS="1886" """)
Spline36Resize(1440,1080)


[Attachment 61942 - Click to enlarge]
[Attachment 61943 - Click to enlarge]

function DitherToolsLinear(clip Source, \
int Width, int Height, float "src_left", float "src_top", float "src_width", float "src_height", \
string "Kernel", string "CurveS", string "CurveD", float "GCorS", float "GCorD", \
bool "SigmoidS", bool "SigmoidD", float "ThrS", float "ThrD", float "ContS", float "ContD", \
bool "LSBIn", bool "LSBOut")  {

Kernel = default(Kernel, "Spline36")
CurveS = default(CurveS, "709")
CurveD = default(CurveD, CurveS)
GCorD = default(GCorD, GCorS)
SigmoidS = default(SigmoidS, false)
SigmoidD = default(SigmoidD, SigmoidS)
ThrD = default(ThrD, ThrS)
ContD = default(ContD, ContS)
LSBIn = default(LSBIn, false)
LSBOut = default(LSBOut, LSBIn)

Input = !LSBIn ? Source.Dither_convert_8_to_16() : Source

Resized = Input\
.Dither_y_gamma_to_linear(curve=CurveS, gcor=GCorS, sigmoid=SigmoidS, thr=ThrS, cont=ContS)\
.Dither_resize16(Width, Height, src_left, src_top, src_width, src_height, kernel=Kernel)\
.Dither_y_linear_to_gamma(curve=CurveD, gcor=GCorD, sigmoid=SigmoidD, thr=ThrD, cont=ContD)

Output = !LSBOut ? Resized.DitherPost() : Resized

return Output  }