swscaler warning: data is not aligned

[ZetaStax]

Hello folks. Just a quick question here. When combining downscaling and cropping, the following warning may appear "[swscaler @ 000002b6231d9640] Warning: data is not aligned! This can lead to a speed loss".

I understand why this happens, but I am unfamiliar with its exact effect. Is it a speed loss on encode only? Can it have some other side effect?

[hello_hello]

I'm pretty sure it's about RAM usage and only effects encoding speed. I'm no expert, but there's some info in the align description here (Avisynth's crop function). If it's ffmpeg you're using it might have a similar option.

[ZetaStax]

Well, I just discovered something. This warning was triggered because I was cropping the video before downscaling it, when working the other way around the warning does not appear (because I downscale to 720p)

But now it begs the question. When downscaling a 1080p to 720p, would you rather crop before or after? I'm using "scale=-2,720" so for example cropping before will result in a 1312x720 video and cropping after in a 1270x696 video (in this particular example) but is there one considered better?

[hello_hello]

I don't use ffmpeg for that sort of thing so I'm not familiar with it, but the advantage of cropping first is you can crop to a particular aspect ratio, such as 16:9, and then resize to any 16:9 dimensions. Or resize to a particular with or height. When you resize and then crop, it's much harder to do. If you don't care about the output being a specific aspect ratio etc, it doesn't matter.

Standard practice for Avisynth would be something like this (just inventing numbers).

Crop(4, 6, -4, -6)
Spline36Resize(1280, 720)

That example crops to 1912x1068 which is 1.79, so you could crop a bit of picture until what's left is very close to 16:9 and then resize, or change the resizing etc. This would be pretty close 16:9 - Crop(10, 6, -10, -6).
And... if there's 54 pixels worth of black being cropped from the height when it's 1080p, there'd be less after downscaling, so ignoring the width for the moment, these would crop roughly the same amount of black if the source was 1080p.

Crop(0, 26, -0, -26)
Spline36Resize(1280, 720)

Spline36Resize(1280, 720)
Crop(0, 16, -0, -16)

I had a look at the ffmpeg documentation for scale until I started having too many suicidal thoughts, which began even before I managed to find it, and it has all sorts of options I'm guessing at, but the way I read it, and assuming you're actually using scale=-2:720, the height is resized to 720 and the width chosen based on that, so I assume you're cropping 24 pixels from the height, therefore it ends up as 696 when you're cropping second.

The standard way to do things in ffmpeg appears to be this:

Apply cropping, then
scale=1280:-2
And after cropping the width will always be 1280 and the height resized to suit. Or you can specify a width and height but I'd have to play around to find out whether it can stretch the picture or ffmpeg prevents that by default etc.

One thing I'm not sure about with ffmpeg, is always determining the default. Sometimes it's specified, but for "force_original_aspect_ratio", there's three choices. How do you know what the default one is? https://ffmpeg.org/ffmpeg-all.html#scale-1

I ask because after copping there's different ways to deal with any aspect distortion. YV12 video must have dimensions divisible by at least 2, but some programs default to 4 for backwards compatibility, and in the XVid days sticking to 16 was safest, but for example...

After cropping, the resized height to prevent distortion should be 714. The program defaults to a mod4 height so it resizes to 1280x712. You can maybe adjust the cropping a little, or fudge the pixel aspect ratio to compensate (it'll be encoded at 1280x712 but should display as 1280x714) or you accept some distortion, which in this example is not enough to care about, but I'm not sure what ffmpeg does by default.

An easy way to check if the storage resolution and display aspect ratio are different is to ask MPC-HC. If they're not the same, the display aspect ratio is shown in brackets.

The main advantage of cropping second is it's hard to go wrong with standard formats. You know 1920x1080 can be resized to 1280x720 as they're the same aspect ratio, and from there you can crop freely without having to worry about distorting the picture.

[ZetaStax]

In this example my source is bluray stored at 1920x1080 so PAR is 1:1. I'm not sure I believe force_original_aspect_ratio is disabled by default, and it should make it easier to stick to the target resolution. Because in my example the cropping for the original 1080p frame is 8:16:2:16 I'd have to switch to a scale of 1280:-2 instead of -2:720 so it doesn't exceed 1280x720 but I believe force_original_aspect_ratio can make that decision on its own so I believe this could be the best solution.

I however just find out that it does nothing to ensure that the final resolution will be mod2, resulting for example in a 1280x703 resolution and forcing me to play with the cropping until the downscale is mod2, not ideal. But force_divisible_by should do the trick.

[hello_hello]

A cropping of 8:16:2:16 should give you an aspect ratio of 1.8225 so it's easy enough to work out how close the output resolution is to that, and you can always check with MPC-HC to see if there's also an aspect ratio being set or if the resolution and aspect ratio are the same.

force_original_aspect_ratio would change the output pixel aspect ratio if necessary, the way I read it, but it doesn't appear you can go too far wrong.

[ZetaStax]

Yes, the final aspect ratio using force_original_aspect_ratio is 1.8234 (1280x702). When cropping after the aspect ratio is 1.8276 (1272x696) while cropping first and downscaling at 720 height gives an aspect ratio of 1.8222 (1312x720). But to be honest those minor deviations are almost impossible to tell, and maintaining the aspect ratio exactly is impossible, you need to round the numbers at some point.

[hello_hello]

I don't think force_original_aspect_ratio would effect the output resolution, just the pixel aspect ratio, which can be almost anything as it's expressed as a fraction.

Having said that, for a mod2 height, which ffmpeg seems to be using (702), the height will be rounded up or down to the nearest mod2 and shouldn't differ from the "perfect" height by more than a pixel, so you're right about it not being enough to worry about.

You can test if force_original_aspect_ratio works as I think it does by enabling it and specifying a silly width and height for encoding. 720x720 for a 16:9 source, that sort of thing. It should encode as 720x720 and display as 16:9, but that's just something to play around with if you're interested.

[ZetaStax]

I'm sorry, but your guess was wrong It doesn't affect the PAR. But like I said this source has a PAR of 1:1 I'm curious to see how a source with a higher PAR would be affected by the rescaler.

[hello_hello]

Yeah I digested that badly yesterday, although I was sure I'd read something about changing the PAR. Eventually I looked in the right spot.....

I'm surprised an ffmpeg user hasn't joined the thread. There must be a few of them here.

38.166 scale
Scale (resize) the input video, using the libswscale library.
The scale filter forces the output display aspect ratio to be the same of the input, by changing the output sample aspect ratio.

Not in the section I remembered, but it still seems fairly unambiguous. If it's not adjusting the SAR automatically for you, chances are it doesn't unless you specify an input aspect for the source, or it's a new moon and Mars isn't aligned with Venus..... or something.

The two resizing "make pixels square" examples both include an input aspect, which is only circumstantial evidence, but wouldn't it be nice if the docs said something like "The scaler filter can force the output display aspect...." or "When you do "X", the scaler filter might... blah... blah... unless setsar=1/1...." or whatever the conditions are.... I think I'll go with that for the moment.

I've used ffmpeg for audio encoding a fair bit, but almost never for video. I should run a few little test encodes myself to make sure I know at it works. One day....

[ZetaStax]

Yeah this doc could be clearer, that's for sure. I believe what it says is that the using the scale filter with default parameters will change the aspect ratio by default (if you provide a target aspect ratio or a target dimension) unless you use -n for the width or height, or force_original_aspect_ratio in which case it will preserve the source ratio.