Random video degradation in different encoding options

[gelo333]

I use bitrate reduction, with downscale, denoise (KNLmeans) and sharpener (Finesharp). In various options. Some frames have defects, as in the examples.

Codecs libx264 or x264 Patman, downscale Spline36ResizeMT.

1. Original. without defect on the second frame.


2. with downscale and Finesharp. defect on the second frame. noticeable on the face.


3. with downscale, without Finesharp. without defect on the second frame.


Such cases occur repeatedly, in different variants, and with different combinations of filters. Sometimes both frames are problematic, sometimes there is a problem frame BEFORE, sometimes there is a problem frame AFTER, sometimes both frames are normal. Sometimes a problematic frame is duplicated from the original, sometimes it is corrected, sometimes a good frame of the original is made bad.

Even if the problem is found in some filter, or in the settings of the codec or filters, or the reason is in the source, why is there such a strong uncontrollable difference in encoding by one codec? Is this a feature of H264?

[poisondeathray]

2. with downscale and Finesharp. defect on the second frame. noticeable on the face.

Finesharp enhances noise and defects. I'd say that's the expected result typical of finesharp with too high settings, and too low bitrate

If you encode lossless, and similar problems are there, then a filter settings issue .

3. with downscale, without Finesharp. without defect on the second frame.

But other defects and loss are present. It looks like crap too. A different kind of crap. Eyes are completely changed. Look at specular highlights, his left eye in the shadow looks like someone else's eye and pointing in the other direction. Details in original missing. Obviously too low bitrate

Such cases occur repeatedly, in different variants, and with different combinations of filters. Sometimes both frames are problematic, sometimes there is a problem frame BEFORE, sometimes there is a problem frame AFTER, sometimes both frames are normal. Sometimes a problematic frame is duplicated from the original, sometimes it is corrected, sometimes a good frame of the original is made bad.

Even if the problem is found in some filter, or in the settings of the codec or filters, or the reason is in the source, why is there such a strong uncontrollable difference in encoding by one codec? Is this a feature of H264?

The distribution might be because of your encoding settings or it's on a GOP boundary. If you want more consistency between frames, use I-frame only (use lots of bitrate for similar percieved quality) or adjust your IP/PB ratios closer, so that p and b-frames get more bitrate (again less efficient, requires more bitrate for certain level of percieved quality)

It's the nature of lossy temporal encoding, and/or the effect of poor filter settings. Fix your settings and/or use adequate bitrate

[Selur]

Just to be sure: Neither your CPU, memory nor GPU are over- or underclocked or run hot, correct ?

[gelo333]

2. with downscale and Finesharp. defect on the second frame. noticeable on the face.

Finesharp enhances noise and defects. I'd say that's the expected result typical of finesharp with too high settings, and too low bitrate

If you encode lossless, and similar problems are there, then a filter settings issue .

Thanks for answers.

Look at the first frame. Finesharp did not affect him in any way, he is without garbage. I've tried different finesharp settings and it doesn't seem to make any difference to the problem. Problems with garbage and blocking sometimes occur in one frame or another. Sometimes I don't use Finesharp, but the problematic frames are there. And sometimes, on the contrary, there are no problematic frames. And, oddly enough, this is caused by increasing sharpness in Finesharp. I also tried increasing the bitrate. The sources are also different. I don't see any logic, it happens randomly.

But other defects and loss are present.

Of course, there will be a decrease in picture quality with a decrease in bitrate. But in this case, these are some kind of faulty frames. Which fall out of the general range of low-quality frames.

The distribution might be because of your encoding settings or it's on a GOP boundary

My settings are keyint=30, keyint_min=5. CQP, let's say, 26, but I tried other values.

If you want more consistency between frames, use I-frame only (use lots of bitrate for similar percieved quality) or adjust your IP/PB ratios closer, so that p and b-frames get more bitrate (again less efficient, requires more bitrate for certain level of percieved quality)

Настройки IP по умолчанию для x264 medium. Need to reduce the b-frame and p-frame values?

[gelo333]

Just to be sure: Neither your CPU, memory nor GPU are over- or underclocked or run hot, correct ?

The hardware is not overclocked. Cooling is normal. I don’t see any overheating of the processor and video card; they are far from the maximum temperatures. It's possible only on the motherboard, but it's unlikely.

[poisondeathray]

Look at the first frame. Finesharp did not affect him in any way, he is without garbage. I've tried different finesharp settings and it doesn't seem to make any difference to the problem. Problems with garbage and blocking sometimes occur in one frame or another. Sometimes I don't use Finesharp, but the problematic frames are there. And sometimes, on the contrary, there are no problematic frames. And, oddly enough, this is caused by increasing sharpness in Finesharp. I also tried increasing the bitrate. The sources are also different. I don't see any logic, it happens randomly.

Of course, there will be a decrease in picture quality with a decrease in bitrate. But in this case, these are some kind of faulty frames. Which fall out of the general range of low-quality frames.

Increasing sharpening requires more bitrate

For the "random" distribution - chances are, the more severely degraded and problem frames with more artifacts (and/or less detail) have lower bitrate, are b-frames. ie. it's not really "random". It's bitrate distribution (or lack of), and macroblock type

You can check the frametype on your encode with ffms2, ffinfo() . If you were to use a stream analyzer, you could see the bitrate distribution per frame or mb

The default IP and PB ratios are 1.4 and 1.3 . I frames get approx, 1.4x more bitrate than P frames. P frames get approx 1.3x more than B. So B are going to look worse every time. You want to adjust the ratios closer to 1 for more even distribution .

GOP boundary matters, because the new GOP is an I-frame, higher quality. Whenever that occurs, you get a bump in quality . With low bitrate encodes it's more visible. The artifact is called "keyframe pumping" because of the fluctuation in quality

[gelo333]

I looked at ffinfo. The problem seems to be with the frame type. This is not easy to figure out. For example, in the Original, frame number 100 looks like trash, if the output video takes it as an I-frame, then this frame and the following frames B and P will also look like trash? I can't clearly formulate the question.

There is one obvious question though. Why do I only change the Finesharp settings, and in one case frame number 100 is B-frame, and in the other case it is P-frame?

[poisondeathray]

I looked at ffinfo. The problem seems to be with the frame type. This is not easy to figure out. For example, in the Original, frame number 100 looks like trash, if the output video takes it as an I-frame, then this frame and the following frames B and P will also look like trash? I can't clearly formulate the question.

Most of the time, but not always.

If the original frame from which the I frame was taken was "bad", but the original other frames were ok, and if you had adequate bitrate, then the other encoded frames can look ok. The reason is the P and B frames store the difference between predicted. If the original frames from which the P or B frame were "good", then you would have high residuals (ie. you need lots of bitrate for it to look like the original), because you're going to have less accurate prediction with a bad I frame in that GOP

There is one obvious question though. Why do I only change the Finesharp settings, and in one case frame number 100 is B-frame, and in the other case it is P-frame?

B frames are usually placed when there is higher redundancy (ie. frames look similar), and you can benefit from "easy" prediction in both directions . If I were to guess, you changed finesharp settings , likely higher, you created more noise, and therefore the frames look more different to each than before. More likely to place a P or I frame in that case

You can adjust the other settings as b-frame bias or even use a qp file if you wanted ultimate control over frametype placement

[gelo333]

If the original frame from which the I frame was taken was "bad", but the original other frames were ok, and if you had adequate bitrate, then the other encoded frames can look ok. The reason is the P and B frames store the difference between predicted. If the original frames from which the P or B frame were "good", then you would have high residuals (ie. you need lots of bitrate for it to look like the original), because you're going to have less accurate prediction with a bad I frame in that GOP

I don't fully understand. Real example:

Original video
frame 107 = P-frame (bad)
frame 108 = P-frame (got better)
frame 109 = P-frame (got better)
frame 110 = I-frame (good)

Problem video
frame 107 = I-frame (bad)
frame 108 = B-frame (bad)
frame 109 = B-frame (bad)
frame 110 = B-frame (bad)
Every new frame becomes minimally better, which is almost unnoticeable and can only be seen when zoomed in. Or maybe not, and it only seems so.

Normal video
frame 107 = B-frame (good)
frame 108 = B-frame (good)
frame 109 = P-frame (good)
frame 110 = B-frame (good)

Frame 107 is the beginning of the same scene, to make comparison easier. But for some reason the original video did not count this as an I-frame.

[poisondeathray]

Frame 107 is the beginning of the same scene, to make comparison easier. But for some reason the original video did not count this as an I-frame.

A beginning of a scene does not necessarily have to be an I frame. 1) Sometimes the difference is not large enough for a scenecut threshold. 2) Open GOP - sometimes it's an "i" frame not an IDR I-frame . An "i" frame in not a true keyframe. ffms2 and ffmpeg do not always accurately distinguish between them. 3) Also, you are viewing display order, not coded order - often the encoded order is different from the display order

Original video
frame 107 = P-frame (bad)
frame 108 = P-frame (got better)
frame 109 = P-frame (got better)
frame 110 = I-frame (good)

**Ignore whether or not the source is I,P,B frame. Frame type is irrelevant in the source because it gets decoded to uncompressed data before encoding. Just pay attention to "good" or "bad" for the source. A bad source frame can NEVER become good with straight encoding (no filtering). You are starting with a bad source frame 107 in the original video - so it will never get better with straight encoding. You can look for a better source, or try filtering or reparing it

[Sharc]

Maybe somehow related to this observation?
https://forum.doom9.org/showthread.php?t=175662

[gelo333]

**Ignore whether or not the source is I,P,B frame. Frame type is irrelevant in the source because it gets decoded to uncompressed data before encoding. Just pay attention to "good" or "bad" for the source. A bad source frame can NEVER become good with straight encoding (no filtering). You are starting with a bad source frame 107 in the original video - so it will never get better with straight encoding. You can look for a better source, or try filtering or reparing it

In my example, a problem video and a normal video, both processed with KNLmeans and Finesharp filters. If it is important, I can make examples with different usage, without filters, with KNLmeans without Finesharp, with Finesharp without KNLmeans, with KNLmeans and Finesharp, while Finesharp will be with different settings. But I can say in advance that the options in this range are if we consider 2 border frames: the first frame is bad - the second frame is good; the first is good - the second is bad; the first is good - the second is good; The first is bad - the second is bad.

[gelo333]

Maybe somehow related to this observation?
https://forum.doom9.org/showthread.php?t=175662

I started reading from the end. Can the build made by jpsdr be used in FFMPEG? I saw mention of FFMPEG in Versions_Libpack.txt in the build files. Or at least in the Staxrip app.

[gelo333]

Can I use FFInfo to see the bitrate allocated per frame?

[poisondeathray]

Maybe somehow related to this observation?
https://forum.doom9.org/showthread.php?t=175662

I started reading from the end. Can the build made by jpsdr be used in FFMPEG? I saw mention of FFMPEG in Versions_Libpack.txt in the build files. Or at least in the Staxrip app.

That's for BD VBV and sliced threads - Not your scenario - you're not using VBV restrictions

Can I use FFInfo to see the bitrate allocated per frame?

FFInfo does not provide that information . You can use ffprobe or h264_parse

[gelo333]

So what should I do anyway? Should I put up with such defects? Because, if in one video I guessed right with the filter settings, if the codec successfully arranged the types of frames, and problematic frames with garbage did not appear, then in another video they may appear, since the trend has not gone away. The bitrate of the problematic video is, for example, 1000 kbps. I don't think it's bad enough for 720p video to cause problems. Increasing -keyint seems to improve things a little, but it's not a solution.
I will clarify once again that in the problem video there is less sharpness, and in the normal video there is more sharpness. Is there a possibility that it is not the sharpening, but Finesharp that creates such problems due to its nonlinearity? Simply trying a different sharpener will not work, as you need to create a similar level of sharpness.

[sophisticles]

I use bitrate reduction, with downscale, denoise (KNLmeans) and sharpener (Finesharp). In various options. Some frames have defects, as in the examples.

Very shocked to hear that reducing the bitrate, downscaling, denoising and sharpening a video results in frames with defects.

I wonder what in the world the reason can be?

[gelo333]

I was unable to make a frame log in FFprobe, but I am looking at the FFMPEG -report encoding log. The information does not match the data from AvsPMod, this is probably correct.

Problem Video:
frame 107 (bad), I - 30221 bytes
frame 108 (bad), P - 3803 bytes
frame 109 (bad), B - 751 bytes
frame 110 (bad), B - 579 bytes

Normal Video:
frame 107 (good), B - 688 bytes
frame 108 (good), B - 190 bytes
frame 109 (good), B - 227 bytes
frame 110 (good), P - 11823 bytes

In the problematic video, the bitrate is even higher. Which is logical, because in bad frames there is a grid and garbage, instead of a smooth image in good frames.

[poisondeathray]

That's the expected result

The garbage is from your filtering . The temporal inconsistency in frame quality is expected from your encoding settings - CQP 28 - obviously bitrate starved

If you check a lossless preview or encode - it will be consistently oversharpened crap, instead of inconsistently oversharpened crap.

When you sharpen something you need more bitrate to preserve the image , otherwise you get splotchy bitrate starved artifacts as you have demonstrated.

Choose a strategy. If you want low bitrate range encoding, most people actually denoise and blur - not sharpen.

You can use other settings improve encoding efficiency (higher quality at lower bitrates, by using slower encoding settings), but higher bitrate overall always fixes encoding specific problems and the easiest fix

[gelo333]

Small correction, I'm using CQP 26. Tried CRF, it's even worse.

When you sharpen something you need more bitrate to preserve the image , otherwise you get splotchy bitrate starved artifacts as you have demonstrated.

Why are these artifacts uneven? This is the crux of the problem, that coding becomes unpredictable.

You can use other settings improve encoding efficiency (higher quality at lower bitrates, by using slower encoding settings), but higher bitrate overall always fixes encoding specific problems and the easiest fix

I increased the CQP to 20. The file size increased by 2.5 times. But the problems have not gone away.

Choose a strategy. If you want low bitrate range encoding, most people actually denoise and blur - not sharpen.

I'm just trying to do something different. The image itself becomes blurred when the bitrate is lowered, this needs to be compensated, and done correctly. Why not? Reducing the bitrate is a fairly common task, why not improve it?

[poisondeathray]

Why are these artifacts uneven? This is the crux of the problem, that coding becomes unpredictable.

Are the artifacts uneven with lossless encoding ? If yes, then it's either a source or filter problem . Fix your source or filters.

If no, then it's an encoding issue . Temopral lossy compression. I,P,B frames. Some frames get 10-100x less bitrate. That's going to affect quality between frames.

Low bitrates can cause artifacts to become uneven within frames - spatial quality issues

You can use other settings improve encoding efficiency (higher quality at lower bitrates, by using slower encoding settings), but higher bitrate overall always fixes encoding specific problems and the easiest fix

I increased the CQP to 20. The file size increased by 2.5 times. But the problems have not gone away.

Maybe 2.5 is way too low for your sharpening filters . Maybe you need 10x or 20x. Sharpened noise is difficult to compress

See above . Rule out whether it's a filter / source issue or encoding issue

Try --qp 0. If problems go away, you have your answer

Choose a strategy. If you want low bitrate range encoding, most people actually denoise and blur - not sharpen.

I'm just trying to do something different. The image itself becomes blurred when the bitrate is lowered, this needs to be compensated, and done correctly. Why not?

You can't have everything at low bitrates . Something has to give. When you figure it out, you will win the Nobel prize in video compression processing

[gelo333]

I set QP=0. It has gotten better, but not absolutely. Worse than the best variant, since I use noise reduction before the sharpener. And in this case, the frame types have already been rearranged. QP=0 simply placed P-frames everywhere. But there is no unpredictability, the changes have become logical, yes.

[poisondeathray]

I set QP=0. It has gotten better, but not absolutely. Worse than the best variant, since I use noise reduction before the sharpener. And in this case, the frame types have already been rearranged. QP=0 simply placed P-frames everywhere. But there is no unpredictability, the changes have become logical, yes.

Finally, that's useful information

--qp 0 is lossless temporal compression. No b-frames are used, but the frame type does not matter - because it's lossless. This is the output of your filtered version in 100% quality. This also confirms that the unpredictability has nothing to do with your filters used, but rather is from the lossy temporal encoding settings used.

If it's not absolutely better, and worse than the "best variant" - then the problem is your filtering choice, because you've removed encoding problems from the equation

From that starting point, you decide how to improve your filtering choices, then go on to make some encoding choices in terms of trade offs

[gelo333]

It's difficult to get the defect I'm talking about. This is not the most obvious example. I intentionally made the video low quality to make it clear. Look at the vertical and horizontal stripes at the bottom of the face.

Problem video. Full settings:

-c:v libx264 -qp 26 -preset medium -g 30 -keyint_min 5
-------
LSMASHVideoSource("*.mp4")
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\ResampleMT.dll") Spline36ResizeMT(480, 270, prefetch=4)
KNLMeansCL(D=1, A=1, h=2, device_type="auto")
FineSharp(mode=1, sstr=2.2, cstr=0.2, xstr=0, lstr=1.8, pstr=7, ldmp=0.1)
Prefetch(6)



I only change FineSharp(mode=1, sstr=1.9, cstr=0.1, xstr=0, lstr=1.2, pstr=6, ldmp=0.1) and the video becomes free of these defects.


This is a video with QP=0. It's better than the problematic video, but worse than the second one.

[poisondeathray]

Problem video. Full settings:

-c:v libx264 -qp 0 -preset medium -g 30 -keyint_min 5
-------
LSMASHVideoSource("*.mp4")
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\ResampleMT.dll") Spline36ResizeMT(480, 270, prefetch=4)
KNLMeansCL(D=1, A=1, h=2, device_type="auto")
FineSharp(mode=1, sstr=2.2, cstr=0.2, xstr=0, lstr=1.8, pstr=7, ldmp=0.1)
Prefetch(6)

I don't understand, do they all use --qp 0 ? The script with -c:v libx264 -qp 0 -preset medium -g 30 -keyint_min 5 and it produces the "problem video" ? Then what is the difference between 1st and 3rd for the workflow ? Maybe the --qp 0 was a typo for the 1st video ?

Another problem is jpg . e.g Are the artifacts caused by jpg compression, or the filter(s) ?

The point of using lossless is to examine the filter chain, without confounding factors such as compression settings, jpeg compression etc... Otherwise you don't know what is causing what.

eg. You downscale to 480x720 , but the jpg screenshots are 1280x720 . But what is being used to upscale ?

ie. You add too many variables to the analysis. If you change 15 things at once , which one is responsible ? It's very poor scientific method. Poor methodology is the reason you came to the wrong conclusion about "random video degradation" . --qp 0 is to see the effect of the filter(s) and their settings, eliminating the effect of lossy compression - so you don't come to the wrong conclusions . jpeg adds lossy compression so it's not as useful. There is no point is using qp0 for analysis, if you use jpg or upscale image

If you can do a proper logical analysis, make some valid points, maybe someone will try to improve the filters. Right now it's a mess of a presentation.

[gelo333]

I don't understand, do they all use --qp 0 ? The script with -c:v libx264 -qp 0 -preset medium -g 30 -keyint_min 5 and it produces the "problem video" ? Then what is the difference between 1st and 3rd for the workflow ? Maybe the --qp 0 was a typo for the 1st video ?

I'm sorry. Yes, I made a mistake. 1 video qp=26, 2 video qp=26, 3 video qp=0.

Another problem is jpg . e.g Are the artifacts caused by jpg compression, or the filter(s) ?

no, on the contrary, JPG smoothed out the problem.

If you can do a proper logical analysis, make some valid points, maybe someone will try to improve the filters. Right now it's a mess of a presentation.

In the original size everything is the same, but it will break your eyes to look at it.

[davexnet]

"no, on the contrary, JPG smoothed out the problem."

yes, but if you're trying to evaluate or judge an issue with your video, using jpg is a poor choice.
It adds it's own compression artifacts on top of what's there from the video

[gelo333]

The defects I'm talking about are clearly visible in these JPG files. I see it.

Okay, original PNG:

[poisondeathray]

Yes, the defects are clearly visible in the jpg. That's not the point; the point is about doing a proper analysis in general .

Test 1 thing at a time and examine the result. Don't add other variables like lossy compression or scaling the result if you're examining the effect of a specific filter. This reduces the cause - effect relationship and you can arrive at the wrong conclusions such as earlier with "random degradation"

If you're providing feedback for a script/plugin developer, they don't want to see all those other problems and extraneous factors bundled in your presentation.

[gelo333]

If defects randomly appear only when Finesharp is changed, then what could be the reason? I don't know.