Is this compression or codec settings causing this?

[killerteengohan]

I'm just curious if this is from x264 codec settings.

Here is a comparison image you can flip through with mouse click or arrow keys on keyboard.

Blu-Ray Source Preview vs 15.2Mbps encode.
https://slow.pics/c/WX88wjr1

Blu-Ray Source Preview vs 25Mbps encode.
https://slow.pics/c/FClNh1F8

15.2Mbps encode vs 25Mbps encode.
https://slow.pics/c/XRupg275


Notice in the uncompressed preview from avs preview, it's still grainy and detailed, while in the compressed encode it kinda looks like there's blocks or smearing that removes the grain and detail. What I am referring to is easiest to see on the brown shell of the turtle. The cleaner look is nice, but not when this is moving all around like crazy and very visible in the video. It's distracting, while if I watch the m2ts, it looks fine.

The source is 25Mbps and the encode is 15.2Mbps so I didn't compress it a ton. This seems to happen even when I increase the encode bitrate to the same 25Mbps as the source. (Which is practically no compression at all?) I am confused as to why it would do this even if I use about the same bitrate as the source material.
It's only slightly better looking, but still does that. This is a pretty grainy source so I know it wont compress that well, but I didn't expect it to end up like this. It kind of looks like I am applying a strong DNR to it, but I am not. I'm only cropping black bars from the top and bottom.

Is this just typical x264 compression, or is it from a codec setting that I can change to help with this?

program --level 4.1 --preset slow --pass 2 --bitrate 15187 --stats ".stats" --threads 8 --deblock 1:1 --qpmin 8 --qpmax 28 --vbv-maxrate 35000 --subme 7 --partitions all --trellis 2 --psy-rd 0.40:0 --no-dct-decimate --no-fast-pskip --output "output" "input"

[poisondeathray]

You used --tune animation . But that's intended more for "cleaner" animation types, not something like this

To retain the grain, the important settings for x264 (in general) are higher --psy-rd and trellis, but it's reduced to 0.4:0 from 1:0 in the default animation preset, and deblock is increased to 1:1 . Both those work against what you're trying to achieve here

--tune film or --tune grain would probably work better as a starting point than --tune animation . But grain is difficult to compress. Also, if the rest of the animation has high complexity, lots of motion it might not be enough bitrate with any settings. At some fixed bitrate you might get other artifacts like splotches in the grain pattern

[killerteengohan]

You used --tune animation . But that's intended more for "cleaner" animation types, not something like this

To retain the grain, the important settings for x264 (in general) are higher --psy-rd and trellis, but it's reduced to 0.4:0 from 1:0 in the default animation preset, and deblock is increased to 1:1 . Both those work against what you're trying to achieve here

--tune film or --tune grain would probably work better as a starting point than --tune animation . But grain is difficult to compress. Also, if the rest of the animation has high complexity, lots of motion it might not be enough bitrate with any settings. At some fixed bitrate you might get other artifacts like splotches in the grain pattern

Okay I tried the different settings at the 15.2Mbps bitrate.

Increasing psy-rd to 1.0 does seem to preserve really small details slightly better than before when it was set at 0.4. Things did smear a little less, but it looks like it added artifacts that were not previously there which sucks because the detail does look slightly better. Some things got slightly smeared away more than before. It seems lines got a little more jagged in places as if it introduced some aliasing. The entire image is also slightly sharpened. Personally I prefer the 0.4 after comparing the two.

Lowering the deblock to 0:0 is a bit mixed. This does seem to preserve small details slightly better than before when it was set at 1:1. Instead of clean looking smearing/blurring, it looks like blocky smearing/blurring. Some part of the image look better (most noticeably backgrounds), and some parts (most noticeably lines) look worse or are completely removed/smudged out. Personally, I prefered the 1:1 deblock setting after comparing the two.

Doing both of the above retained the grain the best from the looks of it. This appears to have retained more detail than either of the above changes individually and better than my original settings. It looks like some slight halo's/ringing artifacts are now visible that were not before. I do see some artifacts that weren't there before, its not as visible as just changing psy-rd on its own though. If it weren't for the artifacts, I would pick this setting choice hands down because it seems closest to the sources details.


I tried doing a 40Mbps with my usual settings and it's not nearly as bad. It still does it a little bit, but it's not awful like the others. It's pretty close to the source.
https://slow.pics/c/sImCphiN
You would think with almost double the bitrate of the source material though that it would be pretty much lossless looking.


I guess when it comes to high grain sources, settings don't mean as much when compared to bitrate. I still have the question of why it does this even when the bitrate is almost double the sources though. I would think that would give a lossless 98%+ identical output. I assume this is possibly a limitation to x264 and it can never be 100% identical to the source.
EDIT: I tried a 0 CRF encode and it does indeed look about 99% the same as the source. It came out at 186Mbps for some reason though. That's WAY more than the source on the Blu-Ray disc. wtf it would need so much bitrate to be almost lossless for I have no idea.

Thanks for suggesting those 2 settings though. I learned some stuff about them today.

[jagabo]

I tried a 0 CRF encode and it does indeed look about 99% the same as the source.

No, it's 100 percent identical to the source (assuming you didn't perform any colorspace conversions or filtering).

It came out at 186Mbps for some reason though. That's WAY more than the source on the Blu-Ray disc. wtf it would need so much bitrate to be almost lossless for I have no idea.

Because the source is decompressed (about 600 Mb/s) then re-compressed by losslessly by x264. Lossless compression typically compresses to about 1/3 the size of the original. So, of course it's much bigger than the 25 Mb/s source file.

[killerteengohan]

No, it's 100 percent identical to the source (assuming you didn't perform any colorspace conversions or filtering).

It looks about 99% to me. I didn't filter it at all or change anything. It was just straight up 0 crf encode with the settings I shared earlier.

https://slow.pics/c/8i0h2kEc

Its pretty hard to see, but when you zoom in and look carefully, the sharpness levels and some pixels (most noticeably on the turtles nose) are slightly different. Have a quick look.

[jagabo]

I see the differences in your images. But if you do the encoding properly it is truly lossless. There was probably a colorspace conversion or something else that you were unaware of. I've done many CRF=0 encodings and there results were always identical. Not just visually, using software tools that could detect a single bit difference.

[poisondeathray]

Then something else is wrong. Maybe your settings didn't pass correctly, maybe some filter you didn't know about, something else in the chain, or you didn't take screenshot correctly

For example, the "source" image has a "+" overlaid on it, that is not on the encode - and it's there when you download the pics (not just the viewer)


For x264, --crf 0 is lossless, same as --qp 0

But not necessarily for x265 . You need to add --lossless switch

[poisondeathray]

Because the source is decompressed (about 600 Mb/s) then re-compressed by losslessly by x264. Lossless compression typically compresses to about 1/3 the size of the original. So, of course it's much bigger than the 25 Mb/s source file.

If that's the case, why cant x264 decompress it then recompress it at the same bitrate as the compressed source and look as good as the source did when done? It looked worse even at double the bitrate of the compressed source when encoded. I'm making the same bitrate as the compressed on disc source is, yet it looks worse. Does it not decompress it when it encodes normally and only does that when using 0 CRF?

Because after the 1st lossy encoding, bits are already discarded. It already has lost some quality, new artifacts generated. That's what "lossy" encoding means . You can't get back the original

[killerteengohan]

I see the differences in your images. But if you do the encoding properly it is truly lossless. There was probably a colorspace conversion or something else that you were unaware of. I've done many CRF=0 encodings and there results were always identical. Not just visually, using software tools that could detect a single bit difference.

Unless this is doing colorspace converting, I didn't do any of that to it.

LoadPlugin("C:\Program Files (x86)\MeGui\tools\dgavcindex\DGAVCDecode.dll")
AVCSource("C:\Users\Admin\Desktop\Encoding\The Swan Princess\BDMV\STREAM\00002.dga")

[killerteengohan]

Then something else is wrong. Maybe your settings didn't pass correctly, maybe some filter you didn't know about, something else in the chain, or you didn't take screenshot correctly

For example, the "source" image has a "+" overlaid on it, that is not on the encode - and it's there when you download the pics (not just the viewer)


For x264, --crf 0 is lossless, same as --qp 0

But not necessarily for x265 . You need to add --lossless switch

It's not the screenshots, I know how to take proper screenshots. That + was mouse cursor in paint program being in the way when i took a screenshot of the zoomed image with prtscr button. I fixed that literally a minute or two after I noticed it. Theres no + in it.

maybe it's a setting I had set for it when trying the lossless. Could it be the qpmax or vbv-maxrate?

program --preset slow --crf 0 --threads 8 --deblock 1:1 --qpmax 1 --qpstep 1 --vbv-maxrate 90000 --subme 7 --partitions all --trellis 2 --psy-rd 0.40:0 --no-dct-decimate --no-fast-pskip --output "output" "input"

[poisondeathray]

LoadPlugin("C:\Program Files (x86)\MeGui\tools\dgavcindex\DGAVCDecode.dll")
AVCSource("C:\Users\Admin\Desktop\Encoding\The Swan Princess\BDMV\STREAM\00002.dga")

DGAVCDecode is depreciated - it is known to give errors for some streams. Possibly it's contributing to differences. I'd use something else like lsmash, ffms2

maybe it's a setting I had set for it when trying the lossless. Could it be the qpmax or vbv-maxrate?

program --preset slow --crf 0 --threads 8 --deblock 1:1 --qpmax 1 --qpstep 1 --vbv-maxrate 90000 --subme 7 --partitions all --trellis 2 --psy-rd 0.40:0 --no-dct-decimate --no-fast-pskip --output "output" "input"

It shouldn't. For x264 - as soon as --qp 0 or --crf 0 is entered, it should be lossless . All the other settings will be automatically set if they interfere with it being lossless or not - whether or not they are explicitly on the commandline or "hidden" as a default . eg. psy options, bframes will automatically be disabled, etc..

eg You have --psy-rd 0.40:0 , but if you look at mediainfo it will report differently, because it's automatically adjusted

You can check with just --qp 0, nothing else. That definitely is lossless.

[jagabo]

If look closely at your two screen shots the chroma is shifted right by a few pixels in the x264 encoding.

[killerteengohan]

If look closely at your two screen shots the chroma is shifted right by a few pixels in the x264 encoding.

Your right, I do see that after you bring it up.

I think I read something about its a known bug with avisynth 2.6 and 2.61 when I went looking for the avisynth+ you told me about a couple days ago. I could be remembering wrong. I wonder if using avisynth+ would fix?

http://avisynth.nl/index.php/Known_Issues

[killerteengohan]

DGAVCDecode is depreciated - it is known to give errors for some streams. Possibly it's contributing to differences. I'd use something else like lsmash, ffms2.

What kind of error could it give that I should be aware of? It seems to have ran fine for me the past couple of years of using it.

I heard lsmash was buggy or not frame accurate or something and have been afraid to use it since.

[poisondeathray]

DGAVCDecode is depreciated - it is known to give errors for some streams. Possibly it's contributing to differences. I'd use something else like lsmash, ffms2.

What kind of error could it give that I should be aware of? It seems to have ran fine for me the past couple of years of using it.

I heard lsmash was buggy or not frame accurate or something and have been afraid to use it since.

Definitely PAFF streams, there will be macroblocking and decoding errors.

But even "normal" streams from anywhere can have random decoding errors, especially newer streams.

Others won't open properly at all or it will crash on them

I used to keep a folder videos that it has problems with, but don't bother anymore... Because this is well known. Not going to be fixed, author said as much, and has withdrawn the tool 10 years ago.



Newest version of lsmash is good, frame accurate

[killerteengohan]

I just tried L-smash, and not only does it look more lossless at 0 crf than the other encode with DGAVCDecode, but it also doesn't have that slight chroma shift either. I guess it must have been the decoder doing those things.