x264 banding/posterizing (Quick Sync also)

[ziggy1971]

I'm trying to rip a TV series from bluray to HDD. While watching the mkv's on my HTPC I've noticed some bad banding/posterizing. I checked the originals and the areas in question are very dark, candle lit room and grainy.

This is the command line from MeGUI, all settings are at their defaults except the CRF.

program --crf 19.0 --output "output" "input"

Other gradients such as skin tones seem to be pretty good, but it's the darker areas that are really "off" and make the picture look horrible.

The video is 1080p @ 23.976 fps

Any ideas on how to improve the image quality? I'm no expert with the many settings available in x264.

[sneaker]

How powerful is your HTPC? 10 bit encoding is the most effective way to counter banding but usually not compatible with hardware decoders. (more often with HEVC/H.265, though)

Other than that:
- decrease CRF
- put more bitrate into dark scenes --aq-mode 3
- http://forum.doom9.org/showpost.php?p=1565176&postcount=38

There's also debanding in some players, e.g. using madvr. (again: if your HTPC is powerful enough)

[jagabo]

Use a lower crf, a slower preset, and tune=grain. I never liked aq-mode 3 but aq-mode 2 shoudl help. Can you upload a short segment of your source that is giving you trouble?

[ziggy1971]

For my HTPC, I'm using a Core i3-6300 (Skylake HD 530) and I use Kodi as the media player. I've also tried Media Player Home Cinema, same results.

Is there a way to trim a section of the original x264 video or which codec should I use to transcode a section. I usually use CineForm.

[ziggy1971]

Attached are some clips; one is Lagarith, the other is CineForm and a clip of the original.

Funny thing is, I was trying out Quick Sync on Skylake for testing purposes to see if or whether any improvements have been made. I came across this problem with Quick Sync, so I checked it with MeGUI only to find that x264 (with the settings I used) are far worse than using Quick Sync with LA-ICQ with a quality of 23, Lookahead of 40 frames.

[sneaker]

Your PC is powerful, so 10 bit x264 or x265 should be fine to use for HD.

I've also tried Media Player Home Cinema, same results.

MPC-HC doesn't have debanding by default. You need to install madVR, select it in MPC-HC's settings ("Playback">"Output") and in madVR settings enable debanding. ("processing">"artifact removal")

Is there a way to trim a section of the original x264 video

mkvtoolnix. Open, go to "Output" tab and select "split mode" "by parts based on timecodes" and enter e.g. "00:01:00-00:02:00" to get minute 1 to minute 2.

[sneaker]

I encoded it with crf19 like you said. While it could benefit from lower crf/slower preset and maybe tune film/grain I don't see all that much banding specifically.

[sneaker]

10 bit preset veryslow tune grain at same bitrate for comparison:

[ziggy1971]

Here is approximately the same clip transcoded with the Quick Sync settings listed before.

I think I also noticed something while trimming the clips; the use of I-frames or GOP sizes, Ref frames and B-frames.

A Bluray source has different GOP sizes, not sure what it is, as well as different Ref frames.

I used the defaults, max key interval: 240
Ref frames: 3
B-frames: 3

Any advice on that?

Also, I think I'll look into setting up Kodi with madVR:
http://babylon.xbmc.org/showthread.php?tid=259188
http://forum.kodi.tv/showthread.php?tid=222576

[sophisticles]

Here is approximately the same clip transcoded with the Quick Sync settings listed before.

I think I also noticed something while trimming the clips; the use of I-frames or GOP sizes, Ref frames and B-frames.

A Bluray source has different GOP sizes, not sure what it is, as well as different Ref frames.

I used the defaults, max key interval: 240
Ref frames: 3
B-frames: 3

Any advice on that?

Also, I think I'll look into setting up Kodi with madVR:
http://babylon.xbmc.org/showthread.php?tid=259188
http://forum.kodi.tv/showthread.php?tid=222576

So it's true, QS has finally beaten x264. What software did you use for the QS test?

[KarMa]

So it's true, QS has finally beaten x264. What software did you use for the QS test?

Pretty bad comparison since the provided QS encoding is 40s long, and the provided source is 20s long. The QS encoding starts with frames not present in the provided source, along with the provided x264 encodings. Thought you had higher testing standards than this. (Going on and on about the perils of encoding short clips, while championing MSU's HEVC tests which were all short clips.) Just seems like you only have a problem with a test when it shows QS in a bad light.

OP needs encode his provided source with QS, not some approximation. The .mkv source in post #5, or provide the longer source.

[sneaker]

So it's true, QS has finally beaten x264. What software did you use for the QS test?

Eh, what? The QS encode looks much worse than the x264 ones. Like KarMa said you need a proper comparison at the same bitrate but still I have no idea how you'd come to your conclusion given the samples posted.

[jagabo]

Yes, the QS encoding has lost all grain and become very posterized. The x264 encoding has retained much more of that grain and is more true to the original.

[ziggy1971]

First, apologies for not posting the exact same clip for testing. I didn't even know whether it was possible to trim a clip from h264 until sneaker said how to do it. Otherwise, I would've started with that clip.

Nonetheless, be the clip 20 seconds or 30 seconds, the area in question is still in both clips. On a previous post, I did upload a longer clip only to have it scrutinized for being too long and that a short 10 second clip would suffice.

Moving on:
- Quick Sync encoding, I noticed the banding. LA-ICQ with a quality of 23, Lookahead of 40 frames

- x264 encoding, I noticed the same banding. - CRF 19.0 (All others - Default)
---- Suggestion --> Use a lower crf, a slower preset, and tune=grain (Result: Higher bitrate, longer encode time, larger file size)

In comparison, shouldn't I also increase the quality of Quick Sync, to make it a fair fight?

Here is a much longer clip, containing the same section as before. The clip is provided solely for the purpose of testing encoders. I have no intentions of infringing any copyrights. If there is any copyright issue, moderators, please remove this clip or ask me to do so.

I'll post some clips with the results soon.

[sneaker]

The main problem is the source. Is it the untouched BluRay? I believe it hasn't been released yet. If you believe there's banding in the source you have to run a debanding filter, e.g. flash3kyuu_deband().
Still, I'd be interested to see where in the screenshots I posted you see the banding problems.

(20s or 40s samples were both fine. The problem was that scene and bitrate between QS and x264 encode were different thus invalid for any comparison.)

[ziggy1971]

The main problem is the source. Is it the untouched BluRay? I believe it hasn't been released yet. If you believe there's banding in the source you have to run a debanding filter, e.g. flash3kyuu_deband().
Still, I'd be interested to see where in the screenshots I posted you see the banding problems.

(20s or 40s samples were both fine. The problem was that scene and bitrate between QS and x264 encode were different thus invalid for any comparison.)

The clip is a Bluray source (Game of Thrones Season 5 Episode 6: Bluray -> makemkv -> mkvmerge (to trim clip)

I didn't say or think there was banding in the source. I'm no video expert, perhaps the term banding or posterizing is inaccurate.

I guess this got sidetracked into an x264 vs. Quick Sync war after I noticed virtually the same results from both encoders.

I think it's a very good source clip. It's challenging for encoders, basically worst case scenario. IMHO

Here are some clips:
Quick Sync: vbr 5000 kbps la 40 - 0:01:00 encode time
x264: CRF 19.0 Medium - 0:02:25 encode time

[sneaker]

So, can you mark on one of the screenshots (using e.g. MS Paint) what you think your problem is? Doesn't make sense for us to talk about banding if that isn't what you really mean.

[ziggy1971]

It's the gradients of color.

I know this photo isn't from the video, but it clearly shows what I mean.
[Attachment 39323 - Click to enlarge]

Instead of a smooth transition of one color to another, it looks "blochy". Seems like large sections of the same color.

I believe banding/posterizing is the correct term. (as jagabo mentioned earlier)

The photos posted earlier have them, though not nearly as bad as this photo, and that's what I was referring to in the thread.

[ziggy1971]

So it's true, QS has finally beaten x264. What software did you use for the QS test?

Pretty bad comparison since the provided QS encoding is 40s long, and the provided source is 20s long. The QS encoding starts with frames not present in the provided source, along with the provided x264 encodings. Thought you had higher testing standards than this. (Going on and on about the perils of encoding short clips, while championing MSU's HEVC tests which were all short clips.) Just seems like you only have a problem with a test when it shows QS in a bad light.

OP needs encode his provided source with QS, not some approximation. The .mkv source in post #5, or provide the longer source.

@KarMa
The source video has been provided. Do your own testing, provide better results and come to your own conclusions. Otherwise a post like this is completely useless and insulting to sophisticles. Not to mention prevents others from asking questions or giving opinions.

This is a HELP forum. If you're going to provide usable results and answers, great. Otherwise, shut up and go find something else to do. Coming onto a thread, insult someone's point of view, then disappear is so not cool. Cowardly, in fact.

I'll await your results....

In the meantime:
I used the same source, encoded it with both x264 and Quick Sync and provided the settings I used. The difference between the two is NOT night and day.

[sneaker]

Do you find this sample better, worse or equal?

I don't really know how to combat what you dislike. I think the grain has this weird effect on the skin. Maybe some degraining, then debanding would be more what you want?

[ziggy1971]

Actually, my goal is simple; get as close to the original as possible (within limits, of course).

If the results, using either Quick Sync or x264, will achieve equal (or at least similar) quality, then so be it.

If I can transcode videos @100+ fps (@ 1080p) using Quick Sync and get really good results, I'll accept that rather than transcoding @30+ fps using x264 just so I have that tiny bit of extra quality which, in all likelyhood, wouldn't even be visible during playback.

I'm not asking perfection. If I wanted the absolute best, I'd watch the original Bluray and save this hassle.

[ziggy1971]

Quick Sync:
Various Settings
LA-VBR - Lookahead - 40 frames, bitrates - as listed below; few other settings
I checked 15000kbps, 11000kbps, 9000kbps, 6000kbps and 5000kbps. At 5000kbps the image really started to band/posturize

Compare these to the source file uploaded earlier.
People will have various opinions...

File Name ------------------------------------------------- Encode time - Ave. Bitrate
Quick Sync
S05-E06 - Unbowed, Unbent, Unbroken-5000.mkv - 0:01:08 - 4901 Kbps
S05-E06 - Unbowed, Unbent, Unbroken-6000.mkv - 0:01:08 - 5895 Kbps
S05-E06 - Unbowed, Unbent, Unbroken-9000.mkv - 0:01:06 - 8808 Kbps

x264 - Using CRF (I've never used x264 for 2-pass or constant bitrate)
S05-E06 - Unbowed, Unbent, Unbroken-CRF-19.6.mkv - 0:02:31 - 4930 Kbps
S05-E06 - Unbowed, Unbent, Unbroken-CRF-18.8.mkv - 0:02:37 - 5928 Kbps
S05-E06 - Unbowed, Unbent, Unbroken-CRF-17.4.mkv - 0:02:55 - 8845 Kbps

Will update as I get more files...

[ziggy1971]

If anyone has another source file I'd be happy to do the encoding for more comparison examples.

[ziggy1971]

Just found this link, not sure if it still applies though:

https://forum.videohelp.com/threads/346097-Black-dark-banding

Would've been nice to know sooner.

[jagabo]

If you want all the grain and low contrast detail smoothed away like in the qs encodes run x264 at the veryfast preset.

[ziggy1971]

Not what I was trying to show with the clips, but oh well, what else can I say...

Forest, trees, pickles, ice-cream, tadpoles... all the same color...

[jagabo]

Not what I was trying to show with the clips

It's exactly what you are showing with the clips.

[poisondeathray]

thanks for sharing the tests ziggy

Just to clarify were the QS encodes using the skylake HTPC ? What quality level and other settings ? You must have set higher quality level because it seems slower than it should be ? Or maybe you have an input/decoding bottleneck with the HTPC - what is the process used for input/decode/encode/mux ?

In my testing on Haswell QS, the slowdown wasn't worth it. Negligible quality gains past balanced for much slower encodes.

Is it this one? HD 530 ?
http://www.notebookcheck.net/Intel-HD-Graphics-530.148358.0.html

The Intel HD Graphics 530 (GT2) is an integrated graphics unit, which can be found in various desktop and notebook processors of the Skylake generation. The "GT2" version of the Skylake GPU offers 24 Execution Units (EUs) clocked at up to 1150 MHz (depending on the CPU model). Due to its lack of dedicated graphics memory or eDRAM cache, the HD 530 has to access the main memory (2x 64bit DDR3L-1600 / DDR4-2133).

[ziggy1971]

I had an Intel Core i3-4350 HD 4600 (Haswell) before and, as far as I know, has all the same features as the newer Skylake (Don't quote me on that, I'm not that technical to make that kind of judgement)

Yes, I used my HTPC, an Intel Core i3-6300 HD 530 (Skylake) for the Quick Sync tests I uploaded.

I used StaxRip 1.3.7.0 (came with QSVEnc 2.51) for the encoding.
I downloaded and extracted QSVEnc 2.57, (I see 2.58 is now available) then copied all those files to StaxRip -> Apps -> QSVEncC folder (rename existing folder and create a new folder with the same name so you can easily revert back to the original with reinstalling)

I was just messing around with settings to see what the encoder can do, nothing special.
It seems that there is little information on the explanation of features of Quick Sync and even fewer comprehensive reviews, so I gotta make do with what I got.

The process used for these clips:
1. avs file with the following

FFVideoSource("path to file\S05-E06 - Unbowed, Unbent, Unbroken.mkv")

2. open avs in VirtualDub and save to avi using CineForm codec

3. avs file with the following

AVISource("path to file\S05-E06 - Unbowed, Unbent, Unbroken.avi", audio=false).AssumeFPS(24000,1001)

4. open avs file in StaxRip and encode using the attached template

Note: I know that I could've followed a different path for encoding (ie. open the 1st avs via FFVideoSource and encode directly from mkv to .264) I chose to use an intermediate codec because both the Quick Sync and x264 tests are based on the same intermediate file.

Had I used the FFVideoSource for encoding, using x264 and Quick Sync may use a different decoding path. (ie. Quick sync using a hardware path, x264 using a software path, depending on PC configuration)

Again, I am no expert nor do I claim to be. I'm just trying to do my best and learn along the way. At no point in time did I or have I claimed one encoder is better than the other. I merely made some observations in one format and, through testing, noticed similar results in the other.

Hope this all makes sense and helps.

P.S. Could you poisondeathray, or someone else, provide a text document with the Quick Sync features of previous versions? I forgot to check my HD 4600 before I sold it. I just want them for comparison.

To find the feature list (for those who don't know):
Open StaxRip -> select Intel H.264 or Intel H.265 encoder -> Encoder Options -> select the dropdown arrow in the lower right corner and select Check Features

[poisondeathray]

@ziggy, look at this post for check features for a HD4600
https://forum.videohelp.com/threads/365259-Question-to-Japanese-speaking-users-%28QSVEn...=1#post2401938

I think you are bottlenecked somewhere, because a haswell should not be getting faster than you are getting assuming you're using the same "balanced" quality setting . I'm getting ~140-145FPS, about 40sec for the encode . But every generation is supposed to be faster, and yield higher quality (and it has before) . I'm using nvidia to decode on a separate card, so there is no decoding bottleneck . But cineform shouldn't be a bottleneck either at 1080p, not with AVISource and the VFW decoder. Are you doing other operations like encoding audio ? But cineform will take away from CPU and primarily CPU encoding tests will be penalized

QSVEncC -i input.ext --la-depth 40 --la 5000 -o output.ext

I can read some bits of the .srip file in a text editor , but not enough to solve the mystery. and I don't have staxrip and I don't feel like installing it; is there any way you can tell, maybe from the log file what commands were passed to QSVEncC ? It might be as simple as you had it set on "higher" or something