x264 artifacts on dark scene

[kartul]

Hello all!

I have problem with x264. When I try to encode video with it, one dark scene get's artifacts that are very annoying and I have no idea on how do remove them. The avs script is plain simple - the source and resize (lancos sharp). So far I have tried disabling AQ, lowering AQ strength, lowering PSY-RD and PSY-trells strength and still nothing. I'm using MeGui for encoding. here is my current command line that MeGui shows me:

program --level 4.1 --preset slower --tune film --pass 2 --bitrate 2000 --stats ".stats" --min-keyint 23 --chroma-qp-offset -3 --vbv-bufsize 50000 --vbv-maxrate 50000 --cqmfile "C:\Program Files\MeGUI\extra\eqm_avc_hr.cfg" --output "output" "input"

And this is Mediainfo

cabac=1 / ref=8 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / fade_compensate=0.00 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=2 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-5 / threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / fgo=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=260 / keyint_min=20 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=2700 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=50000 / vbv_bufsize=50000 / ip_ratio=1.40 / aq=2:1.40 / nal_hrd=none

And finally, in attachments there are pictures of the artifacts that I can't get rid of.

[poisondeathray]

what happens if you don't use cqm?

are the artifacts visible before encoding in avs preview ?

for the other dark scenes, 2000 is quite low bitrate for 1280x720 - it might not be enough . What do the other artifacts look like?

[kartul]

what happens if you don't use cqm?

are the artifacts visible before encoding in avs preview ?

for the other dark scenes, 2000 is quite low bitrate for 1280x720 - it might not be enough . What do the other artifacts look like?

Ok, I removed CQM and new command line that MeGui shows me is

program --level 4.1 --preset slower --tune film --pass 2 --bitrate 2700 --stats ".stats" --min-keyint 23 --chroma-qp-offset -3 --vbv-bufsize 50000 --vbv-maxrate 50000 --aq-mode 2 --output "output" "input"

and Mediainfo shows this info

cabac=1 / ref=8 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=9 / psy=1 / fade_compensate=0.00 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-6 / threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / fgo=0 / bframes=3 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=2pass / mbtree=1 / bitrate=2700 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=50000 / vbv_bufsize=50000 / ip_ratio=1.40 / aq=2:1.00 / nal_hrd=none

x264 version is 1745-1 (Jeep) and no, these artifacts are not there before.

Still artifacts like seen on attachment.

I have seen artifacts mostly on one scene only - the title scene like seen on screens. everything else looks good.

[poisondeathray]

how are you taking screenshots? what player, what decoder , what software?

are the artifacts present when using mpchc or kmplayer ?

[kartul]

I use KMPlayer and I pause the video and use snagit to take the picture. The picture I upload here shows the SAME thing as I see in video.
I use VMR9 Renderless. But when I switched between Haali and other I can still see the artifacts. What decoder You recommand?

[poisondeathray]

It's not a renderer issue

Some older decoders had problems decoding weightp (it's enabled in your settings) , newer ffdshow build, or internal mpchc should have no problems

If you check with mpchc internal decoder, and problem still is there, then it's something else (or encode with --weightp 0)

It might be jeeb's build, try generic build x264.exe from x264.nl (just drop in the replacement, but backup your old x264.exe if you want to keep it)
http://www.x264.nl/

If it's still a problem, you will need to upload the smallest possible sample that reproduces the bug with your settings

[kartul]

I encoded it twice now with the same settings (I disabled weightp this time) and with new x264 exe - artifacts are still there. and they are in different places.
anything else before i upload the sample? should I upload it here?

[poisondeathray]

and they are in different places.

Did you try another player? VLC, MPCHC ?

Did you try the generic build from x264.exe to rule out if it's jeebs's build problem ?

This is a blu-ray source? you can upload it to free hosting site e.g. mediafire.com , sendspace.com etc.... Try to make it small as possible as long as you can reproduce the issue with the sample

Post your avs script as well

[kartul]

and they are in different places.

Did you try another player? VLC, MPCHC ?

Did you try the generic build from x264.exe to rule out if it's jeebs's build problem ?

This is a blu-ray source? you can upload it to free hosting site e.g. mediafire.com , sendspace.com etc.... Try to make it small as possible as long as you can reproduce the issue with the sample

Post your avs script as well

Yes I used new genetic build
Using MPCHC I can't see the artifacts anymore altho the video is all hmm... line'ated - almost like noise but instead of pixels there are lines. So this IS a KMPlayer's fault.
1st - MPCHC 2nd - KMPlayer

[poisondeathray]

Using MPCHC I can't see the artifacts anymore altho the video is all hmm... line'ated - almost like noise but instead of pixels there are lines.

I'm not sure what you mean ? "line'ated" ? Do you see lines ?

You have to describe it better, or post a screenshot or sample

[kartul]

Using MPCHC I can't see the artifacts anymore altho the video is all hmm... line'ated - almost like noise but instead of pixels there are lines.

I'm not sure what you mean ? "line'ated" ? Do you see lines ?

You have to describe it better, or post a screenshot or sample

Never mind that.

But how come KMPlayer shows the artifacts and MPCHC doesn't?
And what settings should I alter to make the blacks nice and remove some squares that come from encoding?

[poisondeathray]

But how come KMPlayer shows the artifacts and MPCHC doesn't?

They are using different decoders

The main difference in the black level is the renderer . You said you were using vmr9 renderless for kmplayer - that will give you the washed out look

You are probably using overlay mixer for MPCHC, check to settings to be sure

For KMPlayer, you can change the decoder to something else in the settings

[deadrats]

you're encoding a 720p movie, using only 2 mb/s and you're wondering why you're seeing artifacts?!? really?!?

[kartul]

in KMPlayer i changed the video render to Advanced Video Render C/A cause that seems to be same as in MPCHC (EVR Custom Pres.)

And deadrats, I have encoded many movies and never had this issue before. always almost the same setting.

[poisondeathray]

So everything is ok now ? It was a playback configuration issue?

[kartul]

Yea, looks like so.
Now, what setting should I alter to make the gray pixels disappear? - the gray pixels can be seen on attachment.

[jagabo]

You need to get your display calibrated. The background is a mix of zeros and ones. On a properly calibrated monitor you shouldn't really be able to tell the difference.

[poisondeathray]

leave chroma-qp-offset default, and aqmode default , you can disable mbtree as well (mbtree tends to make fades worse) . If you want to keep mbtree, try increasing qcomp

This is a fade in or out, so you might benefit from using a build with the fade compensation patch

[kartul]

Wow, that all is very helpful.
Thank You poisondeathray for helping!!