x264 causing color changes

[hihflappy]

Hi, I'm not sure why but x264 keeps messing with colours or the brightness.

This is the source:

[Attachment 59751 - Click to enlarge]

The encode:

[Attachment 59752 - Click to enlarge]

My settings used vspipe --y4m "C:\Users\T4\Documents\SciptsVS\test.vpy" - | E:\[VapourSynth]\x264-r3065-ae03d92.exe --preset veryslow --crf 15 --keyint 240 --min-keyint 23 --psy-rd 0.75:0.0 --aq-mode 3 --aq-strength 0.90 --qcomp 0.72 --b-adapt 2 --ref 6 --bframes 12 --deblock -1:-1 --no-dct-decimate --rc-lookahead 60 --ipratio 1.3 --me umh --merange 32 --trellis 2 --demuxer y4m - --output "E:\output.mp4"

My vapoursynth script is very simple, and the colours in vapoursynth preview were unaltered so I assume it's x264's doing:

import vapoursynth as vs
import lvsfunc as lvf
import fvsfunc as fvf
import havsfunc as haf
import functools
core = vs.core

src = lvf.src(r'E:\VideoWork\med.mkv')
src = src[20318:20947]

scaled = core.resize.Spline36(src, 854, 480)

out = scaled
final = fvf.Depth(out, 8)
final.set_output()

Why is x264 causing these colour shifts? It's driving me mad. I tried searching up a solution and people said use --colourmatrix bt709 but that didn't do anything.

[Skiller]

How were those two screenshots made?

It does look like a colorimetry (Rec.601 vs. Rec.709) mismatch to me.

[hihflappy]

I took them both in mpv, it's appears like that in other players and the default windows one so I don't think that's the issue

[poisondeathray]

Why is x264 causing these colour shifts? It's driving me mad. I tried searching up a solution and people said use --colourmatrix bt709 but that didn't do anything.

It does looks like a 601/709 mismatch. Likely your player or screenshot method is causing the shift, because there is no colormatrix compensation in the script

mpv uses colorimetry flags to override playback. Did you use --colourmatrix with the typo (there is no "u") or the proper --colormatrix ?

Was the source 1920x1080 ? Were you downscaling it ? If so, I would change the actual color in the script and flag it --colormatrix smpte170m during encoding. That way it will play correctly in almost all situations

scaled = core.resize.Spline36(src, 854, 480, matrix_in_s="709", matrix_s="170m")

[pm-s]

Probably the wrong colour space.

[hihflappy]

Why is x264 causing these colour shifts? It's driving me mad. I tried searching up a solution and people said use --colourmatrix bt709 but that didn't do anything.

It does looks like a 601/709 mismatch. Likely your player or screenshot method is causing the shift, because there is no colormatrix compensation in the script

mpv uses colorimetry flags to override playback. Did you use --colourmatrix with the typo (there is no "u") or the proper --colormatrix ?

Was the source 1920x1080 ? Were you downscaling it ? If so, I would change the actual color in the script and flag it --colormatrix smpte170m during encoding. That way it will play correctly in almost all situations

scaled = core.resize.Spline36(src, 854, 480, matrix_in_s="709", matrix_s="170m")

Wow, this fixed it. You're right I was trying to downscale from 1080p. I showed that other encode in 1080p because that was me trying it without resizing to see if that would fix it which it didn't. The colour space for both was still YUV so I don't think it was that. Defining the matrix in the spline36 command fixed it though, thank you