Vapoursynth colour issue

[ashateli]

This vapoursynth script is used to compare the detelecined source video from DVD with two X264 encoded versions (via ffmpeg) having different processing. I detelecine the source clip the same as the encoded clips were done prior to being encoded so that the frame rates match.

When I view the interleaved frames I'm seeing a colour shift in the two encoded clips that use ffms2.Source compared to the original. It's a mild colour shift, things wash out slightly yellow - it might be the problem with full / limited somehow. When creating the d2v for the original source mpeg2, I have the limited option checked in d2vwitch.

If I load the three clips in three instances of mpv and stack the mpv windows over the top of each other, I see no colour shift when I switch windows. Based on this I think it's a problem with the way the encoded clips are imported using ffms2 into the vapoursynth script. It could be a difference within the encoded clips, I guess, but if it is mpv is not rendering the issue.

Any advice or thoughts are much appreciated. Thanks in advance!

import vapoursynth as vs
from vapoursynth import core
import havsfunc as haf
import functools

def postprocess(n, f, clip, deinterlaced):
   if f.props['_Combed'] > 0:
      return deinterlaced
   else:
      return clip

input_clip = core.d2v.Source(r'original-clip1.d2v')
matched_clip = core.vivtc.VFM(input_clip, 1)
deinterlaced_clip = haf.QTGMC(matched_clip, Preset='Very Slow', FPSDivisor=2, SourceMatch=3, Lossless=2, TFF=True)
postprocessed_clip = core.std.FrameEval(matched_clip, functools.partial(postprocess, clip=matched_clip, deinterlaced=deinterlaced_clip), prop_src=matched_clip)

src0 = core.vivtc.VDecimate(postprocessed_clip)
src1 = core.ffms2.Source(r'encoded-clip1.mkv')
src2 = core.ffms2.Source(r'encoded-clip1-degrain.mkv') 

video = core.std.Interleave(clips=[src0,src1,src2])
#video = core.std.StackHorizontal(clips=[src0,src1,src2])

video.set_output()

[poisondeathray]

How are you viewing the interleave script ?

ffms2 passes metadata as FrameProps, and that is used by vsedit when converting to RGB for display

you can check with core.text.FrameProps(clip) , or with mediainfo. How are the videos flagged in terms of matrix, transfer, and primaries?

If an encode is flagged with primaries and matrix, that can give different results than a media player. A media player typically uses only matrix for the conversion, and they typically bases this on dimensions, not metadata. They typically ignore metadata

Alternatively, you can control the RGB conversion explicitly in the script, or use core.std.SetFrameProp to override parameters

Are these all 8bit ? Another semi common difference is if you're using 10bit encodes, and the 10bit YUV =>8bit RGB conversion for display is done slightly differently

If you can't figure it out post samples from the clips

[ashateli]

How are you viewing the interleave script ?

ffms2 passes metadata as FrameProps, and that is used by vsedit when converting to RGB for display

you can check with core.text.FrameProps(clip) , or with mediainfo. How are the videos flagged in terms of matrix, transfer, and primaries?

If an encode is flagged with primaries and matrix, that can give different results than a media player. A media player typically uses only matrix for the conversion, and they typically bases this on dimensions, not metadata. They typically ignore metadata

Alternatively, you can control the RGB conversion explicitly in the script, or use core.std.SetFrameProp to override parameters

Are these all 8bit ? Another semi common difference is if you're using 10bit encodes, and the 10bit YUV =>8bit RGB conversion for display is done slightly differently

If you can't figure it out post samples from the clips

Thanks for the reply!

Viewing with vsedit. FrameProps as follows:

source clip from DVD
matrix: 5
transfer: not reported
primaries: not reported

encoded clips (both the same)
matrix: 2
transfer: 2
primaries: 2

(is there a reference page for the frame properties? I can't seem to find it.)

I guess this explains why there could be differences, but it's still unclear whether the encodes have issues or vsedit isn't rendering them as it should due to the properties.

These are all 8 bit. I'm not touching them other than detelecine and a degrain in one of the encoded clips. My encode cli looks like this:

vspipe -a "file=title_t00.d2v" --y4m ~/ivtc.vpy - | ffmpeg -i pipe: -i title_t00.mkv -vcodec libx264 -tune film -crf 16 -preset veryslow -vf "setdar=4/3" -acodec copy -map_metadata 1 -map 0:0 -map 1:1 testoutputveryslow16film.mkv

Could I be getting into problems with map_metadata? I'm using that to preserve the DVD chapters in the clip. Or does that only affect the container metadata as opposed to the streams?

[ashateli]

I did some reading and found that adding the following to my ffmpeg encode cli seems to correct the colour in VSedit without affecting the mpv playback:

-x264opts colormatrix=smpte170m:transfer=smpte170m:colorprim=smpte170m

Still not 100% certain this resolves the issue correctly. The re-encoded clips using the above options now report the following which still do not match the source clip:

matrix: 6
transfer: 6
primaries: 6

If media players such as mpv commonly ignore metadata information it makes it difficult to understand what is actually in the source!

[poisondeathray]

You can either A) take control of the RGB conversion in the viewing script for all 3 matrix, transfer, primaries; Or B) SetFrameProp to set the metadata so all clips are the same



Those values are taken from Table E.3, E.4, E.5 from the h.265 document

Matrix 5 is bt470bg PAL . The actual values for matrix are the same as 170m (5, 6 or PAL, NTSC use the same value), so you can actually use them interchangeably for SD NTSC/PAL matrix (no functional difference). But the values are different for primaries (NTSC and PAL use different primaries, but in actual practice - very few hardware or software take that into account for the RGB conversion - usually only matrix is used. Most media players ignore primaries - so this is the most common difference between vsedit and almost everything else). Transfer actual values are the same for SD (NTSC and PAL), and HD 709 .



eg. if you 've set the VUI parameters for the encode as 6,6,6 (ooohhh "666" ) that means matrix 170m (ntsc) , transfer 601, primaries 170m (ntsc)

You can override the src0 to the same . Or override the encode, whatever you feel like
src0 = core.std.SetFrameProp(src0, prop="_Matrix", intval=6)
src0 = core.std.SetFrameProp(src0, prop="_Transfer", intval=6)
src0 = core.std.SetFrameProp(src0, prop="_Primaries", intval=6)

If you're still getting a difference, then there is likely an actual difference somewhere. Maybe your filter did something else, or something in your process caused a differnce

Note in all cases, you still have YUV video - this is just affecting the RGB preview. The actual YUV values are unaffected

[_Al_]

matrix: 6
transfer: 6
primaries: 6

After you encoded it with that 170m flag, ffms2 would recognize it and pass it to vapoursynth as you can see.

Here is table for Matrix those values within Vapoursynth props:

MATRIX =      {
                #matrix_in or matrix : matrix_in_s or matrix_s
                0:'rgb',
                1:'709',
                2:'unspec',
                3:'reserved',
                4:'fcc',
                5:'470bg',
                6:'170m',
                7:'240m',
                8:'ycgco',
                9:'2020ncl',
               10:'2020cl' ,
               12:'chromancl',
               13:'chromacl',
               14:'ictcp'
              }

sp that previous value 2 means , it is not specified and defaults are guessed. If there is a problem still, you perhaps need to set it in vsedit settings. Not sure if vsedit uses that setting value only if it is not available from vapoursyth or all the time uses its settings value. In that picture there is 470bg, but you can set 170m. It should not matter which anyway. I think it uses Vapoursynth value so make sure they are correct ones.

[_Al_]

image from vsedit:

[ashateli]

thanks poisondeathray and _AI_ !!!

You two were also the helpful commenters in the thread where I found the resolution that I used. I have a lot to learn, I can see.

Kind regards

[_Al_]

vapoursynth values for all MATRIX, PRIMARIES and TRANSFER if you needed are here on github
Those values, if set in vapoursyth are only for vapoursynth internally. It does not get passed to videostream during encoding, so it is a good habit always set them during encoding, as you did in that x264 test.

One thing to realize with subject like this, Vapoursynth does not use defaults for matrix if converting YUV to RGB. This is very good actually and getting away from ffmpeg or avisynth, where defaults in background can silently pass by you. So if source plugin does not identify matrix or you do not manually set _MATRIX prop you have to select it as matrix argument during conversion - avoiding error message. It is not going default for you.