View Profile
View Forum Posts
Private Message
I often record “catch-up” broadcasts from french TV channel Arte, and recently, when comparing videos of programs which have already been broadcast a year or two ago, and which I already recorded at the time (either because I forgot that I already had them, or to check if the encoding quality had improved in the mean time), I noticed a dictinct color discrepancy. I know that this is generally due to the use of a correct or incorrect YUV-RGB color conversion matrix. It would seem like they now do the encoding with the wrong matrix. But before I try to report the issue (which is going to be tricky, both because there doesn't seem to be an easily accessible “contact” area anymore, and because those videos are not supposed to be downloaded...), could someone confirm that the colors appear to be wrong on the most recent videos, and not the other way around ? There seems to be a general blue/purple tint on the most recent videos, and the skin tones seem off, am I right ? Is there a likely explanation to this, considering that the encoding parameters haven't changed as far as I can see ? According to MediaInfo : same AVC profile, approximately same bitrate... But strangely the newer videos report a “BT.709” color conversion matrix, which should be correct, while the older ones didn't display that field – although it seems to vary, some recent videos don't have that field either. Is this relevant at all ?
“Zétwal”, documentary broadcast on 2019-07-16 (screenshot 1) and 2020-07-21 (screenshot 2)
“Zapped”, documentary about Frank Zappa broadcast on 2017-03-17 (screenshot 1) and 2020-07-25 (screenshot 2)
(This one would have been an improvement if not for that issue, as the vertical logo is less disturbing over 4:3 archive footage which form the bulk of this documentary – which is excellent by the way.)
“Toni Erdmann”, movie broadcast on 2018-11-12 (screenshot 1) and 2020-07-29 (screenshot 2)
MediaInfo report for “Zapped” 2017-03-17 :
MediaInfo report for “Zapped” 2020-07-25 :Code:Général Nom complet : F:\A CLASSER 2\Arte\2017\201703172223 - Arte - Zapped - Frank Zappa par Frank Zappa {vu 20170325 +++}.mp4 Format : MPEG-4 Profil du format : Base Media / Version 2 Identifiant du codec : mp42 (isom/mp42) Taille du fichier : 1,02 Gio Durée : 1 h 0 min Type de débit global : Variable Débit global moyen : 2 420 kb/s Date d'encodage : UTC 2017-03-15 22:40:56 Date de marquage : UTC 2017-03-15 22:40:56 Vidéo ID : 1 Format : AVC Format/Info : Advanced Video Codec Profil du format : Main@L3.1 Paramètres du format : CABAC / 3 Ref Frames Paramètres du format, CABAC : Oui Paramètres du format, RefFrames : 3 images Identifiant du codec : avc1 Identifiant du codec/Info : Advanced Video Coding Durée : 1 h 0 min Débit : 2 290 kb/s Largeur : 1 280 pixels Hauteur : 720 pixels Format à l'écran : 16/9 Type d'images/s : Constant Images par seconde : 25,000 Im/s Norme : PAL Espace de couleurs : YUV Sous-échantillonnage de la chrominance : 4:2:0 Profondeur des couleurs : 8 bits Type de balayage : Progressif Bits/(Pixel*Image) : 0.099 Taille du flux : 991 Mio (95%) Date d'encodage : UTC 2017-03-15 22:40:57 Date de marquage : UTC 2017-03-15 22:40:57 Gamme de couleurs : Limited Codec configuration box : avcC Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Identifiant du codec : mp4a-40-2 Durée : 1 h 0 min Type de débit : Variable Débit : 125 kb/s Débit maximum : 168 kb/s Canaux : 2 canaux ChannelLayout : L R Echantillonnage : 48,0 kHz Images par seconde : 46,875 Im/s (1024 SPF) Mode de compression : Avec perte Taille du flux : 54,3 Mio (5%) Date d'encodage : UTC 2017-03-15 22:40:57 Date de marquage : UTC 2017-03-15 22:40:57
Code:Général Nom complet : M:\2020_07_2500_46 - Arte - Zapped - Frank Zappa par Frank Zappa.mp4 Format : MPEG-4 Profil du format : Base Media / Version 2 Identifiant du codec : mp42 (isom/mp42) Taille du fichier : 1 000 Mio Durée : 1 h 0 min Type de débit global : Variable Débit global moyen : 2 311 kb/s Date d'encodage : UTC 2020-07-21 11:38:13 Date de marquage : UTC 2020-07-21 11:38:13 Vidéo ID : 1 Format : AVC Format/Info : Advanced Video Codec Profil du format : Main@L3.1 Paramètres du format : CABAC / 3 Ref Frames Paramètres du format, CABAC : Oui Paramètres du format, RefFrames : 3 images Identifiant du codec : avc1 Identifiant du codec/Info : Advanced Video Coding Durée : 1 h 0 min Type de débit : Variable Débit : 2 200 kb/s Largeur : 1 280 pixels Hauteur : 720 pixels Format à l'écran : 16/9 Type d'images/s : Constant Images par seconde : 25,000 Im/s Norme : PAL Espace de couleurs : YUV Sous-échantillonnage de la chrominance : 4:2:0 Profondeur des couleurs : 8 bits Type de balayage : Progressif Bits/(Pixel*Image) : 0.095 Taille du flux : 944 Mio (94%) Date d'encodage : UTC 2020-07-21 11:38:15 Date de marquage : UTC 2020-07-21 11:38:15 Gamme de couleurs : Limited Coordonnées de chromaticité : BT.709 Caractéristiques du transfert : BT.709 Coefficients de la matrice : BT.709 Codec configuration box : avcC Audio ID : 2 Format : AAC LC Format/Info : Advanced Audio Codec Low Complexity Identifiant du codec : mp4a-40-2 Durée : 1 h 0 min Type de débit : Variable Débit : 125 kb/s Débit maximum : 170 kb/s Canaux : 2 canaux ChannelLayout : L R Echantillonnage : 48,0 kHz Images par seconde : 46,875 Im/s (1024 SPF) Mode de compression : Avec perte Taille du flux : 54,3 Mio (5%) Date d'encodage : UTC 2020-07-21 11:38:15 Date de marquage : UTC 2020-07-21 11:38:15
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 1 to 16 of 16
Thread
-
Last edited by abolibibelot; 4th Aug 2020 at 23:29.
-
I've seen this affect on my LG TV; the TV comes with xumo.tv built-in, a thing they call "channel plus".
It's an adaptive stream over the internet. Quite often, when a channel is first selected, the picture
is fuzzy (low resolution) and the color leans towards the magenta side.
After a few seconds, the higher res picture comes in, and the color also corrects.
I had always assumed the TV was using rec609 for a few seconds when the feed is low-res SD.
then switches to rec709 when in locks in at 720p or better.
Do you see this issue on the TV and PC? Perhaps the files are incorrectly encoded. -
All screenshots have been made with VLC Media Player on the same system. The discrepancy seems constant throughout each set of videos of the same program. TV / PC seems irrelevant here (anyway I don't have a TV, watch everything on the computer screen). The videos were downloaded as standalone MP4 files from basic HTTP URLs (with a dedicated software called Captvty), not some sort of adaptive streams.
Two of those are still available (the 2020 versions of course) :
https://www.arte.tv/fr/videos/086911-000-A/zetwal/ origin HTML page which leads to JSON page with video URLs :
https://api.arte.tv/api/player/v1/config/fr/086911-000-A which leads to direct link in 1280x720 :
https://arteptweb-a.akamaihd.net/am/ptweb/086000/086900/086911-000-A_SQ_0_VOF-STF_0510...1MysEm7aG8.mp4
https://www.arte.tv/fr/videos/048387-000-A/zapped-frank-zappa-par-frank-zappa/
https://api.arte.tv/api/player/v1/config/fr/048387-000-A
https://arteptweb-a.akamaihd.net/am/ptweb/048000/048300/048387-000-A_SQ_0_VO-STF_05128...1NGWUmreOk.mp4
That's what I asked : based on those screenshots, which of them are most likely incorrectly encoded ? The ones from 2017-2019 or the ones from 2020 ?Perhaps the files are incorrectly encoded. -
Well what? I already inferred that the earlier images look more accurate (to me)Originally Posted by abolibibelot
-
Well, no (or it was very subtly inferred), but now you have.Well what? I already inferred that the earlier images look more accurate (to me)
I expected some sort of technical insight ; for instance the second set of screenshots shows a color chart, which must be used for the very purpose of checking if the colors are about accurate, so, shouldn't an experienced video technician, seeing the color chart on the second screenshot, immediately realize that something is wrong, with that glowing green strip in the middle ? -
Perhaps somebody else will provide some more specific info. I downloaded the Zappa file,Originally Posted by abolibibelot
the video levels are within the normal (limited) range. -
One problem is VLC. Always double check with other tools than VLC, which is known to have various issues
This is what it should look like for "048387-000-A_SQ_0_VO-STF_05128682_MP4-2200_AMM-PTWEB_1NGWUmreOk.mp4" for 709
You can check with another player, or explictly control the RGB conversion
[Attachment 54389 - Click to enlarge]
But it's still "off". Black is correct, but white is neither 75% or 100% . Usually PAL EBU bars have white at 100%. Maybe they are supposed to be something else, but everything except black is off.
I can't comment on the older videos, because you took those screenshots with VLC too... -
Yes I remember being told, most likely by you, about VLCMP's unreliability with regards to screenshots, and color rendering in general, but in this case, since the screenshots were made from individual downloaded files which are otherwise similar, the discrepancy has to come from somewhere within the files themselves.
Anyway, screenshots taken with AVSPMod (supposedly reliable) show the same pattern :
screenshot from 20170317 file
screenshot from 20200725 file
20170317 + 20200725 StackVertical
Note : This is not the exact same frame as in the first post. In fact in the first post the two screenshots don't show the exact same frame, as I took them right at the beginning of the first subtitle, but I noticed that the first subtitle appears 2 frames earlier on the 2020 video. Here both are frame 100 and the actual footage begins at frame 122.
Another set (frame 16315) :
screenshot from 20170317 file
screenshot from 20200725 fileLast edited by abolibibelot; 5th Aug 2020 at 18:47.
-
You can examine it using color picker. e.g. avspmod has one
75% bars , when using full range RGB (computer RGB 0-255) should be 191 in the appropriate channel . For example "75% red" would be RGB 191,0,0 . "75% yellow" would be RGB 191,191,0 . It's normal to have slightly off +/-3 , rounding errors. Yes there is a discrepancy between each other, and both are "off" significantly if you assume those are supposed to be 75% color bars. You have to assume "white" is 75% white (it should be 191,191,191) . But EBU bars typically use 100% white -
Where ? Is it the “hex” value at the bottom ?You can examine it using color picker. e.g. avspmod has one
Well, I don't know much about those broadcasting standards, but this is old archive footage edited into a documentary meant to be broadcast on current television devices, perhaps it was deemed too bright and was darkened somewhat to keep the global levels about consistent throughout the movie. What matters is the relative levels of colors and how accurately the video preserves the tones or the original material (in this case that means the “master” video file of that documentary used for the broadcast, which I would suppose is the same that was used in 2017). It seems to me that the colors are more “balanced”, for lack of a better word, on the 2017 version, and the skin tones seem more natural. Is this verified by the measurements ?75% bars , when using full range RGB (computer RGB 0-255) should be 191 in the appropriate channel . For example "75% red" would be RGB 191,0,0 . "75% yellow" would be RGB 191,191,0 . It's normal to have slightly off +/-3 , rounding errors. Yes there is a discrepancy between each other, and both are "off" significantly if you assume those are supposed to be 75% color bars. You have to assume "white" is 75% white (it should be 191,191,191) . But EBU bars typically use 100% white
Also, what about the newer standards, Rec. 2020 / Rec. 2100 ? How are the colors affected when footage encoded with Rec. 709 is mistakenly rendered with Rec. 2020 for instance ? (Those settings are not available in AVSPMod as far as I can see. Are they available in current versions of Avisynth, natively or through a plugin ?) Would it be possible that the broadcaster now uses those standards geared toward “4K” broadcast ? (Assuming that this makes sense, of which I'm not so sure right now.) -
Originally Posted by abolibibelot
If you don't see it - you might not have it enabled for RGB
options=> program settings => video => customize video status bar => add %RGB after %HEX . You can add %YUV too. In the status bar, when there is an * asterisk, it means converted value. So if input was RGB, RGB would not have an asterisk, but YUV and HEX would have an asterisk
I don't have the old version to look at. "balanced" can be subjective ; and the lighting is not neutral in all shots; and the color and grading is inconsistent across different shots (rightly so, it's composed from different sources, cameras, times)Well, I don't know much about those broadcasting standards, but this is old archive footage edited into a documentary meant to be broadcast on current television devices, perhaps it was deemed too bright and was darkened somewhat to keep the global levels about consistent throughout the movie. What matters is the relative levels of colors and how accurately the video preserves the tones or the original material (in this case that means the “master” video file of that documentary used for the broadcast, which I would suppose is the same that was used in 2017). It seems to me that the colors are more “balanced”, for lack of a better word, on the 2017 version, and the skin tones seem more natural. Is this verified by the measurements ?
You can convert using 2020 if you wanted to check. Or other matrices, or any combination. Those are available to you in avisynth / vapoursynth/ ffmpeg etc...Also, what about the newer standards, Rec. 2020 / Rec. 2100 ? How are the colors affected when footage encoded with Rec. 709 is mistakenly rendered with Rec. 2020 for instance ? (Those settings are not available in AVSPMod as far as I can see. Are they available in current versions of Avisynth, natively or through a plugin ?) Would it be possible that the broadcaster now uses those standards geared toward “4K” broadcast ? (Assuming that this makes sense, of which I'm not so sure right now.)
e.g.
ConvertToRGB24(matrix="Rec2020")
Note many broadcasters still use 709 for UHD. And the assumption that the attached colorbars are valid or representative of the program is not necessarily true. Maybe they were attached at some 3rd or 4th generation stage, but different than the original program. There are dozens of places in the workflow where something could have gone wrong . Some guy pushes a button and you get a different color 3 years later.Last edited by poisondeathray; 5th Aug 2020 at 21:23.
-
Alright, works. And so what is “hex” ?options=> program settings => video => customize video status bar => add %RGB after %HEX . You can add %YUV too. In the status bar, when there is an * asterisk, it means converted value. So if input was RGB, RGB would not have an asterisk, but YUV and HEX would have an asterisk
I uploaded a 10s segment cut with ffmpeg -c copy -t 10.I don't have the old version to look at.
I mean, on those screenshots I provided. The green seems too bright and the red too dark, which from my admittedly limited experience indicates that a conversion was made with the wrong matrix."balanced" can be subjective ; and the lighting is not neutral in all shots; and the color and grading is inconsistent across different shots (rightly so, it's composed from different sources, cameras, times)
Works indeed – I didn't try it before since it wasn't among the options listed by AVSPM, so I assumed that it wasn't available the currently installed version of Avisynth.ConvertToRGB24(matrix="Rec2020")
If I convert the 2017 version with "Rec2020" it doesn't produce a similar aspect to the 2020 version (the green is slightly lighter but not as much).
But if I convert the 2017 version with "Rec601" the aspect is very close to the 2020 version converted with "Rec709" (frame 100 again).
Again, what I noticed is a general pattern, seemingly affecting all videos from that provider recently. And the main question was : was it correct before and is it incorrect now, or was it incorrect before and is it correct now ? The first hypothesis seems more likely, but I'd like to be sure before I report it.Note many broadcasters still use 709 for UHD. And the assumption that the attached colorbars are valid or representative of the program is not necessarily true. Maybe they were attached at some 3rd or 4th generation stage, but different than the original program. There are dozens of places in the workflow where something could have gone wrong . Some guy pushes a button and you get a different color 3 years later. -
Hexidecimal value describing a "color" . A "color" can be expressed as HSL tooOriginally Posted by abolibibelot
e.g
https://htmlcolorcodes.com/
Yes, but both have issues.I mean, on those screenshots I provided. The green seems too bright and the red too dark, which from my admittedly limited experience indicates that a conversion was made with the wrong matrix.
This is not as simple as a single matrix conversion error. Maybe primaries were adjusted (e.g. PAL has different slightly primaries than NTSC, but primaries are usually ignored), maybe matrix conversion a few times (other than 709/601) . There are other parameters that can be adjusted (other than matrix), and there are other matrices than 601/709/2020
Likely that's what happened.But if I convert the 2017 version with "Rec601" the aspect is very close to the 2020 version converted with "Rec709" (frame 100 again)
If you assume the bars are correct, then neither is "correct" . But I'd say it's probably less correct nowAnd the main question was : was it correct before and is it incorrect now, or was it incorrect before and is it correct now ? The first hypothesis seems more likely, but I'd like to be sure before I report it. -
This is not as simple as a single matrix conversion error.This seems a bit contradictory...Likely that's what happened.
What are primaries, in a nutshell ?Maybe primaries were adjusted (e.g. PAL has different slightly primaries than NTSC, but primaries are usually ignored)
What would be the other commonly used matrices for standard YUV broadcasting ?There are other parameters that can be adjusted (other than matrix), and there are other matrices than 601/709/2020
Of course many things can be adjusted in video, but this shift to lighter greens and darker reds is quite characteristic.
Would it be possible to tweak the levels in such a way that the bars appear perfectly standard, to see how it would affect the rest of the footage ?
And is it possible to display a video converted with a wrong color conversion matrix so as to obtain the same aspect it would have if it had been converted with the correct matrix, or has some part of the color information been lost in the process ? -
Originally Posted by abolibibelot
2017 version was wrong to begin with (if you trust the bars). 2020 version is 2017 version + ~Rec601 shift on top (so it's "more" wrong) . That's what your observation suggested and I agree :
But if I convert the 2017 version with "Rec601" the aspect is very close to the 2020 version converted with "Rec709" (frame 100 again)
What are primaries, in a nutshell ?
They describe the x,y coordinates of the "primary" colors (red, blue, green) . 709, "NTSC" and "PAL" have different values. (But in practice, the primaries are usually not adjusted for when conversion between different formats, but maybe that's contributing to why 2017 is "wrong" too)
Those are it.What would be the other commonly used matrices for standard YUV broadcasting ?
But it's people (some doofus) that make mistakes. This is common. Some guy runs the video though some software, but it uses different parameters. E.g. Many types of mac software use slightly different gamma /transfer characteristics.
Yes, for the 2017 version to 2020 version differenceOf course many things can be adjusted in video, but this shift to lighter greens and darker reds is quite characteristic.
But I'm looking at the colorbars - they should be "known" reference colors. That's what they are there for. If it was a simiple 709/601 mistmatch then specifying the correct matrix or using colormatrix and reversing the transform should get you the correct results. ie. You should be able to fix 2017 to whatever earlier original "correct" result was
That's what the bars are for - if you assume they were correct and there since the beginning. Yes, it should possible , with color mixer or masksWould it be possible to tweak the levels in such a way that the bars appear perfectly standard, to see how it would affect the rest of the footage ?
And is it possible to display a video converted with a wrong color conversion matrix so as to obtain the same aspect it would have if it had been converted with the correct matrix, or has some part of the color information been lost in the process ?
If it was a simple 601/709 shift, you should be able to easily "fix" 2017, but that's not the case . That suggests a bunch of other things were done to get that 2017 result. You can easily "fix" 2020 to get 2017 or "reverse" what had been done (e.g. colormatrix(mode="rec.709->rec.601",clamp=0) ) . But 2017 is still "wrong" according to the bars, and there is no easy 1 line fix to get the original barsLast edited by poisondeathray; 6th Aug 2020 at 16:42.
QuoteSimilar Threads
-
ffmpeg 4.1.4 question regarding "limited color range" output file
By bokeron2020 in forum Newbie / General discussionsReplies: 12Last Post: 1st Aug 2019, 17:28 -
Windows 10 Updates "Broke" DVD Shrink And Other Programs on PC
By devilcoelhodog in forum ComputerReplies: 21Last Post: 1st Aug 2018, 13:29 -
Video color to "black & chrome" conversion
By leghorn in forum EditingReplies: 6Last Post: 4th Feb 2018, 08:35 -
How the "see" correct BT.709 color using VLC in a PC?
By marcorocchini in forum Newbie / General discussionsReplies: 2Last Post: 25th Jun 2017, 10:57 -
[Sony Vegas] how to automate the "color gradient" plugin's Control Points?
By rocco123 in forum Newbie / General discussionsReplies: 1Last Post: 31st Jan 2017, 23:34