photoshop elements levels VS ffmpeg colorlevels question

[maypo]

I am attempting to use ffmpeg to color correct a video. I was under the impression that the levels control in photoshop elements corresponds directly to the colorlevels filter in ffmpeg when setting input white point, but evidently not.

The workflow was this:
extract a frame from the video as .BMP
import the .BMP into photoshop elements
use the levels adjustment to get the correction I want,
apply these same levels in ffmpeg to the video using colorlevels

Now, in PSE, i found that the following adjustment - only the input levels, output unchanged - gave me an adjustment i was happy with:
ri 0-232
gi 0-192
bi 0-158
(yes, the video was wwaaaayyy red heavy)

So, i thought simply scaling the PSE values (0-255) -> (0.0 - 1.0) would give me the same levels correction in ffmpeg using the white point like this:

ffmpeg -i infile.mts -pix_fmt yuv420p -c:v libx264 -preset fast -crf 22 -vf "colorlevels=rimax=0.909:gimax=0.753:bimax=0.6 19" -c:a copy outfile.mp4

However the resulting video, while much improved, still differs significantly from the corrected image in PSE.
Skin tone in the video appears to be more pasty white, with more prominent red highlighting, as though the person had never been outside and had rosacea

Any idea how setting the white point in PSE differs from setting white point colorlevel in ffmpeg?
What adjustment in ffmpeg would correspond most closely with setting levels in PSE?

TIA, maypo

[jagabo]

Some sample images would help. A rec.601 vs. rec.709 difference will show up mostly in reds and greens.

https://forum.videohelp.com/threads/329866-incorrect-collor-display-in-video-playback#post2045830

The differences may be amplified in your case since you are scaling the colors.

[maypo]

right, should have been more detailed. here is ffprobe output for the original source file:
Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(top first), 1440x1080 [SAR 4:3 DAR 16:9], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc

original file:


[Attachment 43186 - Click to enlarge]

PSE adjusted file:


[Attachment 43187 - Click to enlarge]

FFMPEG adjusted file:


[Attachment 43188 - Click to enlarge]

[poisondeathray]

It's very likely a YUV<=>RGB matrix issue . Because if I take your "raw" image , decoded as RGB in avisynth (jpeg can be decoded as YUV, because it's stored as YUV), feed it to ffmpeg and apply -vf "colorlevels=rimax=0.909:gimax=0.753:bimax=0.6 19" , I get something close to your PSE image . So that suggests at least the ffmpeg colorlevels works as intended in RGB

The problem is you have multiple conversions going on and the details on how you do them matter. For example how did you convert the YUV video into an image for PSE ? Also, ffmpeg is doing a few conversions because you're applying a RGB filter on a YUV filter, then converting back to a YUV video

[jagabo]

Your ffmpeg filtering sequence assumes rec.601. That is, it converts incoming YUV to RGB with a rec.601 matrix, performs the RGB adjustments, then converts the video back to YUV with a rec.601 matrix.

We don't know how your prepared your "source" image so we don't know if it's colors accurately represent what's in your source video. Same with the adjusted video.

[maypo]

Ok, I'm following. Is there a way to accomplish the same effect in YUV space without the conversion to RGB and back?

[poisondeathray]

Ok, I'm following. Is there a way to accomplish the same effect in YUV space without the conversion to RGB and back?

Probably, but most people find it more difficult to make adjustments in YUV . RGB is usually easier for most people and more intuitive way of manipulating color

Either that, or be more specific and control how you are doing each YUV<=>RGB conversion. eg. Conceivably you could specify the matrix in ffmpeg or other tools, and in the ffmpeg filter chain to control each conversion at each stage. There is no question that staying in YUV is more ideal if you can get the results you want

When you post a screenshot, it's already been converted to RGB (so it's no longer the "original" YUV file). How you converted to RGB matters, because it's just one possible RGB representation of the original. It might have used rec601 vs 709 or something else

If you post a native video sample, I bet jagabo can come close using YUV filters only

[poisondeathray]

Or are you "set" on using ffmpeg ? Because another option might be virtualdub / virtualdubfiltermod , where you can use RGB levels in colormill, or even RGB curves (like photoshop curves).

[maypo]

Thanks for the replies; based on this i took a look at YUV on wikipedia. The concept seem quite understandable in theory, but as you suggested, i have almost no intuitive feel for making adjustments.
I have access to premiere elements 15, and started playing around with the 3 way color corrector. Interestingly, i got the exact same color shift issue - adjustments that looked good in PSE15 had changed when output to a video file, and the change was very much like what i saw in ffmepg. I'm wondering if the h.264 codec (i used lib264 in ffmpeg and h.264 output in PSE15) is part of the issue?

[jagabo]

First you need to figure out whether your source is rec.601 or rec.709. MediaInfo might tell you this. If not, you can guess rec.709 if your source is HD, rec.601 if your source is SD. Then you need to know whether the way you view videos and make your images uses the correct matrix. Some software will always use one matrix or the other. Some will follow the matrix flagged in the video. If the matrix isn't flagged the software may assume rec.709 for HD, rec.601 for SD. In between SD (720x576) and HD (1280x720) is tricky, you just have to use your judgement.

[maypo]

below is what MediaInfo came up with, don't see anything about 601 or 709. I checked around forums, and it seems people don't know whether the adobe 3way color corrector does a conversion betweeen YUV and RGB or not. Evidently some FX plugins are exclusively YUV and others convert.
Not sure where to go at this point other than give it my best shot through experimentation. Thanks jagabo and poisondeathray for the input.


[Attachment 43211 - Click to enlarge]

[poisondeathray]

Since forcing RGB from your jpg worked with your -vf colorlevels arguments worked (or at least was "close" to your target), there should be a combination or workaround in ffmpeg only that you can get this working, for example you can use -vf colormatrix to convert between 601 and 709 in YUV (or you can use -vf scale and specify the in and out matrix). You migth have to convert several times, maybe pre filter and post filter . (Because if colorlevels converts to RGB using 601, you'd want to use -vf colormatrix=bt709:bt601 before to convert in YUV "as if" it were a 601 video. Then after the colorlevels filter, you need to convert it back to 709, because it's HD

But we don' t know how you converted for the jpg, or how you are viewing the output file (each step involves a YUV<=>RGB conversion) , so those are other variables

If you cut (stream copy) a piece of the sample video , I'm sure we can figure out a workaround in ffmpeg