Windows+ffmpeg+x264 = brightness(?) problems

[orangeboy70]

Hi,
I'm trying to create a Windows batch file to do mass conversion for * to h264/ac3/mp4 that is compatible with my TiVo. So far, I'm happy with using ffmpeg and x264 for the conversions since they are CLI, but I am finding some videos produce a bit brighter picture. An example I found on the web that depicts the situation:



The solutions I have found for this involve GUI plugins which run counter to my end goals for an automated solution. I'll first describe a 10,000 foot view of my script, then post the code I'm using.

Set a target directory on a remote computer to store the resulting mp4 file.
Set a source directory on a remote computer for all input files.
Start loop of files found on "source".
Copy from "source" to a local temp directory to reduce network overhead during conversion.
Use MediaInfo.exe to gather some key information about the source file.
Set the audio bitrate to a valid AC3 value (for example, 159kbps upconverts to 160kbps).
Set some variables to reduce the actual commandline length for easier viewing in text editor.
Invoke ffmpeg to do the actual conversion, using x264 as the video encoder and AC3 as the audio encoder.
(Eventually delete the temp and source files, and move resulting .mp4 file to the "target" directory.)
Remove network mapped drives.
Exit batch program.

The code:

::Convert2mp4.bat
@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION

SET HOME=D:\ffmpeg

:: Establish working directories
PUSHD "\\Sy-6ba-100\Big Files One\Video\MP4"
SET target_path=%CD%
PUSHD "\\Sy-6ba-100\Big Files Two\F Drive\Video\AVI"
SET source_path=%CD%

:: Gather files to convert
FOR /F "delims=" %%A IN ('DIR /B /S *.avi *.mpg *.mpeg') DO (
    IF NOT EXIST "%tmp%\Convert\%%~nA.*.mp4" (
        IF NOT EXIST "%tmp%\Convert\%%~nxA" (
            ECHO COPY "%%A" %tmp%\Convert
            COPY "%%A" %tmp%\Convert
        )
        
        FOR /F "usebackq" %%B IN (`"MediaInfo.exe --Inform=Video;%%FrameCount%% "%tmp%\Convert\%%~nxA""`) DO (
            ECHO Frames to convert: %%B
        )
        
        FOR /F "usebackq" %%B IN (`"MediaInfo.exe --Inform=Video;%%FrameRate%% "%tmp%\Convert\%%~nxA""`) DO (
            SET /A fps=%%B
        )

        FOR /F "usebackq" %%B IN (`"MediaInfo.exe --Inform=Audio;%%BitRate%% "%tmp%\Convert\%%~nxA""`) DO (
            SET /A orig_ab=%%B
            SET /A orig_ab=!orig_ab!/1000
        )

        SET ab=
        FOR %%B IN (32 40 48 56 64 80 96 112 128 160 192 224 256 320 384 448 512 576 640) DO (
            IF NOT DEFINED ab (
                IF /I %%B GEQ !orig_ab! (
                    SET /A ab=%%B
                )
            )
        )
        
        SET input_parm=-threads 2 -i "%tmp%\Convert\%%~nxA"
        SET audio_parm=-acodec ac3 -ab !ab!k
            
        ECHO.
        ECHO.!time!
        ECHO.
        SET mode=slow
        SET video_parm=-vcodec libx264 -vpre !mode! -crf 22 -level 41 -r !fps!
        ECHO ffmpeg !input_parm! !audio_parm! !video_parm! -y "%tmp%\Convert\%%~nA.!mode!.mp4"
        ffmpeg !input_parm! !audio_parm! !video_parm! -y "%tmp%\Convert\%%~nA.!mode!.mp4"
        ECHO.
        ECHO.!time!
        ECHO.
        SET mode=medium
        SET video_parm=-vcodec libx264 -vpre !mode! -crf 22 -level 41 -r !fps!
        ECHO ffmpeg !input_parm! !audio_parm! !video_parm! -y "%tmp%\Convert\%%~nA.!mode!.mp4"
        ffmpeg !input_parm! !audio_parm! !video_parm! -y "%tmp%\Convert\%%~nA.!mode!.mp4"
        ECHO.
        ECHO.!time!
        ECHO.
        SET mode=fast
        SET video_parm=-vcodec libx264 -vpre !mode! -crf 22 -level 41 -r !fps!
        ECHO ffmpeg !input_parm! !audio_parm! !video_parm! -y "%tmp%\Convert\%%~nA.!mode!.mp4"
        ffmpeg !input_parm! !audio_parm! !video_parm! -y "%tmp%\Convert\%%~nA.!mode!.mp4"
        ECHO.
        ECHO.!time!
        ECHO.
        REM IF !ERRORLEVEL! EQU 0 (
            REM MOVE "%tmp%\%%~nA.mp4" "%%~dpA"
            REM DEL "%tmp%\%%~nxA"
        REM )
        PAUSE
    )
)
POPD
POPD

EXIT /B

As I said, I'm happy with the batch program so far; it (mostly) does what I want. You'll note that I'm actually invoking ffmpeg three times. This is to benchmark the different x264 presets (slow, medium, and fast). I'll eventually decide on one of those I think.

Here's the recipe in it's expanded form, with example file names, audio bitrate, fps, and using x264's "fast" preset:

ffmpeg -threads 2 -i "<temp file.*>" -acodec ac3 -ab 160k -vcodec libx264 -vpre fast -crf 22 -level 41 -r 25 -y "<temp file.mp4>"

From what I understand, which may be way off base, is that it's not the problem with the x264 encoder, but rather decoding of the source file.
Is that true? Is there a way to optimize the decoder to account for the "washed out" look?

If it's not the decoder, is there a way to match the brightness/chroma/luma/whatever(?) of the source?

Thank you in advance for helping a complete novice to this sort of thing! I wanted to be as thorough as possible, and not just ask "need x264 help". But doing so, I hope I didn't get into the tl;dr territory!

Edit: I did search for the term x264, and viewed 25 pages of results, but didn't find topics that matched my situation. I didn't check *every* thread; only those that sounded similar to my problem, and those with ambiguous thread titles (need x264 help, for example).

x264: x264 core:104 r1688 0b36c6d
ffmpeg: FFmpeg version SVN-r25512

[jagabo]

What player are you using? Are you playing two videos at the same time in media players to compare?

x264 does not change brightness or contrast levels. Your problem is elsewhere.

Also try the veryfast preset.

[orangeboy70]

Thanks for the reply! I'm spawning two instances of VLC media player on a Windows XP laptop to do the comparisons. The transcoding is taking place on a Windows Server 2003 box that has more processing power and RAM. My goal is to have my TiVo Premiere as my primary playback device that has a Broadcom BCM7413 system chip that does the decoding. Transferring unsupported formats to the TiVo involves transcoding to mpeg2, so it's not entirely possible to do a true apples to apples comparison there. However, I will look for native mpeg2 files to transcode to mp4, then transfer both the mpeg2 and mp4 files for comparison there (if I'm correctly interpreting you inferring that the playback decoder in VLC is producing this washed-out look on my laptop.)

[Baldrick]

It's because of overlay/hardware video acceleration. The first vlc media player uses it and the other not so it will look different.

[jagabo]

You have to be careful when using media players to compare two videos side by side. Media players usually use the graphics card's video processing features (video overlay, etc.) to display video. But those features are often limited to a single player. So one player will be using the graphics card's video rendering device and the other will be using the Desktop. Each has different proc amp settings (brightness, contrast, hue, saturation, gamma, sharpening, etc.) so the two players can look very different -- even if they are playing the exact same video. Be sure that's not your problem.

You can set VLC to output to Windows GDI. That will bypass any video processing the graphics card does, but will raise the CPU usage.

[orangeboy70]

Great explanations guys! If I keep only one instance open and take a VLC video snapshot of both videos at the same timestamp, I should see no differences in the resulting jpeg or png files? Or do stills also take advantage of hardware?

Edit: As expected, launching the mp4 file first, followed by the original avi, the original avi had the "washed out" look (not that I doubted you!)
I'm going to adjust my script, start transcoding, and not look back

[txporter]

orangeboy- You don't need to force level 4.1 for Tivo if you don't want to. If you just leave that setting off, x264/ffmpeg will just take the lowest level that fits the resolution/fps for the given video. You will not have any source that requires more than 4.1 anyhow and Tivo Premiere (or HD) really doesn't care as long as the level isn't above 4.1 (not really even sure that this is a problem since I haven't tested it myself).

Are any of these videos that you are converting directly from the Tivo itself, or are they all downloads from the net? If you have some Tivo downloads, you should consider looking into avisynth so that you can reverse telecine that material. I can supply you with some avs scripts that I use for that, if needed.

[orangeboy70]

orangeboy- You don't need to force level 4.1 for Tivo if you don't want to. If you just leave that setting off, x264/ffmpeg will just take the lowest level that fits the resolution/fps for the given video. You will not have any source that requires more than 4.1 anyhow and Tivo Premiere (or HD) really doesn't care as long as the level isn't above 4.1 (not really even sure that this is a problem since I haven't tested it myself).

Are any of these videos that you are converting directly from the Tivo itself, or are they all downloads from the net? If you have some Tivo downloads, you should consider looking into avisynth so that you can reverse telecine that material. I can supply you with some avs scripts that I use for that, if needed.

I seem to recall some issues with my Series 3 (648250) model with h264 @3.1. I'd have to check TCF to get the specifics, but I figured I'd be safe with always specifying 4.1. But yeah, my archived videos are 'net downloaded. Some are old (1970's) "home movies" of Led Zeppelin in concert, then transferred to digital and shared by the Zep community of traders. My TiVo recordings are transient; commercial-cut with kmttg then deleted after transfer. TV stuff I watch once, and that's it. I started looking into avisynth the past couple of days, but didn't get much further than the documentation - I didn't even look at the "first avs script" example. I WILL want to look into it in more detail when I focus on some of the more degraded Zep videos to see if they can be cleaned up. I may take you up on some of those scripts! I tend to try to work things out myself, but am not too proud to learn by example!

[txporter]

I hear you. Be warned though, trying to "fix" material that has already been encoded and sent out into the ether is normally much more difficult than dealing with the original source. I would recommend learning to play with avisynth with some easier footage before trying to recover some of your worst-case videos (try DVD rips, non-ABC/BBC Tivo downloads before those).