H.264 video is dim with overlay / x264vfw question

[serexl]

Hello,

I have two questions:

1. I compressed a source video using VDub+x264vfw, CRF 1. The output looks great in VDub, but the entire video is dimmed when I play it Media Player Classic (MPC) or Windows Media Player, unless I disable overlay or change Overlay Mixer in MPC to something else like VMR9 (I'm also unable to take a screenshot of the dimmed video with overlay). If I compress the video with Camtasia+Quicktime H.264, I get a very high quality video without the overlay/dim problem. What is the source of this problem, and how can I solve it?
2. How come even with CRF 1, the VDub video is so much smaller than the Camtasia+Quicktime video (see details below)? Is there a quality difference?

Any help greatly appreciated!

Thanks,
Serexl

Source video
Size: 38.2 MB
Codec: FPS1, FRAPS Codec
kbps: 53575
Screenshot: original.bmp

Camtasia+Quicktime H.264 with the "best quality" single-pass
Size: 10.5 MB
Codec: avc1, H.264
kbps: 14691
Screenshot: camtasia.bmp

VDub+x264vfw, CRF 1 single-pass
Size: 1.3 MB
Codec: H264, H.264/MPEG-4 AVC
kbps: 1738
Screenshot: vdub.bmp (without overlay)

[poisondeathray]

1) if it's too "dim" with overlay mixer, then your graphics card settings aren't calibrated

read the posts by "jagabo" in this thread, and he posts a video so you can calibrate it
https://forum.videohelp.com/threads/313178-A-problem-with-dynamic-ranges

2) crf 1, but what other settings? and what version of x264 vfw

for example if qpmin is left at default, which is usually 10 , then your quantizer won't go that low...

The important thing is can you tell the difference

[serexl]

1. Thank you so much poisondeathray, that solved my problem.
2. I'm using x264vfw r1195 from here. I'm just curious why I'm unable to replicate the high-quality video using Camtasia. When I use qpmin=1, I get the following error when running "Multipass - 1st pass" (to be followed by "Multipass - nth pass"):

x264vfw [warning]: Error: 2pass curve failed to converge
x264vfw [warning]: target: 4000.00 kbit/s, expected: 3594.44 kbit/s, avg QP: 1.0296
x264vfw [warning]: try reducing target bitrate or reducing qp_min (currently 1)

To be sure, I don't need such a high bitrate, but why am I unable to use a higher bitrate than 3,895 kbps, when (A) the source video is 53,575 kbps, and (B) it is possible to use Camtasia to make a compressed video that is 14,691 kbps. What am I missing?

Thank you for your help,
Serexl

[creamyhorror]

That's a very strange encoding result, unless your video is extremely simple. Post the source, or a sample of it.

To be sure, I don't need such a high bitrate, but why am I unable to use a higher bitrate than 3,895 kbps, when (A) the source video is 53,575 kbps, and (B) it is possible to use Camtasia to make a compressed video that is 14,691 kbps.

Why do you need a bitrate higher than 3895? If x264 refuses to go above a certain bitrate, that generally means it's simply not necessary to for the quality (CRF) level you've selected. Conversely, if Camtasia makes a big file when the source doesn't require it, then it's a lousy encoder.

Of course, it's possible that x264vfw has screwed up. How good/bad does the tiny x264 encode look exactly? You might try using mainstream x264, since x264vfw is an offshoot that hasn't been actively developed (much) in the past 2 years, and this new version may be buggy.

edit:
ahaha, subspace/continuum. With a source like that, it's no wonder x264 is producing a tiny file. I doubt you'll see any difference between the source and encode (after you fix the levels/colors problem). You shouldn't be judging "quality" by the bitrate that the encoder turns out.

For sources like this, I suggest bumping up max consecutive b-frames to 8 or 9 and ref frames to about 7. Also turn on b-pyramid and bump subpixel ME refinement to 8 or 9, assuming you want additional coding efficiency. Unless you have reasons for turning them off, that is.

[poisondeathray]

serexl - your screenshots don't match the information you've conveyed, and it shows r1195 - please update them

for example , your error log shows 2pass error, but your settings show you used crf. Or post your logfile


What happens when you use the same settings with the CLI version? x264vfw has been very buggy in the past, and r1471 is rather new, there could be other bugs

Source video will always be more bitrate because it's likely RGB 4:4:4 (if using lossless FRAPS mode), whereas your h.264 encodes are YV12 4:2:0 and lossy, they will always look worse than the source by this fact alone.

I'm guessing you messed up some settings, but if you wanted an even higher bitrate you could use all I-frame (intra only) or reduce your I:P and P:B ratios

[jagabo]

With Min QP set to 10 you are not getting a CRF 1 encode. You're getting a CQP 10 encode (Min QP overrides CRF). A true CRF 1 encode will be a huge file -- maybe even bigger than your source. Most people consider CRF 18 to be close to visually lossless for most sources. If you want to be extra safe use CRF 15.

[serexl]

poisondeathray -- I updated the screenshots

I've been trying very hard to follow the suggestion to use the CLI version, but I don't know how! I attempted using MeGUI (x264 1376 - Jeebs's patched build v2) but to no avail. Could someone kindly point me in the right direction?

MeGUI Settings
AviSynth Script:

AVISource("C:\Documents and Settings\Serexl\My Documents\test\source.avi", audio=false)
#deinterlace
#crop
#resize
#denoise

ConvertToYV12()

The source video is the FRAPS video. Whatever preset encoder setting I use, the output is always a blank video.

[poisondeathray]

Do you have fraps installed? The megui preview is black? If you don't have a decoder (ie. fraps isn't installed), install ffdshow and enable fraps in the vfw configuration (yellow icon)

You need ConvertToYV12() to the end of the script (megui will ask to append it later anyway)

Save your script and open up that .avs in vdub => is it black too ?

I don't know if 2pass works for the vfw version (I never use vfw). It works fine for CLI

[serexl]

Fraps is installed. In the ffdshow vfw configuration, I changed Fraps from disabled to libavcodec.

The video displays properly in both the MeGUI preview and in VDub.

[poisondeathray]

what are you saying then? you can't see the encoded video? how are you watching it? do you have a directshow avc decoder installed (e.g. enable h.264 in the ffdshow configuration (red icon)) ? or just use vlc or smplayer or mpchc => which contain decoders with the player

I just did some quick tests on that x264 vfw version and it appears to be working fine for me 2pass. I did some earlier tests with crf mode when it first came out and integration with another program and it worked fine for that too.

[serexl]

Yes, the encoded video is entirely blank, e.g., when I open it, it is stuck at 5/5 seconds, I cannot seek through it, and the screen is black.

H.264/AVC's decoder is libavcodec in ffdshow video decoder configuration (the other option is ffmpeg-mt).

[poisondeathray]

Is the video in a container? like .mp4, .mkv or is it rawavc stream .264 ?

what does mediainfo say about the encoded file (view=>text , and copy & paste the text back here)

[serexl]

Yes, mp4:

General
Complete name                    : C:\Documents and Settings\Serexl\My Documents\test\source.mp4
Format                           : MPEG-4
Format profile                   : JVT
Codec ID                         : avc1
File size                        : 3.25 MiB
Duration                         : 5s 833ms
Overall bit rate                 : 4 678 Kbps
Encoded date                     : UTC 2010-03-09 01:36:43
Tagged date                      : UTC 2010-03-09 01:36:43

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.1
Format settings, CABAC           : Yes
Format settings, ReFrames        : 3 frames
Codec ID                         : avc1
Codec ID/Info                    : Advanced Video Coding
Duration                         : 5s 833ms
Bit rate mode                    : Variable
Bit rate                         : 4 674 Kbps
Nominal bit rate                 : 8 000 Kbps
Maximum bit rate                 : 6 679 Kbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate mode                  : Constant
Frame rate                       : 30.000 fps
Resolution                       : 8 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.169
Stream size                      : 3.25 MiB (100%)
Writing library                  : x264 core 80 r1376M 3feaec2
Encoding settings                : cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / sliced_threads=0 / slices=4 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=3 / b_pyramid=1 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=1 / wpredp=2 / keyint=24 / keyint_min=2 / scenecut=40 / rc_lookahead=24 / rc=2pass / mbtree=1 / bitrate=8000 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=40000 / vbv_bufsize=30000 / ip_ratio=1.40 / aq=1:1.00
Encoded date                     : UTC 2010-03-09 01:36:43
Tagged date                      : UTC 2010-03-09 01:37:04

I also attached the MeGUI log (I'm using the x264 Bluray - 2 pass preset).

[poisondeathray]

Did you try another player ? like SMPlayer, KMplayer or VLC ? It could be that your directshow mp4 splitter has issues

BTW , You're using a preset meant for avchd/blu-ray (it's not the same settings that you used for the other vfw version if you were trying to compare) , also your x264 version is different r1376 vs. r1471

[serexl]

I made a custom encoder setting to compare with x264vfw:

program --pass 2 --bitrate 4000 --stats ".stats" --slow-firstpass --thread-input --bframes 9 --b-pyramid normal --ref 7 --vbv-maxrate 31250 --subme 9 --partitions all --trellis 2 --output "output" "input"

The encoded video plays perfectly on VLC. Should I reinstall ffdshow to fix the mp4 splitter problem?

MediaInfo of video:

General
Complete name                    : C:\Documents and Settings\Serexl\My Documents\test\source.mp4
Format                           : MPEG-4
Format profile                   : JVT
Codec ID                         : avc1
File size                        : 3.46 MiB
Duration                         : 5s 833ms
Overall bit rate                 : 4 978 Kbps
Encoded date                     : UTC 2010-03-09 01:52:13
Tagged date                      : UTC 2010-03-09 01:52:13

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.0
Format settings, CABAC           : Yes
Format settings, ReFrames        : 7 frames
Codec ID                         : avc1
Codec ID/Info                    : Advanced Video Coding
Duration                         : 5s 833ms
Bit rate mode                    : Variable
Bit rate                         : 4 974 Kbps
Nominal bit rate                 : 4 000 Kbps
Maximum bit rate                 : 7 022 Kbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate mode                  : Constant
Frame rate                       : 30.000 fps
Resolution                       : 8 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.180
Stream size                      : 3.46 MiB (100%)
Writing library                  : x264 core 80 r1376M 3feaec2
Encoding settings                : cabac=1 / ref=7 / deblock=1:0:0 / analyse=0x3:0x133 / me=hex / subme=9 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / sliced_threads=0 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=9 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=1 / wpredp=2 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=4000 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Encoded date                     : UTC 2010-03-09 01:52:13
Tagged date                      : UTC 2010-03-09 01:53:05

[poisondeathray]

If it plays on VLC, but not MPC or WMP, that suggests you have directshow system filter issues. You need a mp4 splitter (since you already have ffdshow installed as the decoder). I suggest the one from the mpchc project standalone filter (or just avoid using MPC or WMP)

You can find the standalone filters links on this homepage. You could register the .ax with radlight filter manager or graphstudio
http://www.xvidvideo.ru/media-player-classic-home-cinema-x86-x64

Now your initial complaint was you couldn't reach >3.8Mbps with the vfw version, but this megui encode is over that.

Notice you set 8Mbps, but only achieved ~4.6Mb/s. If this trend was the same on a longer sample size (yours was like 5sec long), this supports what creamyhorror postulated - that your content complexity isn't really that high. ie. it's pretty much saturated and any more is a waste of bitrate. You can try all Intra, or adjusting the IP, PB ratios , lowering qpmin. Or even using lossless mode.

I tried some quick tests on some other movie content with the vfw version and it seem to work fine (I was able to easily surpass 4Mbps)

If you want to test the corresponding 1471kmod version, download the correct binary, and replace the .exe in the x264 folder in the megui folder . You may have to update the GUI, because some of the switches and commands have changed

http://komisar.gin.by/

[serexl]

poisondeathray -- Thank you so much for your help. I will experiment with your suggestions and stick with x264 CLI instead of vfw (not sure what the problem is on my end).