Looking for H.264 conversion help

[fordprefect138]

I currently store all my movies on a system with over 2TB of disc space, and it's full. I've been looking at converting my movies to H.264 for a while. This weekend I picked up a new TV that allows playback of H264 (mkv and DivX) content without the need for an external device. Up until now I've been using my Xbox running XBMC to playback the ISO files. Obviously there was limitations with this, such as no HD, and other items I won't go into here.

Anyway, I have two DVD samples that I've been working with for 3 days now. One is animated and the other contains some complex scene detail such as wheat grass blowing below a blue sky with some light clouds. So far I've tried the conversion with Handbrake and MeGUI.

I've tried a number of different setting in Handbrake for the content. Both Constant Quality - up to 80% - and setting the bitrate - up to 4 Mbps - along with toying with a number of the different options.

I'm trying to get a conversion that could be considered indistinguishable from the original. On the non-animated content, I see TONS of compression artifacts in the sky and the blowing wheat grass. On my animated material, it's not too bad, but the area along the black lines of the characters contains heavy compression artifacts.

I work with some commercial H.264 Real Time Encoders (RTE) and the output of the H.264 content looks very close to the original MPEG2 content. I understand Handbrake and MeGUI aren't commercial application, but they're also not doing real time encoding. I would expect to be able to get the same level of quality from the x264 encoders even if it takes 10 hours to run the conversion.

It seems so far, I've not found the correct combination of settings to some anywhere close to the source material on complex scene material.

Can anyone point me to some resources that may help in my endeavor?

[disturbed1]

If you want "indistinguishable from the original" don't bother converting at all This is the holy grail that has yet to be achieved. Beauty is in the eye of the beholder and sometimes we have to settle for good enough. What is good enough for you, I, and everyone else is completely different.

For SD content this is the x264 options I use with HandBrake (CLI)

--x264opts level=3.1:ref=4:mixed-refs:bframes=2:b-rdo:bime:weightb:direct=auto:filter=-2:-1:subme=6:trellis=1:partitions=p8x8,b8x8,i4x4,i8x8:8x8dct:me=umh:vbv-bufsize=14000 --two-pass --turbo

I change the --size option or --quality option depending on the content. Out of 1500 movies, there are only a handful that are larger than 999MB. This looks (for the most part) good enough on a 56" screen. I personally do see a slight, very slight quality difference between the original DVD and the mkv. The rest of the friends and family notice no difference. The above settings blow Xvid out of the water using ~= compression rates. Xvid needs just about 1.5-2x the bitrate on most if it ever reaches the same quality the h264 encodes do.

Before you attempt to copy and paste the above - it will not work on the Windows client verbatim. You'll have to convert the Linux options for the version of x264 you are using. I only posted it as an example - and to show you how bad my eyes are

Things that will give you better quality at the cost of time to encode -

increase the number of reference frames. There is a standard here for each level. Level 3.1 (SD) can go as high as 13 reference frames IIRC. I personally have playback issues with more than 8 reference frames. Nor do I see any difference using more than 3 B frames on most encodes. Matter of fact - on some of the tests I've done, increasing B-frames beyond 3 has is some cases hampered quality, as expected.

adjust the filter settings. -2:-1 seems to retain a good portion of detail. Increasing this number adds more deblocking. Let your eyes be the judge.

Up the subme to 9

Change the me=umh to me=tesa

Making the changes above (ref=8:bframes=3ubme=9:me=tesa) changes my encode rate from ~40FPS to ~6FPS. And yes there is a difference, but I'm just too damn impatient to wait that long .

This only shows how I convert my personal SD DVD collection to mkv. The main purpose for my settings are a good balance between size, quality, and speed. You can do alot better than the settings I use, but I found it to be the sweet spot for my viewing application.

[disturbed1]

Can anyone point me to some resources that may help in my endeavor?

Forgot this -
http://www.digital-digest.com/articles/x264_options_page1.html
http://trac.handbrake.fr/wiki/x264Options
http://mewiki.project357.com/wiki/X264_Settings

[minerva.thegift]

Hmm, I'm afraid that I'm not the best judge of video quality - I think my eyes might not be the greatest. For me, Handbrake or MeGUI (I've used them both, though I'm generally switching to Handbrake because of ease of use) with SD video using x.264 at about 1000 kbps looks fine, so I up it to 1200 kbps just to be on the safe side. A few things might get a bit grainy, but it generally isn't noticeable to me.

Perhaps a bit more data could help - would you mind posting some MediaInfo summaries for your sample files? This handy little program (which should be linked thanks to the nifty way this forum works) is great for troubleshooting and explaining yourself here on the forums. You could also post the encoding parameters - I know there is a tab for this in the stable release (.9.3.0 I think) of Handbrake and I'm sure you could find it in the logs or something in MeGUI. This info doesn't mean as much to me, but other forum members might be able to catch things in it (like some of the suggestions disturbed1 has already made, such as bframes=3) and these might help up your quality.

By way of example, here's the MediaInfo from a short clip I have - source was a DVD, encoded it using Handbrake at 1200kbps. Note that it mentions mkvmerge because I used it to 'touch up' things a bit (set the languages for the audio tracks from "[und]" to "[eng]" and other similar cosmetic changes.)

General
Complete name                    : [file].mkv
Format                           : Matroska
File size                        : 63.2 MiB
Duration                         : 5mn 33s
Overall bit rate                 : 1 590 Kbps
Encoded date                     : UTC 2009-10-22 03:16:54
Writing application              : mkvmerge v2.9.8 ('C'est le bon') built on Aug 13 2009 12:49:06
Writing library                  : libebml v0.7.7 + libmatroska v0.8.1

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : Main@L3.0
Format settings, CABAC           : Yes
Format settings, ReFrames        : 2 frames
Muxing mode                      : Container profile=Unknown@3.0
Codec ID                         : V_MPEG4/ISO/AVC
Duration                         : 5mn 33s
Bit rate                         : 1 174 Kbps
Nominal bit rate                 : 1 200 Kbps
Width                            : 720 pixels
Height                           : 480 pixels
Display aspect ratio             : 16:9
Original display aspect ratio    : 16:9
Frame rate                       : 29.970 fps
Standard                         : NTSC
Resolution                       : 24 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.113
Stream size                      : 46.6 MiB (74%)
Writing library                  : x264 core 76
Encoding settings                : cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x1:0x111 / me=umh / subme=6 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=12 / nr=0 / decimate=1 / mbaff=0 / bframes=2 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=0 / keyint=300 / keyint_min=30 / scenecut=40 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=1200 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / aq=1:1.00
Language                         : English
colour_primaries                 : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M
transfer_characteristics         : BT.709-5, BT.1361
matrix_coefficients              : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M

I'm trying to think of ways to help you, but I'm perhaps not the best person to help you with this (since the video quality isn't that big of a concern for me.) At least with some more info others might be able to give you some more tips. In general, though, I do think you've got the right idea - my experience is that x.264 video is the best encoder out there (at least that I've tried) in terms of balancing quality and size. I also strongly favor the .mkv container (since it can easily store multiple audio and/or subtitle streams) but of course opinions vary there.

Post up whatever info you can and we'll see what we can do

[creamyhorror]

edit: Wait - are you sure you're preprocessing your source correctly? NTSC DVDs generally need to be IVTCed ("Detelecine" in Handbrake), while some PAL DVDs need to be deinterlaced ("Decomb"). Not doing these will result in a mess of an encode, no matter your x264 settings.

I've tried a number of different setting in Handbrake for the content. Both Constant Quality - up to 80% - and setting the bitrate - up to 4 Mbps - along with toying with a number of the different options.

There's not much point going above 65% (= CRF 18 ) quality for most encodes, and definitely no point going above 69% (= CRF 16). Start at around 64-65% for high quality and modify from there. Don't use 2-pass, use CRF (aka Handbrake's constant quality).

Use the latest beta version of Handbrake, not the old obsolete one. MeGUI is more up-to-date, but ideally switch to the development server and update to the latest x264.

To retain more detail, reduce your deblock settings (try -2,-1 or -2,-2). You can try increasing aq-strength to 1.2-1.8, which will shift bits from complex areas (e.g. edges) to simpler, flatter areas; this is more useful at lower bitrates, though. Try these as a basic starting point (for live action material):

--crf 18 --ref 6 --bframes 5 --direct auto --deblock -2:-2 --subme 10 --analyse all --trellis 2 --aq-strength 0.8 --psy-rd 1.0:0.0 --b-adapt 2 --me umh --merange 24

I'm trying to get a conversion that could be considered indistinguishable from the original. On the non-animated content, I see TONS of compression artifacts in the sky and the blowing wheat grass. On my animated material, it's not too bad, but the area along the black lines of the characters contains heavy compression artifacts.

I suspect you aren't using very high-quality profiles.

I work with some commercial H.264 Real Time Encoders (RTE) and the output of the H.264 content looks very close to the original MPEG2 content. I understand Handbrake and MeGUI aren't commercial application, but they're also not doing real time encoding. I would expect to be able to get the same level of quality from the x264 encoders even if it takes 10 hours to run the conversion.

x264 at non-realtime speeds will definitely beat out RTEs. You can expect very close to transparent output for most sources.

Post an extract of the source you're trying to encode and I'll give it a shot. To Mediafire, maybe.

[mh2360]

There's not much point going above 65% (= CRF 18 ) quality for most encodes, and definitely no point going above 69% (= CRF 16). Start at around 64-65% for high quality and modify from there. Don't use 2-pass, use CRF (aka Handbrake's constant quality).

I'm not a fan of CRF. On most recent "clean" looking DVDs it might be fine, but on some older "grainy" movies the file size can become monsterously large. This can be a problem where a 4GB limit comes into play, for example devices that only accept FAT32 formatted drives.

Here is my x264 command line which I use for DVD conversions. The results are pretty much identical to the DVD orginal.

x264.exe --pass 1 --stats "1.stats" --bitrate 2800 --fps 25 --no-deblock --merange 16 --b-adapt 1 --qcomp 0.6 --ref 3 --direct auto --partitions p8x8,b8x8,i4x4,i8x8 --subme 1 --level 3.1 --no-8x8dct --keyint 250 --min-keyint 25 --me hex --mixed-refs --weightb --no-fast-pskip --trellis 0 --vbv-bufsize 14000 --vbv-maxrate 14000 --rc-lookahead 60 --sar 889:625 --threads auto --output NUL "%Filename1%.avs"

"x264.exe --pass 2 --stats "1.stats" --bitrate 2800 --fps 25 --no-deblock --merange 16 --b-adapt 2 --qcomp 0.6 --ref 3 --direct auto --partitions p8x8,b8x8,i4x4,i8x8 --subme 9 --level 3.1 --no-8x8dct --keyint 250 --min-keyint 25 --me umh --mixed-refs --weightb --no-fast-pskip --trellis 1 --vbv-bufsize 14000 --vbv-maxrate 14000 --rc-lookahead 60 --sar 889:625 --threads auto --output "%Filename1%.264" "%Filename1%.avs"

It could be improved upon, but my settings are limited to what the Xbox 360 will play.

[stiltman]

Here's what I do in Handbrake
https://forum.videohelp.com/topic374148.html

Under video, I have it set to 65%
Under Audio, I have it set to AC3

[creamyhorror]

There's not much point going above 65% (= CRF 18 ) quality for most encodes, and definitely no point going above 69% (= CRF 16). Start at around 64-65% for high quality and modify from there. Don't use 2-pass, use CRF (aka Handbrake's constant quality).

I'm not a fan of CRF. On most recent "clean" looking DVDs it might be fine, but on some older "grainy" movies the file size can become monsterously large. This can be a problem where a 4GB limit comes into play, for example devices that only accept FAT32 formatted drives.

I always do test encodes, so on difficult sources I increase the CRF accordingly. Using a fixed bitrate will be either wasteful or insufficient, in my view. I'd rather keep roughly the same quality and make exceptions for the anomalies that pop up.

[fordprefect138]

Thanks for all the feedback. As I looked into this further, I found I was not doing an apples to apples comparison between my ISO files and the transcoded mkv files. Up until now, I have been watching all of my ISO material via my xbox using XBMC. I began wondering if the pixel shader rendering was helping the video quality.

Soooo... yesterday I picked up a WD TV Live. When I playback and compare the original ISO and transcoded mkv on the same device, I'm hard pressed to tell ANY difference - and that's standing within 12" from a 42" TV.

I will continue to play with the settings to see what works best for each type of move, but I think I finally have a handle on this now.

[totalz]

Is anyone using MainConcept Reference? I'm also trying to transcode a dvd to h264 to maintain quality and to achieve smaller file size. The reason I use MainConcept Reference is that it let me crop the video if necessary. However, I cannot find any option on H264 CRF, there's the 2-pass option, but I think CRF will produce better result.