Handbrake Question (Mpeg2 to .264)

[awtm]

I have some movies I've recorded from TV on my PVR and I'm wanting to compress them to put on my NAS for my kids to watch. The source files are now in the Mpeg2 format and I was wanting to use Handbrake to compress them to .264 files. I have done one test of a 3.39GB source file (1.25 hr long movie) and used a setting of RF 20 in Handbrake (keeping all other video/audio settings the same as the source).

The end result was an mp4 file that is 2.97GB. I was hoping it might be a smaller sized file as I've read .264 can give similar quality at a 3rd the size of Mpeg2. Although looking at the "Data Rate" and "Total bitrate" at in each file's properties the .264 file is about a 3rd of the Mpeg2. Everything else in the properties box is the same in both files. But the end result is only a 0.42GB reduction in size.

I realise that by making the RF value higher I'll get a smaller file, but my research has shown around 20 seems to be what most people prefer (and Handbrake itself recommends this). So my question is, is this about right, or should I be tweaking HB to get a better/smaller end result.


cheers in advance.

[LMotlow]

Why are you recompressing? Your NAS can't play MPEG? If not, I'd suggest a better system.
MPEG2 is not an audio format.
Re-encoding never improves anything. Use whatever looks least damaged.

[redwudz]

I'm guessing you are compressing a HD file? What audio format are you converting to? You might take a look at the file with MediaInfo and show us what settings you use.

I use VidCoder instead of HB as I like it's GUI better.
If I backup a 'average' commercial DVD of around 7 - 8GB to MKV/H.264/AC-3 at a CQ of 18.5, I end up with a MKV of about .7 to 2.0GB, depending on the content and run time.
That's a fair bit of compression, but still decent quality.

And welcome to our forums.

[awtm]

Thanks for the responses.

The source files are Mpeg2 files that my PVR has recorded from TV (movies and TV shows). Some are HD some SD when broadcast. My NAS will play Mpeg, I was just thinking I could reduce the size to about a third if I compress to .264 (and get 3 times as many movies on my NAS disk).

As redwudz says, he's getting the files down to around a 1/4 of the size, I must be doing something wrong as I only shaved a small fraction off the original file. I could change the audio a bit for more savings (48 to 44.1 & 383 to 192) but I kept them the same for my test run to see what changing the video from Mpeg2 to .264 would do. And my answer so far is "not that much!" But as redwudz says he's making much greater savings. Seems it must be something I'm doing (or not doing). Guess I'll give Vidcoder a go and see if it's more friendly to folks who have no idea!

Thanks for the welcome redwudz.

[hello_hello]

Vidcoder and Handbrake shouldn't produce different results if you use the same settings (resizing, filtering etc) and the same x264 encoder settings.

Sometimes video is hard to compress if it's noisy or it already contains compression artefacts, and the bitrate reduction can depend how heavily the source is compressed to some extent. For example you might easily re-encode 7GB worth of DVD vob files and output a 2GB file, but if that same movie was on a single layer DVD you might end up compressing 4GB worth of DVD vob files down to 2GB, so the ratio changes a bit.

If a standard definition video is anamorphic and you resize to square pixels, the resolution can make a difference. ie a 720x576 16:9 PAL DVD resized to 1024x576 for encoding means there's more pixels to encode than the source.

There's lots of variables, but probably the closest you could get to a bitrate comparison would be "encoding the same video at the same quality with x264 will require one third the bitrate of an mpeg2 encoder".... that sort of thing, but when you're encoding a video with one encoder and then re-encoded that version with another, it might be a different story.

[awtm]

I bumped the audio down a level and gave Vicoder a go at redwudz suggested 18.5 (keeping aspect ratio, frame rate, size, etc the same) and my file came out bigger than the source!!

Just for clarity hello_hello - are you saying my PVR is recording the TV signal as a compressed signal to start with, so trying to re-compress it into .264 leaves me with little room to reduce the file size. But if I started with a DVD source I could a much bigger file reduction?

[hello_hello]

Just for clarity hello_hello - are you saying my PVR is recording the TV signal as a compressed signal to start with, so trying to re-compress it into .264 leaves me with little room to reduce the file size. But if I started with a DVD source I could a much bigger file reduction?

They're both compressed, but how much is the question. Assuming the DVD video bitrate is much higher than the bitrate used for broadcasting (they're both mpeg2), then it can be somewhat relative.

Take a pristine video and encode it with an mpeg2 encoder at (just making up numbers) 4000kbps and that results in a 3GB file. Take that same video and encode it again with an mpeg2 encoder at 8000kbps and that gives you a 6GB file.
Now re-encode the original video and the two mpeg2 encodes with x264 at the same quality setting. Assuming it's a perfect world, they'd all come out the same size, let's say 2GB in this case, so for the mpeg2 sources you'd have a one third bitrate reduction for one and a two third reduction for the other.

I say "in a perfect world" as neither of the mpeg2 encodes would be a perfect replica of the original. The lower bitrate encode could potentially introduce more compression artefacts and/or removed some of the finer picture detail, so in the real world the x264 encodes wouldn't be exactly the same size as the sources are different.
It mightn't necessarily be a huge difference, but it's possible the x264 encode of the 8000kbps source might result in a slightly smaller file size than the encode of the 4000kbps source, and it's possible the re-encode of the original video would result in the smallest file size of all, because that's the "cleanest" version and the easiest one to compress.

You can't really think of it in terms of the source video bitrate too much. The encoder doesn't know. The source is de-compressed and sent to the encoder which re-compresses it as best it can, oblivious to it's previous life.

Can you post a small sample of one of the files you're re-encoding? Although to be honest, I think it probably is was it is and your result won't be different to anyone else's, unless noise reduction or some sort of filtering might help, but that's probably not a job for Handbrake/Vidcoder and it'd require more time and learning.

[awtm]

Thanks hello_hello, that's much clearer to me now. I had just read somewhere .264 is 1/3 the size of Mpeg2 somewhere and so thought great I'll save some disk space. But like life itself, things are never that easy! I did another pas at CF25 and managed to get from 3.39 down to 2.2GB and it still looks ok on the computer monitor. Still need to check on the big screen.

Not sure if you wanted to see the source (Mpeg2) or encoded file (.264)? In case it helps, here are the specs of both from MPC-HC

SOURCE
Format : MPEG-PS
File size : 3.39 GiB
Duration : 1h 13mn
Overall bit rate mode : Variable
Overall bit rate : 6 635 Kbps

Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 2
Format profile : Main@High
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : Variable
Format settings, picture struc : Frame
Duration : 1h 13mn
Bit rate mode : Variable
Bit rate : 6 119 Kbps
Maximum bit rate : 15.0 Mbps
Width : 1 440 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.157
Time code of first frame : 00:00:00:00
Time code source : Group of pictures header
Stream size : 3.13 GiB (92%)

Encoded to .264
Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42
File size : 2.20 GiB
Duration : 1h 13mn
Overall bit rate : 4 301 Kbps
Encoded date : UTC 2036-02-06 06:28:16
Tagged date : UTC 2036-02-06 06:28:16
Writing application : HandBrake 0.10.2 2015060900

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 1h 13mn
Bit rate : 4 070 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.079
Stream size : 2.08 GiB (95%)
Writing library : x264 core 142 r2479 dd79a61
Encoding settings : cabac=1 / ref=2 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=hex / subme=6 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=30 / rc=crf / mbtree=1 / crf=25.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=62500 / vbv_bufsize=78125 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Encoded date : UTC 2036-02-06 06:28:16
Tagged date : UTC 2036-02-06 06:28:16
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

[awtm]

Here's a clip from the source.
http://files.videohelp.com/u/255597/Bruno%20(clip).mpg

[hello_hello]

Thanks hello_hello, that's much clearer to me now. I had just read somewhere .264 is 1/3 the size of Mpeg2 somewhere and so thought great I'll save some disk space. But like life itself, things are never that easy!

It might be a case of the same quality at one third the bitrate, as long as both encoders are compressing the same source.

You seem to be using the "fast" x264 speed preset. A slower preset might increase the quality for a particular CRF value, although there's no guarantee it'll decrease the file size. Sometimes it'll increase.

(References to anamorphic video that follow refer to video where the resolution and display aspect ratio are different and the picture is stretched to the correct aspect ratio on playback)

MediaInfo says for your sample the resolution is 1440x1080 (4:3) and the aspect ratio is 16:9, which means it's anamorphic. Nothing out of the ordinary, but if you resize and encode at 1920x1080 (16:9) you're encoding a bunch of pixels that weren't there in the first place (the source is 1440x1080). That's not going to help reduce the file size.
Rather than resize by increasing the width, you can reduce the height and encode at 1440x808, but to be honest I'd resize that one down to 720p (ie 1280x720) as I don't think there's much more picture detail than that.

When 1080i is really 1080i (your sample isn't, it's progressive) I'd generally de-interlace to 50fps and resize to 720p (it retains the smoothness of motion when the source is interlaced). For Handbrake that means selecting "bob" as the de-interlacing method and resizing to 720p, but in this case that'd just repeat every frame as the source is 1080p at 25fps, not interlaced, so you don't need any de-interlacing. Just resize to 1280x720 for 1280x720 at 25fps.

Or.... instead of selecting "Anamorphic None" for Handbrake, which is what I assume you're using (it resizes to square pixels), you could try "Anamorphic Strict". That'll encode using the same 1440x1080 resolution as the source while setting a 16:9 display aspect ratio. You'll have to test it to make sure your player displays it correctly. Some hardware players don't understand anamorphic MKVs/MP4s and will display them as though the pixels are square, so in this case it'd display as 4:3 and look "squished". If that happens you need to resize to square pixels (Anamorphic None) but I'd reduce the resolution when resizing to help reduce the file size.

The sample is noisy and contains compression artefacts (probably typical Australian free to air broadcast quality) there's fine background detail and a lot of movement, all making it harder to compress (harder than lots of fairly static clean images), but to give you an idea, when I re-encoded your sample at 1920x1080, CRF 23 and the medium x264 speed preset, the bitrate was 9990kbps and it was bigger than the source. Encoding again while resizing to 1280x720 dropped the bitrate to 4950kbps. There's a slight blurring of fine background detail, but under the circumstances....
Or encoding it anamorphically at 1440x1080 resulted in 9210kbps, which I'll admit was less of a reduction in bitrate compared to 1920x1080 than I expected.

Me.... I'd probably resize to 1280x720 and reduce the CRF value to CRF21 to compensate for the reduction in resolution a bit, and maybe hang on to a little more detail that way. That took the bitrate back up to 6570kbps for your sample, but if reducing the bitrate is a priority, I think in this case reducing the resolution is a reasonable way of achieving that, rather than encode at a higher resolution but crappier quality. You could even try resizing down further to something like 1024x576 or 960x540 etc, to see how far you can go before there's too much loss of detail for you. It's all personal preference really.

[awtm]

Thanks for the in depth reply hello_hello. You've gone above and beyond the call of duty.

Just a last question on de-interlacing, is it something I should worry about if all my recordings are coming from my PVR and FTA TV?

[hello_hello]

To be honest I have no idea what's broadcast as progressive or interlaced.
Until I found this, that is. It looks like just about everything is broadcast as interlaced, but that doesn't mean it's necessarily interlaced. As was the case for your sample, the video was progressive.

I see some of the stations have stepped out of the video dark ages and started broadcasting in MPEG4 instead of MPEG2. That must have happened while I wasn't paying attention.

Anyway, if you see "combing" where there's movement such as the combing in the pic here, after re-encoding without the de-interlace or decomb filter enabled, then you need to de-interlace. If not, you don't. It's better not to de-interlace progressive video as it can potentially blur a little. Maybe run an encode over a small section without any filtering enabled to see how it looks.

If it's interlaced, I'd select "bob" for the deinterlacing filter. That'll de-interlace to 50fps progressive rather than 25fps progressive and motion will look smoother.
If you don't want to mess around, the decomb filter tries to be clever for you. If you enable "bob" for the decomb filter it'll leave frames without combing alone (the combing effect only happens when there's movement), it'll blend de-interlace minor combing and it'll "bob" de-interlace where there's substantial combing the same way the de-interlace filter does it. The idea being if a video is progressive it won't be de-interlaced so it won't be blurred, and frames are only de-interlaced as required. The downside is it's not perfect. Sometimes it misses small amounts of combing in a frame.

When you enable the de-interlacing filter the output should always be a 25fps progressive constant frame rate, or a 50fps progressive constant frame rateif you select "bob".
The decomb filter works differently so if you want a constant frame rate you have to specify a constant frame rate output. Either 25fps, or 50fps, or "same as source". I'm not sure. I'd have to check again to see how it works exactly as I don't use Handbrake myself. I think by default, the output is set to variable frame rate and the decomb filter will drop frames it considers to be duplicates, giving you a variable frame rate output. In a perfect world that probably doesn't matter, but I'd stick with a constant frame rate. A variable frame rate makes sense for NTSC which can be a mixture of 23.976fps film and 29.970fps video, but I'm not sure there's any benefit to outputting a variable frame rate for PAL.