Looking for best size/quality ratio encoding options for MP4 container

[CasanovaFly]

An HTPC is a goal of mine, but that's a distant goal. To make the wait easier, I've purchased a WDTV Live media player. As such, I'm looking at getting into the MP4 game. I haven't had much experience with it (save for converting some stereo sound stuff to MP4 via Handbrake) so I'm looking for some advice on what I should throw in the container.

The video is pretty obvious. I'm looking at getting a reasonable amount of quality and looking for variable bitrate encoding options, or something similar that will give me the fluctuations I want while staying within a reasonable file size limit.

Audio is really what I'm here for. I want 5.1 channel audio. The WDTV Live can support a bevy of options, but I don't really know what audio works best in an MP4 container, especially for my needs.

I'm also unsure of which encoder to use. Handbrake is nice, but I'd like something with a little more depth. I've got my eyes on potentially MediaCoder or MeGUI, but again, I'm awaiting the advice of people who know more than me.

Thanks in advance!

[jagabo]

x264 gives the best compression at this time. I'd recommend MKV rather than MP4. Keep the original AC3 audio.

[CasanovaFly]

Thanks for the quick reply jagabo!

I'm looking to decrease the bitrate of audio and video to get the file size down to an agreeable compromise (if I could afford a field of hard drives... holy moly!) Would you recommend staying with MKV on a reduced bitrate, while trying to reduce and keep AC3 as the audio format of choice?

[theewizard]

if you haven't done it already

check the FORMAT specs for WDTV for the different formats it supports

then make your choice

i prefer mp4 to mkv, but that is me

[CasanovaFly]

if you haven't done it already

check the FORMAT specs for WDTV for the different formats it supports

then make your choice

i prefer mp4 to mkv, but that is me

I've checked the formats and it's easier to list the ones it doesn't. The point of this thread is to get some input on which of the many it supports would best suit my needs, while being compatible.

That said, why do prefer MP4 over MKV?

[theewizard]

NOT just the formats ... the specs it will PLAY in those formats !

MKV is very open, it allows you do things other containers will not

frame resolutions, bit rates, mix and match codec pairs 'audio & video' in combinations not allowed in other containters

but they don't always play well together, on other players or different PC's, and can be a real PITA if you decide to change to another format using the MKV as the source

[jagabo]

MKV is much more flexible than MP4. Apple's main interest is making their life easier by supporting only a small number of options/features. MP4 didn't even support AC3 until a few years ago. Quicktime on Windows is a disaster.

The audio is usually much smaller than the video in an A/V file. So there's usually no need to compress it. For example, a Blu-ray rip I have on my computer right now is 18 GiB of h.264 video, 0.5 GB of AC3 audio.

The WDTV Live is very flexible. If you stick with Blu-ray disc compatible settings or less you'll be fine.

[hello_hello]

An HTPC is a goal of mine, but that's a distant goal. To make the wait easier, I've purchased a WDTV Live media player. As such, I'm looking at getting into the MP4 game. I haven't had much experience with it (save for converting some stereo sound stuff to MP4 via Handbrake) so I'm looking for some advice on what I should throw in the container.

The video is pretty obvious. I'm looking at getting a reasonable amount of quality and looking for variable bitrate encoding options, or something similar that will give me the fluctuations I want while staying within a reasonable file size limit.

The simplest way (and probably the best for 99.99% of us) to encode with the x264 encoder is to pick the slowest x264 speed preset you can stand (I generally just use the default medium or the slow preset myself) and then the CRF value which gives you the quality you're happy with. The bitrates/file sizes will vary quite a bit but the quality relative to the original will always be pretty much the same. CRF 18 is around where the x264 encoder is considered to be "transparent" (the higher the value the lower the quality). I use CRF 18 for everything except 1080p where I use CRF 20 to keep the file size down a bit (higher resolutions can get away with higher CRF values).
Aside from that, you can select one of the x264 tunings and a profile and level if you wish, but there's no real need to mess around with advanced encoder settings.

Audio is really what I'm here for. I want 5.1 channel audio. The WDTV Live can support a bevy of options, but I don't really know what audio works best in an MP4 container, especially for my needs.

MP4 sucks. MKVs are much easier to work with thanks to MKVMergeGUI. If the audio is AC3 I keep it (re-encoding it doesn't reduce the file size enough) while I convert DTS to AAC.

I'm also unsure of which encoder to use. Handbrake is nice, but I'd like something with a little more depth. I've got my eyes on potentially MediaCoder or MeGUI, but again, I'm awaiting the advice of people who know more than me.

I've never been a fan of MediaCoder. I've used MeGUI for years. It can have a bit more of a learning curve than other programs, but I think it's worth it. MeGUI is an AVISynth based encoder GUI but it doesn't require you to know how to create AVISynth scripts. It does that for you when setting up an encode. It does however, make it easy to experiment with modifying scripts manually and learning more about AVISynth as you go.

[CasanovaFly]

NOT just the formats ... the specs it will PLAY in those formats !

MKV is very open, it allows you do things other containers will not

frame resolutions, bit rates, mix and match codec pairs 'audio & video' in combinations not allowed in other containters

but they don't always play well together, on other players or different PC's, and can be a real PITA if you decide to change to another format using the MKV as the source

You're right. Good call. I didn't give you enough credit on your suggestion, and I apologize.

From what I can tell from the user manual, it again seems quicker to list what it doesn't support. There doesn't appear to be any kind of restriction on MKV, and maybe only the following on MP4's: "MPEG4.2 Max (ASP@L5 1920x1080p30 or 1280x720p60), bit rate (SD:20, HD:40)" Same thing with audio. There's a note about 2-channel and Dolby Digital (which I assume to be AC3): "2-channel only. 2+channel signal is sent via digital pass through to a receiver." I assume that it preserves the 5.1 AC3 and pushes it out to a receiver to handle it, since the box only has default 2-channel output. It does have a digital audio output, though, so it's able to achieve 5.1. This is fine for now, since the shitty speakers I have are standard 2-channel I believe, that extend the sound to 5.1. I make the Dolby Digital/AC3 assumption, though, because that's how I've been operating for years. I note this assumption because in the manual it says, under supported audio, that it supports Dolby Digital AND AC3. They're separate entries in the same "supported audio" column, so I just wanted to make note.

Anyway, it seems I'm pretty wide open for what is and isn't supported, which is nice.

MKV is much more flexible than MP4. Apple's main interest is making their life easier by supporting only a small number of options/features. MP4 didn't even support AC3 until a few years ago. Quicktime on Windows is a disaster.

The audio is usually much smaller than the video in an A/V file. So there's usually no need to compress it. For example, a Blu-ray rip I have on my computer right now is 18 GiB of h.264 video, 0.5 GB of AC3 audio.

The WDTV Live is very flexible. If you stick with Blu-ray disc compatible settings or less you'll be fine.

I'm liking the idea of keeping the MKV, assuming playback is a breeze (haven't tested anything, yet). I'd encoded to WMV-HD in the past, and the settings I developed and would like to keep (assuming a near 1:1 ratio in file size based on similar bitrates) is 3500kbps variable video, 440kbps variable audio. Again, this is all in the interest of saving HDD space (and money, two things which I severely lack).

The simplest way (and probably the best for 99.99% of us) to encode with the x264 encoder is to pick the slowest x264 speed preset you can stand (I generally just use the default medium or the slow preset myself) and then the CRF value which gives you the quality you're happy with. The bitrates/file sizes will vary quite a bit but the quality relative to the original will always be pretty much the same. CRF 18 is around where the x264 encoder is considered to be "transparent" (the higher the value the lower the quality). I use CRF 18 for everything except 1080p where I use CRF 20 to keep the file size down a bit (higher resolutions can get away with higher CRF values).
Aside from that, you can select one of the x264 tunings and a profile and level if you wish, but there's no real need to mess around with advanced encoder settings.

I'm gonna ask you to dumb this down for me a tad and explain what CRF is (or not necessarily explain, but tell me where to find it in an encoder, maybe what it stands for, and what it does).

MP4 sucks. MKVs are much easier to work with thanks to MKVMergeGUI. If the audio is AC3 I keep it (re-encoding it doesn't reduce the file size enough) while I convert DTS to AAC.

If you wouldn't mind, what do you use to convert your DTS to AAC?

I've never been a fan of MediaCoder. I've used MeGUI for years. It can have a bit more of a learning curve than other programs, but I think it's worth it. MeGUI is an AVISynth based encoder GUI but it doesn't require you to know how to create AVISynth scripts. It does that for you when setting up an encode. It does however, make it easy to experiment with modifying scripts manually and learning more about AVISynth as you go.

Music to my ears. I've heard great things about MediaCoder. I enjoy opening it and being assaulted with a bevy of options I've never seen (that must mean its good!) But whenever I've tried to use it, I've had errors. I tried briefly to work it with WMV and failed horribly. I tried recently to get something working in the way of MP4's to try and solve my own queries and got massive errors when trying to push out AAC or AC3 audio. I've used MeGUI for a while for this and that, so I'm comfortable with it. I've never been able to get AviSynth scripting, though. It's just above me. I need someone to sit me down and teach me exactly what to do, step for step, but this isn't grade school so I'll make due.

[Kerry56]

Perhaps I've missed it, but what format are you starting with? If you are converting dvds or blu ray to mkv or mp4 files, the simplest, and most commonly used tool is Handbrake. I prefer mkv, since there are tools available that make it easier to work with if necessary.

Try a test encode using Handbrake with these settings in the main window on the right: Regular-->High Profile

In the video tab have the framerate same as the source, and put a dot in the circle for constant framerate.
Also in the video tab, use the X264 Preset slider and set it to Slow. And last in the video tab, set the CRF to 20 (should be default)
These settings will not produce the highest quality, but should be a good compromise on size/quality. If it isn't good enough, do another test and change the CRF to 19 or 18.

The audio tab is more of a question mark. I normally let AC3 audio pass through. If you are starting with DTS, I'd convert to AC3. The bitrate you choose depends on how fussy you are with audio, but at least 192 up to 448.

[CasanovaFly]

Perhaps I've missed it, but what format are you starting with? If you are converting dvds or blu ray to mkv or mp4 files, the simplest, and most commonly used tool is Handbrake. I prefer mkv, since there are tools available that make it easier to work with if necessary.

Try a test encode using Handbrake with these settings in the main window on the right: Regular-->High Profile

In the video tab have the framerate same as the source, and put a dot in the circle for constant framerate.
Also in the video tab, use the X264 Preset slider and set it to Slow. And last in the video tab, set the CRF to 20 (should be default)
These settings will not produce the highest quality, but should be a good compromise on size/quality. If it isn't good enough, do another test and change the CRF to 19 or 18.

The audio tab is more of a question mark. I normally let AC3 audio pass through. If you are starting with DTS, I'd convert to AC3. The bitrate you choose depends on how fussy you are with audio, but at least 192 up to 448.

Will likely be starting with high bitrate MKVs and shrinking them down. I don't like the CRF method (I realize what it is after having checked out Handbrake) and prefer the variable bitrate method (which Handbrake also presents). I'll have to check up on the size "gains" (as in gained hard drive space) from converting AC3 to a lower bitrate. Again, I'd like to stay around 3500/440. That seems to be my sweet spot for both size and audio/visual enjoyment.

[hello_hello]

I'm gonna ask you to dumb this down for me a tad and explain what CRF is (or not necessarily explain, but tell me where to find it in an encoder, maybe what it stands for, and what it does).

CRF = Constant Rate Factor. It's x264's quality based, single pass encoding method.
If you ran 2 identical encodes which resulted in exactly the same average bitrate, one using CRF encoding and the other using 2 pass encoding, the video would be encoded in exactly the same way (there's very slight differences but effectively it's the same). Therefore the usual choices are generally either 2 pass encoding, where you pick the file size/bitrate in advance and the quality is unknown, or CRF encoding where you pick the quality in advance but the file size is unknown.
I prefer using the latter method, and because CRF encoding doesn't need to run 2 passes, it's quicker than 2 pass encoding.

With MeGUI, if you open the x264 encoder configuration and use the button to load the default settings, MeGUI defaults to Encoding Mode: Targeting Quality, and the default quality is 23, which is just a user friendly way of specifying CRF23. If you check the box to show x264's advanced settings, the encoding mode changes to Constant Quality with a default value of 23. It's still CRF23. I've never quite understood why MeGUI needs to give it two different names. Either way though, when you change the quality setting to something other than 23, you'll see MeGUI adds --crf to the command line along with the chosen CRF value.
Everything else you need is pretty much on the initial x264 configuration page. The x264 speed preset and tuning can be specified there, you can select an appropriate target playback device if it's in the list and MeGUI will change the necessary x264 settings accordingly. You can also specify a profile and level if you want to. High Profile, Level 4.1 is pretty much standard these days. Or selecting DXVA as the target playback device does the same thing.

For the record, MeGUI makes it easy to switch to 2 pass encoding if you want to even with the encoder configured for CRF encoding. If you use the AutoEncode button to add an encoding job to the Queue, the AutoEncode window lets you specify a file size (including audio). If you do, MeGUI will switch to 2 pass encoding mode for you.

If you wouldn't mind, what do you use to convert your DTS to AAC?

The NeroAAC encoder. MeGUI can convert with it, or often I'll use foobar2000. I just use the default quality setting of 0.5 for NeroAAC. It's pretty much the same principle as x264 CRF encoding. It gives you a VBR encode at the bitrate required for the quality you specify.

I've used MeGUI for a while for this and that, so I'm comfortable with it. I've never been able to get AviSynth scripting, though. It's just above me. I need someone to sit me down and teach me exactly what to do, step for step, but this isn't grade school so I'll make due.

I'm certainly no expert when it comes to AVISynth but I've learned the basics using MeGUI's Script Creator and it's preview button in combination with trial and error. If I want to experiment with a new AVISynth filter I download it, put the required files in the AVIsynth plugins folder, follow the instructions for adding the correct syntax to the script, and then I use the trial and error method again until it actually works.

[hello_hello]

I don't like the CRF method (I realize what it is after having checked out Handbrake) and prefer the variable bitrate method (which Handbrake also presents).

I'm not actually sure what Handbrake's variable bitrate method is (unless your just referring to 2 pass encoding), but CRF encoding definitely uses a variable bitrate.

I'll have to check up on the size "gains" (as in gained hard drive space) from converting AC3 to a lower bitrate. Again, I'd like to stay around 3500/440. That seems to be my sweet spot for both size and audio/visual enjoyment.

Using the default NeroAAC quality setting, the average bitrate isn't lowered enough to make converting AC3 audio worthwhile. You probably only save around 100MB for a movie. Unless of course you downmix it to stereo.
When re-encoding DTS it's a different story. It's more like a 600MB to 700MB size reduction. I just happened to be converting a DTS file from an episode of a TV show. The default NeroAAC quality setting gave me 5.1ch audio with an average bitrate of 398Kb/s, but it will vary.

[CasanovaFly]

I'm not actually sure what Handbrake's variable bitrate method is (unless your just referring to 2 pass encoding), but CRF encoding definitely uses a variable bitrate.

Using the default NeroAAC quality setting, the average bitrate isn't lowered enough to make converting AC3 audio worthwhile. You probably only save around 100MB for a movie. Unless of course you downmix it to stereo.
When re-encoding DTS it's a different story. It's more like a 600MB to 700MB size reduction. I just happened to be converting a DTS file from an episode of a TV show. The default NeroAAC quality setting gave me 5.1ch audio with an average bitrate of 398Kb/s, but it will vary.

A ran a couple tests through Handbrake today, testing the CRF method. I think I'll still stick with choosing the variable (or adaptive) bitrate over it, just because I'm a control freak and I have to know what's going on at all times. I like 2-pass, as well, so that works out in my favour.

This has been good and productive; a lot of knowledge transference. I like it. I think it's time to run some tests on my own and go from there.

Thanks everyone! Does this forum have some kind of rep or reward system for helpful posters?

[jagabo]

A ran a couple tests through Handbrake today, testing the CRF method. I think I'll still stick with choosing the variable (or adaptive) bitrate over it, just because I'm a control freak and I have to know what's going on at all times.

That's nonsense. Using 2-pass VBR you control the size but not the quality. Using CRF encoding you control the quality, not the size. Otherwise, you have the same control of all the other settings using either method.

[hello_hello]

A ran a couple tests through Handbrake today, testing the CRF method. I think I'll still stick with choosing the variable (or adaptive) bitrate over it, just because I'm a control freak and I have to know what's going on at all times. I like 2-pass, as well, so that works out in my favour.

If the variable or adaptive bitrate method is x264's average bitrate encoding I wouldn't use it. The way I understand it the encoder just keeps taking bitrate guesses as it encodes to achieve the target average bitrate by the end of the encode, so the quality will probably vary as goes.

CRF and 2 pass both give you a constant quality encode. CRF just encodes at the specified quality. 2 pass runs the first pass to work out how to distribute the bits according to the specified bitrate/file size, then the second pass for the actual encoding.

If by being a control freak you're referring to the final file size, like jagabo I'd also prefer to control the quality.
I recall when I first started encoding Xvid/AVIs years ago, everyone used 350MB for an episode of a TV show so I thought I'd do that too. It didn't take long to work out some encodes looked great while others looked pretty crappy according to how hard the video was to compress, so I shifted my OCD tendencies from file size to quality. I was just looking at some old AVI encodes of a TV series. All the same resolution, all the same duration, all the same quality. The smallest is 224MB. The largest is 930MB. The average is 427MB.
At least x264 makes it easy..... just pick a quality and go. No more compression tests and 2 pass encoding like the Xvid days.

[CasanovaFly]

That's nonsense. Using 2-pass VBR you control the size but not the quality. Using CRF encoding you control the quality, not the size. Otherwise, you have the same control of all the other settings using either method.

I'm not sure I understand. Don't they do the same in the end? If I specify a certain quality, it will give it whatever bitrate it needs. If I feed it the same bitrate that it determined it needed for the CRF method, do I not arrive at the same result?

[poisondeathray]

That's nonsense. Using 2-pass VBR you control the size but not the quality. Using CRF encoding you control the quality, not the size. Otherwise, you have the same control of all the other settings using either method.

I'm not sure I understand. Don't they do the same in the end? If I specify a certain quality, it will give it whatever bitrate it needs. If I feed it the same bitrate that it determined it needed for the CRF method, do I not arrive at the same result?

Yes, the end result will be quite similar

But the problem is how do you know what bitrate to chose in the first place? Because it's going to be different for different sources and scenarios

[CasanovaFly]

That's nonsense. Using 2-pass VBR you control the size but not the quality. Using CRF encoding you control the quality, not the size. Otherwise, you have the same control of all the other settings using either method.

I'm not sure I understand. Don't they do the same in the end? If I specify a certain quality, it will give it whatever bitrate it needs. If I feed it the same bitrate that it determined it needed for the CRF method, do I not arrive at the same result?

Yes, the end result will be quite similar

But the problem is how do you know what bitrate to chose in the first place? Because it's going to be different for different sources and scenarios

Hmm.

Realistically, how different can it be? I'll have to test this for myself, but still, any ideas?

[poisondeathray]

Hmm.

Realistically, how different can it be? I'll have to test this for myself, but still, any ideas?

Realistically very different. An action movie might require 2-3x more bitrate than a slow moving drama to achieve a certain level of "quality". Different sources will compress differently. Things like noise, grain, movement, action will all be harder to compress. So something like a low light ,noisy , shaky home video will be extremely difficult to compress compared to a very clean, brightly lit, slow moving "talking head" interview piece

Arbitrarily picking a certain "x" bitrate might overallocate to some content, and underallocate to others

[enim]

Arbitrarily picking a certain "x" bitrate might overallocate to some content, and underallocate to others

Same holds true for encoder settings and processing filters. Certain bit-rate, encoder settings and processing filters will give similar results on similar sources. These may produce absolutely different results with source having diffrent characteristics.

I could not comment much on x264 settings which are the true barriers for me that I could ever jump.

[hello_hello]

Realistically, how different can it be? I'll have to test this for myself, but still, any ideas?

I'm guessing a bit, but to give you an idea.... I use the same settings/quality pretty much all the time. When re-encoding Bluray I generally resize to 720p and keep any original AC3 audio. My movie encodes would probably vary between around 2GB and 5GB.

As well as how hard/easy the video is to re-compress, aspect ratios differ by quite a bit. 1280x720 is more video to encode than 1280x532, even if the duration is the same for both.

As a rule of thumb, many people use single layer DVD size for 720p movies and dual layer DVD size for 1080p. That should ensure most encodes are quite good quality, but many could probably be encoded using a lower bitrate without a noticeable drop in perceived quality. That's fine if you're going to burn each movie to a disc after encoding.... why not use the maximum bitrate the disc will hold.... but I don't burn to disc, and ensuring the video fits nicely on a disc requires specifying the bitrate and using 2 pass encoding which takes longer than CRF.

[jagabo]

Realistically, how different can it be?

See the videos in this post:

https://forum.videohelp.com/threads/295672-A-problem-for-video-experts?p=1811057&viewfu...=1#post1811057

That's Xvid and an extreme example. But the issues are the same with x264 or any other high compression codec.

[CasanovaFly]

I'm about to head out of town for the week (and out of civilization) so I'll have to leave this stuff until I get back. Thanks to everyone for everything so far. I'll drop back in when I return, get my feet back under, and get the opportunity to run some tests.

Thanks again!