Bit rate for DV(720x576px) over Xvid/X.264/DivX? (and some other questions)

[Gew]

Greets.

Guess this is one of the most repeated questions that pops up here, but old posts on various web boards regarding this matter could be outdated and what-not, so I prefer a fresh topic with - hopefully - some fresh answers.

I have a Sony Handycam that I use to shoot DV video. Next stop, the compression lab.
I don't expect 100% top-of-the-line/noticible-only-in-the-sharpest-pixel quality, but just an general opinion on what's "ordinary" for regular everyday footage. On one board, I read that encoding Xvid with 1300-1500kbps equals to encoding with H.264 with only 1000-1200kbps? Is this correct? I guess that's really my main question in all this. Cause if that's true, I see no reason for using Xvid instead of let's say the free X264 codec for just about everything. I don't plan on playing Xvid's on xvid-supporting DVD-players, but just to have almost-original-quality-archives in some format not as space-demanding as MPEG2. Right now, I'm encoding my video with Xvid on 1940kbps. I've tried +/- 300 kbps but can't really notice any difference. On one board I read that this different is easier to see if you playback on a large output device such as a 48" LCD television (right now the most I've seen my work on is a 28" 4:3 CRT television).

So, I'm aware that "there's no right or wrong", or to phrase it "you need to find your own optimization", I'm just here for some tips. When I see a couple of replies with recommended bit rates, I guess I'll mean/median those and have my own starting point there.

I'm totally new to all this.
So, there'll be lots of things to learn.

Oh, btw, regarding the audio. I don't think the DV-captured original is more than 32kHz, so it would be no meaning in encoding with 44.1/48kHz then, right? 32 is enough. Bit rate for the audio? What's sufficient? 128? 224 like MPEG1? 256kbps audio would be sort of abundance, right?

Let's say.. 1300kbps video, 128kbps/32kHz audio, Xvid.
Would that decrease quality lots(!) from the original DV data?

How about the same settings but X264? Would that be nice?
Would let's say someone who get's a TV-OUT-playbacked2VHS copy of the output notice any difference on this as MPEG2/6000kbps and the settings above?

Finally..
One last question.
May sound lame but, I'm really not getting this..
Does the lenght(!!) of the video conclude difference in the bit rate setting???
HOW? I've read the Wiki article on 'Bit rate' to get a clue, but there's nothing.
Why am I wondering this? ..'cuz of all calculators wanting video lenght to decide appropiate bitrate. My only other guess is that this is just for those who wants a certain size on the output size, let's say 700mb for a CD-R. Because like, if 10 minutes of material would need a different bitrate than 20 minutes of the same material (with the same resulation, etc settings) sounds really CRAZY. If that's the case, I must be taking some serious math lessons.

Please, don't be rude because I'm new!

[jagabo]

Unfortunately, there is no way to tell you what bitrate is appropriate for your DV material. If you filmed a bowl of fruit under bright lights with a really good camcorder (low noise) on a tripod you can get away with very low bitrates. If you used a cheap handheld camcorder while whitewater rafting at dusk (high noise) you will need a very high bitrate.

If you want guaranteed quality you should use a constant quality encoding. In Xvid that's Single Pass, Target Quantizer mode. x264 has two options: Single Pass Quantizer Based, and Single Pass Ratefactor Based. Pick the quality (1/quantizer) you want and each frame is encoded at that quality level using whatever number of bits is necessary for that frame.

In general h.264 requires less bitrate than Xvid. This is also hard to quantify because the way the image degenerates is different. Xvid typically breaks up into blocky artifacts. h.264 becomes smeared and oily looking. Most people would say h.264 requires somewhere between 50 and 100 percent the bitrate of Xvid.

Bitrate is usually expressed in Kilo Bits Per Second. You don't have to use different bitrates for different lengths of video*. But as the length of the video goes up, so does the file size:

file size = bitrate * running time

*) Well, a short video may require a higher bitrate. But this is a matter content not length. A 60 second movie trailer is going to be full of lots of high action scenes, quick cuts, expolosions, and other shots that don't compress well. It will require a higher bitrate than the full movie which has a lot of low action scenes that compress far better. If you repeated the 60 second trailer 90 times to make it movie length it would require the same bitrate as the single iteration. It would simply be 90 times bigger.

[Gew]

Thanks for a _very_ good answer jagabo.
Detailed, yet not using too complicated terms.

The fact that the adequate bitrate is determined by (how) rapid movements the footage contains has been told/read lots of times already, but I guess (since suffering ADD etc) I need to hear it lots of times to be sure it isn't blocked by my (too much) selective perception in any way.

You gave good info on the quantaizer also. I'm interested in the quantaizer. Right now I'm encoding with Q:3.0 using Xvid. Seems to be alright. So, I guess it wouldn't be too much dropping a follow-up question. So, after looking through the tabs with Xvid encoding settings a couple of times, I found more options which interests me. "The big 100.000$ question" right now is the "quantization type". I can chose from H.263 (which I - after some reading - guess is meaning the compression technology used by Xvid/DivX) and MPEG. What's the difference, anyways? Is there better quality in any of those? I ran a few tests, and came to the conclusion that the same clip (duration/bitrate) got a little little bit smaller in size by using H.263 quantization. Does that mean MPEG offers better quality instead? Because they talk about this thingie with size/quality like scales. Really confuzing.

Then there is this "VHQ" (if I remember the letters correct) setting, which I can set to 0-4 (after what I recall), with a default at zero. In parasesis under the numbers I read like "mode searching" "fast searching" and suchlike. What is this? Does it matter for my quality/size? In what way, sort of?

Thanks again folks!

[jagabo]

The MPEG quantization matrix is a little sharper and retains a little more small detail than the h.263 matrix. If you are using constant quantizer encoding the MPEG matrix will give slightly sharper (and larger) results. With bitrate based encoding it becomes a trade off between blurriness and macroblock artifacts. For a certain bitrate you can get more sharpness but you'll also get more macroblocks with the MPEG matrix, or less sharpness and less macroblocks with the h.263 matrix. The differences are subtle with most material. You can also enter your own matrix (you can find many if you search). This can lead to better results for some types of material.

The Motion Precision settings (Precision and VHQ) control how wide a search is made for motion vectors. One of the major space saving techniques of MPEG encoding is finding blocks of pixels that have moved from frame to frame. Instead of re-encoding those parts of the picture those blocks are flagged simply to move from the old location to the new location.

For example, scrolling credits at the end of the movie may cause each pixel to be different from one frame to another. Encoding all those differences would require a lot of bitrate. But with motion vectors the encoder can just say "move all the blocks up 2 pixels, then add this little bit at the bottom". That can save a lot of bitrate.

The trade off of here is encoding speed. Searching for all those motions takes a lot of time. The more you search the more time it will take to encode. There can be a several fold difference in encoding speed between the fastest and slowest settings. The default "General Purpose" setting is a good compromise for most situations.

When using Target Quantizer based encoding the quality will be similar regardless of the Motion Precision settings -- but the file size will differ. As the search gets wider more motion vectors are found and the file will get smaller. In my experience there is rarely a difference of more than 5 percent between the General Purpose and the max settings. Often as little as 1 or 2 percent. Going the other way, to the fastest settings, will result in a significantly bigger file. Maybe double the size.

With bitrate based encoding the quality goes up as the search precision goes up. But going higher than the GP settings will only get you a tiny bit better quality. You can think of it like this: if changing from GP to highest only gets you a 2 percent smaller file in Target Quantizer mode, then using a 2 percent higher bitrate and the same precision setting in bitrate mode would get you the same quality difference as the same bitrate and the highest quality setting.

If you don't care how long the encoding takes just use the highest motion search precision settings. The penalty for the highest settings used to be much higher. But more recent versions of reduced this quite a lot.

One other thing you might consider if you want the highest quality Target Quantizer encoding: turn off b-frames (B-VOPs). B-frames are encoded with a higher quantizer (lower quality) to make them smaller. The idea being you won't notice a slight drop in quality for one or two frames as it will be restored once the following P and I frame comes along. File size will go up 10 to 20 percent though.

If you have a multicore processor be sure to get a multithreaded "experimental" release of Xvid 1.2x.