unexpected video bitrate

[Environmentalist]

this has happened to me yet again and so now i seek an answer. the latest occurrence is as follows:

i'm attempting to convert wmv to xvid. 590mb wmv cbr is 3874 kbps (according to GSpot). the problem is that, whether i set xvid target bitrate to 1500 or 3500, the encoded avi is around 200mb with bitrate around 1220 kbps.

i'm no expert at this stuff, but i've converted a lot of video, nevertheless. i'm doing everything the same as usual, so this is indeed unexpected and i can't explain it. i can only guess that there is something about the wmv that is causing this, but what?

MEDIAINFO
Format : Windows Media
FileSize/String : 591 MiB
Duration/String : 20mn 31s
OverallBitRate_Mode/String : Constant
OverallBitRate/String : 4 025 Kbps
OverallBitRate_Maximum/String : 4 041 Kbps
Encoded_Date : UTC 2008-10-27 13:32:29.150

Video
ID/String : 2
Format : VC-1
Format_Profile : MP@ML
CodecID : WMV3
CodecID/Info : Windows Media Video 9
CodecID/Hint : WMV3
CodecID_Description : Windows Media Video 9 - Professional
Duration/String : 20mn 31s
BitRate_Mode/String : Constant
BitRate/String : 3 848 Kbps
Width/String : 640 pixels
Height/String : 480 pixels
DisplayAspectRatio/String : 4:3
FrameRate/String : 25.000 fps
BitDepth/String : 8 bits
ScanType/String : Progressive
Compression_Mode/String : Lossy
Bits-(Pixel*Frame) : 0.501
StreamSize/String : 565 MiB (96%)
Language/String : English
NumberOfFrames : 30766

[Baldrick]

What do you convert with? A ffmpeg frontend?

[Environmentalist]

i use both TMPGEnc and Virtualdub. the result is the same in both.

[jagabo]

The source video may not have enough detail/action to hit the bitrate you requested with the settings used. How does the video look? That's what matters.

You could try setting the Minimum Quantizers to 1 to get higher bitrates. And disable B frames. And use very short GOPs. But these are settings to raise the bitrate, not really to improve the quality.

[Environmentalist]

well, considering my limited knowledge, my general rule is higher bitrate = higher quality. besides, i don't make those changes any other time so why should i have to now? what makes this wmv different from the rest? and if the source video doesn't have the detail/action to hit the bitrate i request, then how does it have the high bitrate itself?

incidentally, there is a visual difference, i think. but regardless, i'm interested in this phenomenon as much as anything.

curiously, i just encoded the wmv again in Vdub using ffdshow. xvid setting was 3000 kbps. resulting video has bitrate of 2142 kbps and encoded 5-6 times faster than normal xvid encoder. still an unexpected bitrate, but a better one. and why so much faster?

[budwzr]

What profile are you using in Xvid? And did you dink around with any other settings?

[Environmentalist]

hmm, so that last ffdshow encode i did was actually of the wmv embedded in an avi shell (directstream copy via Vdub). i just did it again using the original wmv and got the 465mb, 3000 kbps avi file i expected all along. but this avi has a very distinct grainy look to it that the previous ffdshow encode did not (no ffdshow settings were changed in between).

xvid profile is advanced simple @ L5 with adaptive quantization, quarter pixel, and gmc. i use a 'user defined' quality preset with only changes from default being a max iframe interval 150, vhq mode 3, and min quantizers all set to 2. i don't even remember what all that means. i read about it in the past. i adjusted what made sense to me; the results were good so i stuck with it. maybe something in that is the reason? i'll test it. maybe ffdshow is using xvid encoder with all default settings? i'll investigate.

[jagabo]

Different encoders + different settings + different videos = different speeds + different quality + different bitrates.

Why don't you use Xvid with Target Quantizer 2 encoding? You'll always get the quality you want and you'll save a lot of time. Let the bitrate fall where it will. Bitrate based encoding is only useful when you want a particular file size. Examine the videos in this post:
https://forum.videohelp.com/threads/295672-A-problem-for-video-experts?p=1811057&viewfu...=1#post1811057

[Environmentalist]

thx, jagabo. i'll be sure to take a look at that.

an update as to what i've been doing the last 30 mins....

i set xvid to default settings and test encoded the wmv, each test changing one setting from the default. now all settings are back the way i had customized them and, for better or worse, i'm getting the expected result. that is to say, i can't replicate the unexpected result! so, needless to say, now i'm really confused. but i still have one of the files. too bad it doesn't say anything about the xvid bitrate setting when it was encoded. i'll just have to investigate more the next time it happens.

Format : AVI
Format/Info : Audio Video Interleave
File size : 199 MiB
Duration : 20mn 31s
Overall bit rate : 1 357 Kbps
Writing library : TMPGEnc 4.0 XPress Version. 4.7.4.299

Video
ID : 0
Format : MPEG-4 Visual
Format profile : Advanced Simple@L5
Format settings, BVOP : Yes
Format settings, QPel : Yes
Format settings, GMC : 3 warppoints
Format settings, Matrix : Default (H.263)
Muxing mode : Packed bitstream
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 20mn 31s
Bit rate : 1 189 Kbps
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.155
Stream size : 175 MiB (88%)
Writing library : XviD 1.2.1 (UTC 2008-12-04)

btw, all test encodes had that same grainy look to them, but this unexpected result did not. if can't simply be a bitrate vs. quantizer thing, b/c i've been doing bitrate for a while now. something diff about this particular wmv? i donnae. more testing. more learning.

[jagabo]

I assume you're runing 2-pass VBR encodes -- maybe the stats file wasn't being updated for some reason? And you are running first pass, then second pass, right?

By the way, using Xvid's GMC is asking for trouble -- no standalone media player I know of can handle it (many players can handle Divx's single warp point GMC, but none Xvid's three warp point GMC). Even QPel support is spotty.

[budwzr]

OK, great! Now stop dinking with it.

[Environmentalist]

single pass usually. single pass in this case. basically i'm reencoding to reduce file size. my pc is my media player. no worries there.

how can i learn if i don't dink?

[budwzr]

True

how can i learn if i don't dink?

[jagabo]

single pass usually. single pass in this case.

Single pass CBR is the worst possible encoding method in Xvid. If you use enough bitrate for the action shots you'll waste bitrate on the rest of the video. If you use a lower bitrate the slow shots will still look fine but the action shots will break up into a mess of macroblocks. If you insist on bitrate based encoding you should use 2-pass. During the first pass the encoder examines the video to determine which shots need more bitrate, which less (the information is saved in the "stats" file). During the second pass bits are allocated based on that information. In the end the average bitrate matches your requested value. A 2-pass VBR encode has much better overall quality than a single pass VBR encode of the same size (unless you are using very high bitrates).

I wonder if you thought you were running a single pass encode earlier but had Xvid set to "twopass 2nd pass". That could cause the type of problem you were seeing.

[manono]

A 2-pass VBR encode has much better overall quality than a single pass VBR encode of the same size (unless you are using very high bitrates).

Since you were talking about CBR encoding earlier in the post, this is a typo, isn't it? Did you mean to say, " A 2-pass VBR encode has much better overall quality than a single pass CBR encode of the same size..."?

[jagabo]

A 2-pass VBR encode has much better overall quality than a single pass VBR encode of the same size (unless you are using very high bitrates).

Since you were talking about CBR encoding earlier in the post, this is a typo, isn't it? Did you mean to say, " A 2-pass VBR encode has much better overall quality than a single pass CBR encode of the same size..."?

Oops. Yes, it's a typo. Thanks for catching it.

Although, as you know, even with the typo it's correct! LOL

[Environmentalist]

alright, i was looking at your target quantizer post, jagabo, and i'm not sure i see the difference. for the wmv in question here, i could set the bitrate to 3000 kbs and get a 465mb file; or i can set the TQ to 3.5 and get a similar size and bitrate file. i know its 3.5 only b/c i first tried 2, 3, and 4. a lot of testing to get the desired result, that is, a smaller (but not too much smaller) file with little to no visual difference. by selecting a bitrate, i can effectively select a % reduction of the original file.

so if i understand correctly, by my single pass/bitrate method i can specify and get what i'm looking for more quickly, but at the expense of some untapped compression potential. either way, the resulting file is the same % reduction in quality of the original.

beyond that, i can't take much from your post. the noisy video has more data to be stored, hence the higher bitrate. in the 2 pass video, you forced a much lower bitrate, thus losing the detail that is the noise. you'd get a similar result doing a single pass of the noisy video at 1400 kbps, albeit slightly larger.

i get the whole cbr/vbr single/2pass thing. i do cbr at what i feel is high enough bitrate not to degrade the 'busy' scenes. i figure any additional compression i could get isn't worth the additional time it takes. for example, 3 hrs to turn 1gb into 500mb is fine; why spend 5 hrs just to make it 450mb instead?

[budwzr]

Your thinking doesn't work because there are other dynamics involved. It's an algorithmic process, not just bitrate. Some things in video are "exponential", like a 640X480 reolution is 4 times bigger that 320X240, but in numbers only looks twice as big.

[Environmentalist]

admittedly, i don't understand a great deal on this matter, but atm i feel that my thinking/understanding is indeed adequate, however my ability to express entirely those thoughts here in words is certainly lacking. i could be wrong. i can't be sure, though, unless you elaborate on my shortcomings.

[budwzr]

It's simple. Cutting half the bitrate does not equal half the quality. It depends on other factors like motion, frames per second, resolution, color changes, etc.

Every video is like a human being, each one is different. If you take off half a woman's makeup does that make her 50% uglier? It depends!

admittedly, i don't understand a great deal on this matter, but atm i feel that my thinking/understanding is indeed adequate, however my ability to express entirely those thoughts here in words is certainly lacking. i could be wrong. i can't be sure, though, unless you elaborate on my shortcomings.

[poisondeathray]

so if i understand correctly, by my single pass/bitrate method i can specify and get what i'm looking for more quickly, but at the expense of some untapped compression potential. either way, the resulting file is the same % reduction in quality of the original.

beyond that, i can't take much from your post. the noisy video has more data to be stored, hence the higher bitrate. in the 2 pass video, you forced a much lower bitrate, thus losing the detail that is the noise. you'd get a similar result doing a single pass of the noisy video at 1400 kbps, albeit slightly larger.

i get the whole cbr/vbr single/2pass thing. i do cbr at what i feel is high enough bitrate not to degrade the 'busy' scenes. i figure any additional compression i could get isn't worth the additional time it takes. for example, 3 hrs to turn 1gb into 500mb is fine; why spend 5 hrs just to make it 450mb instead?

The reason you're not seeing a big difference with CBR, is probably because your source file was CBR encoded. It already has attenuated peaks, and wasted bitrate. It already has a flat distribution.

If you used a different source higher quality source, say blu-ray, you can often get huge spikes. Things like explosions cause massive peaks maybe 40Mb/s (or higher instanteous buffered peaks) , but things like titles and credits might only need 0.5Mb/s . Using CBR approach here will produce very very poor results. You waste a lot on low sections (valleys) and clip spikes (peaks). Overall quality , and efficiency is a lot worse

[Environmentalist]

i get that. i resize some vids, too. 1/2 resolution is around 1/4 bitrate.

what i may or may not be confused about is why i should set the TQ instead of setting a bitrate. i just don't see the distinction between the two b/c the resulting quality for any TQ setting is a bitrate, X, which could simply be set directly. if i set the TQ to 2 for my wmv file above, the final file would've been over 1gb, twice as large as the original. if i know the bitrate of the original file, why play a guessing game setting TQ? i can get the same % reduction by setting the bitrate relative to the original bitrate; and if i'm reducing the resolution, i can calculate the appropriate bitrate required to maintain quality at that smaller resolution (then bump it up 5% or so for safe measure).

yeah, i'm not dealing with blue ray quality sources here. some pushing dvd quality at best. and yes, the wmv discussed in this post is cbr. but i've encountered some vbr files, too. i haven't noticed any seriously degraded segments after converting to cbr. probably i'm keeping the bitrates up high enough to avoid that, at the expense of a little file size, sure.

[budwzr]

The goal is to set a minimum acceptable uniform playback quality level across the entire file. The only way to do that is to use a quantizer and leave the filesize up to the encoder. To set the filesize first and work backward puts the cart before the horse.

The only time filesize matters is when burning a disc. And since DVD is dying, why bother.

[jagabo]

what i may or may not be confused about is why i should set the TQ instead of setting a bitrate.

Because with Target Quantizer you know what the picture quality is going to be before you encode (regardless of the frame size, frame rate, video complexity, etc.), but you don't know the file size. With Bitrate based encoding you know what the file size will be but you don't know how the picture quality will turn out.

[budwzr]

As a side note:

When you render a full length video, you don't want to watch and check the whole thing for compression artifacts. Quality based encoding gives you peace of mind.

[Environmentalist]

if, for example, i know that a TQ of 4 will produce a video of 1000 kbps, is that rendered video exactly the same as a video rendered with a 1000 kbps bitrate setting? or does TQ produce something more equivalent to a vbr file? and why isn't TQ an option in 2pass encoding?

btw, i did a 3000 kbps single pass of the 590mb wmv and got a 465mb video. i did a 2pass encode at same bitrate and it was 461mb. the latter took 25+ mins longer. finally, i did a TQ of 3.85 and the rendered video had a bitrate of around 2970 kbps and size 458mb. which of the 3 is better? presuming you'll say the TQ, how would i set TQ so as to achieve my goal? ...that is, a file that is reduced in size by an amount corresponding to the relative value i place on a video's content. a TQ of 3.85 might reduce this particular wmv by 20%, but will it be the same for the next one? will TQ of 3.5 be like an 18% reduction? how do i use TQ to obtain repeatable, consistent results like i've been getting via bitrate settings?

i tried applying the vdub smoother filter @ a mild setting of 5 and the TQ 3.85 video size dropped all the way to 120mb and 670 kbps. how do i know that that video is still the same quality as the other without rendering it so many different ways? by my way of thinking, i don't know how i can trust that kind of reduction not to come with a significant loss in quality. perhaps that is my inexperience.

[budwzr]

There's no way to answer your questions because your test is only on one file. Why don't you run it on 5 different files and report the results? Then you'll be asking why one file had such a small size, and that other one was bigger, and how that doesn't fit the model.

See if you can prove out your hypothesis one way or the other, otherwise, it's just a HeSaid/SheSaid.

[jagabo]

i tried applying the vdub smoother filter @ a mild setting of 5 and the TQ 3.85 video size dropped all the way to 120mb and 670 kbps. how do i know that that video is still the same quality as the other without rendering it so many different ways?

Because Target Quality always delivers the same quality.

i did a 3000 kbps single pass of the 590mb wmv and got a 465mb video. i did a 2pass encode at same bitrate and it was 461mb. the latter took 25+ mins longer. finally, i did a TQ of 3.85 and the rendered video had a bitrate of around 2970 kbps and size 458mb. which of the 3 is better?

The 2-pass VBR and the TQ file should be nearly the same quality. The CBR file should be of lesser quality during action/noisy shots, very slightly better during still shots.

Once again, you use bitrate encoding when you want a specific files size. You use TQ encoding when you want a specific quality.

[Environmentalist]

the TQ still seems to me like just an arbitrary number, especially when you consider that it all depends on the source video. the quality of any video is measured in kbps &or Bits/(Pixel*Frame), not TQ's. there is no section in GSpot or mediainfo for TQ's.

so how are TQ videos encoded? is it a form of vbr like 2pass encodes? if you can specify a bitrate/file size in 2pass encoding, why can't you do so with a TQ encode? why can TQ1 or TQ2 actually produce videos larger than the original? the quality couldn't possibly have increased, could it?

[jagabo]

the TQ still seems to me like just an arbitrary number

And bitrate is just an arbitrary number. Think of the quantizer as how much detail is removed from each frame while encoding. A CBR encode first compresses a frame with a guessed quantizer. If the result too large it uses a larger quantizer. If it's too small it uses a smaller quantizer.

the quality of any video is measured in kbps &or Bits/(Pixel*Frame)

Sure you can measure it that way. It doesn't mean much of anything though. Look at what it reports about the noisy vs. clean videos I posted earlier. By that measure of "quality" the 1400 kbps 2-pass encode should be much higher than the clean 400 kbps TQ encode.

so how are TQ videos encoded? is it a form of vbr like 2pass encodes?

Yes. During a TQ encode the encoder simply encodes every frame with the quality/quantizer you specify. A 2 pass encode first examines the entire file then goes back and uses whatever quantizer (or range of quantizers) is necessary to give the specified average bitrate/size.

if you can specify a bitrate/file size in 2pass encoding, why can't you do so with a TQ encode?

Because the encoder has no idea what's coming later in the video. It simply encodes each frame with the quantizer/quality you specified.

why can TQ1 or TQ2 actually produce videos larger than the original? the quality couldn't possibly have increased, could it?

Quality in this sense is relative to the source -- how closely the new video matches the source. The process of reencoding a compressed video consists of first decompressing the source to uncompressed frames, then compressing those frames with the new encoder. That decompressed video is, in essence, a "new" video that may look something like the original source but has artifacts from the first compression. If you use a very low quantizer the the encoder will be preserving all the artifacts created by the prior compression, resulting in a larger file.