TMPGEnc 2Pass VBR Confusion

[banjazzer]

In regards to the CQ vs Multipass VBR, VBR would be superior, given suitable settings equivalent to CQ, simply because VBR would have advanced knowledge of scene changes, where CQ mode has to use prediction.

You cannot in practice use equivalent settings, because in multipass VBR you set an average bitrate and in CQ you set a quality level, with no idea until the encode finishes what the average bitrate is. CQ mode does not have to use any prediction, as it is not encoding in real time, and it is not restricted to fixed number of bits. It seems people have problems with accepting that CQ is as good, if not better, than 2-pass VBR because of an assumption that an extra exploratory pass must be a good thing, even though with CQ there is no need for a first pass.

Note, I am comparing TMPG's modes - not trying to compare TMPG to CCE.

Although I believe Adam does not agree, the following is an informative reference:

http://tangentsoft.net/video/mpeg/enc-modes.html

[DJRumpy]

Certainly you can. I can use settings, with a certain expectation of quality from those settings. You can set quantization to 1, and expect a resonably close copy of your source video. I can do the same using VBR settings, setting my min, max, and avg to settings suiting my video source. The 'avg' setting isn't some sort of drawback, that they threw into the tools to punish you for choosing VBR. It's a feature that can be leveraged for better quality. No matter how you state it, CQ mode has to guess when it comes to scene changes. Multi-pass does not.

This is a direct quote from your link:

With 1-pass VBR, the encoder doesn't have advance knowledge of what is coming up in the stream, so it has to simply guess. 2-pass VBR lets the encoder make a purely exploratory pass over the video before doing the actual encoding. With 3 and more passes, the encoder can re-encode sections of the video a second time using the data from previous tries. This asymptotically approaches the best possible video stream that the encoder is capable of.

CQ is 1-Pass VBR.

[banjazzer]

It seems you didn't read the reference or my post properly. I said "You cannot in practice use equivalent settings". You choose your bitrate and you hope your experience will give you a good result. If you have difficult material or many scenes with high motion you can get it wrong. CQ can't, because it will take account of this, and use a higher bitrate where necessary, since you have only imited the bitrate by stipulating a maximum and minimum.

CQ is 1-Pass VBR.

"CQ" is not the same as "1-Pass VBR" in the context of this reference or in TMPGEnc, even though it works with one pass and is variable bitrate encoding. Rather than pick something totally out of context as you have I will quote a little more extensively, and encourage others to read properly and fully the reference. I expected a little more of you.

There are four variables you can add to any of the three main VBR styles: you can ask it to keep a minimum, maximum or average bit rate, or you can ask it to keep to a given Q level. All of the modes above imply at least one of these variables. Common combinations are:

1. Simple 1-pass VBR based on average bit rate only.

2. 1-pass VBR with minimum and maximum bit rate caps. This allows for some variances, but it will not allow the bit rate to stray too far.

3. 2-pass VBR with minimum and maximum. Unlike the previous mode, the min and max aren't just caps, they're used to modify the encoder's goal-seeking algorithm: the encoder tries to keep to an average of the two limits, so that it never approaches the limits unless it's absolutely necessary. A 1-pass min/max encoder will sometimes blindly bump up against the limits repeatedly, forcing it to sacrifice quality or use extra bits to force the stream to comply with the user's stated goals.

4. 2-pass VBR with min, max and a fixed Q level. This asks the encoder to try and stick to the given Q level, but when it hits either the upper or lower limits, to adjust the Q level temporarily to stay within bounds. This is useful with things like DVD encoding: the spec says you have to stay over 2 Mbit/s but under 9.8 Mbit/s. This lets you come close to a pure CQ mode without the danger of creating an out-of-spec output stream.

5. Simple CQ mode: the output bit rate is totally unpredictable, but the quality level will stay constant throughout the video.

Note the difference between options #1 and #2 and CQ (options #4 and #5)

[banjazzer]

No matter how you state it, CQ mode has to guess when it comes to scene changes. Multi-pass does not.

This is absolute tosh. When it comes to a scene change CQ encodes with the bitrate necessary, since it can detect the scene change, and does not have to make the best of a pre-determined average bitrate. And providing you don't set a daft quality setting. Why on earth does it have to guess any more than multi-pass?

[Anonymoususer]

Thanks for the advice DJRumpy. Here is the summary of events so far.

To begin with their is NO VAF file in the folder, because I cannot even get CCE to begin encoding, even IF you say that it gets created, well I made sure that their is no VAF file in the folder, by creating a new one.
Here are some of the options that I have tried and had no luck with.



Changed the directory of the output of th VAF file & the avi.
Changed the AVI destination.
Installed GSpot which told me that I HAVE the right codecs for playing the AVI file.
Made sure that their is no VAf file, to begin with it doesn't even get created, double checked anyway.
Checked the AVI for errors, no errors,
Tried using a different AVI with NO ERRORS, which I checked using Virtual Dub Mod, same stupid error.
Unchecked the YUV Settings, still no luck,
Unchecked the Close all GOP's option, still no luck.

Now these are the following options that I have tried out, and still cannot get CCE to start.

Also out of the blue where my created M2V file is supposed to go a Log appears now detailing as to what is happening. Here it is.


***** Cinema Craft Encoder SP started at 2003/02/04 19:15:57 *****
SDK version 2.50 (built at Aug 17 2000 21:02:49)
fexp thread created.
fexp thread created.
biCompression = DIV3
fccHandler = DIV3
--- BITMAPINFOHEADER begin ---
biSize = 40
biWidth = 640
biHeight = 320
biPlanes = 1
biBitCount = 24
biCompression = 0
biSizeImage = 614400
biXPelsPerMeter = 0
biYPelsPerMeter = 0
biClrUsed = 0
biClrImportant = 0
--- BITMAPINFOHEADER end ---
cce created.
encoding canceled.
fexp thread terminated.
***** Cinema Craft Encoder SP shutting down at 2003/02/04 19:17:32 *****


It seems that everything is suppossed to be going good until it reaches the BitmapINfoHeader End [No clue what that is]. So if anyone had some more advice please don't hesitate to tell me.

P.S- Was starting to give up hope, but now I am determined to solve this because I read another post that had the same problem, their was no solution to the problem, I PMED the user haven't gotten a reply back. Now I will help myself as well as mankind.

[Superman Pose]

[banjazzer]

Now these are the following options that I have tried out, and still cannot get CCE to start.

If it's any consolation, I could never ever get CCE 2.5 to work on an AMD K6-2 500 or on an AMD Thunderbird 1200. My advice is to try another machine, or CCE 2.6x. 8)

[Anonymoususer]

Oh really....I have a 450MHZ Processor, with Windows 98SE...It's a 3 year old computer that runs like the day I bought it. Plus I reformatted my hardrive a week ago...DAMN, this is a real dissappointment.

I can't believe I wasted a week on this problem....

[banjazzer]

I really think you should try CCE 2.6x before giving up.

[Anonymoususer]

I tried installing 2.62 and it says that it requires a computer that supports SSE or Enahanced 3d Now....which I guess I don't have either, or know what that is...

I've lost...

Defeat has claimed me....

[banjazzer]

You don't say what your CPU is, except that it is 450 MHz. I suspect if it not an AMD then it may be a Pentium II. SSE is an update to MMX which was introduced with the Pentium III. According to the Doom9 site, the audio encoding in CCE 2.5 also requires SSE. You might try encoding the audio separately if you are not already. If I recall correctly, I could not get CCE 2.5 to run on my Pentium II 400 either.

Apart from that, I think you must bite the bullet, and think about a new PC, because I'm sure this is the root cause of your problems. And 3 years is a long time in PC architecture.

[DJRumpy]

Banj, I guess we'll just have to agree to disagree, and leave it at that. I definately agree it's time for an upgrade for DivXerouS, depending on how old your PC is, it's possible it uses an ATX form factor, meaning an upgrade could be done using the existing case. You could upgrade the motherboard, and processor for under $200.00 dollars. Many of the board manufacturers offer motherboards that do not require the additional power plug for the Pentium IV. Many of the new boards also offer a choice of DDR, or RDRAM, meaning you might be able to keep your existing memory, and just put out for the board/processor combo.

[banjazzer]

Banj, I guess we'll just have to agree to disagree, and leave it at that.

I suppose so. I always resolve never to enter this recurrent topic, but the flesh is weak. Without wishing to continue it ad infinitum, I would like to add I encoded a lot of mpeg2 using TMPG's CQ mode before trying its 2-pass VBR mode. I honestly could tell no difference in quality. The latter was perfectly good, apart from the time. The former was perfectly good, apart from the unpredictable filesize. I must say that I don't spend hours looking at particular sections trying to find faults. Now I use CCE which gives me predictability of size in a short time. And I still can't tell the difference - although I don't tweak the bitrate graph.

And the reason I can't resist contributing to this topic is I figure people come here and read some of the negative things that are said about CQ, and dismiss it out of hand. With a pocket calculator, and a couple of short test encodes a perfect result with a good level of predictability in a reasonable time can be had, which would be particularly useful for those using filters, or with less advanced hardware. And I find it slightly ironic, because a reading of TMPG's online help suggests to me that its creators think CQ mode is equal to, if not better than, its other modes.

[ZippyP.]

You can cut your 2-pass VBR encoding time in half by enabling the Cache setting under Environment. After the first pass, all data is pulled from cache instead of analysing again.

Thanks for the tip, I've been using 2-pass for several months and didn't know about it 'til now. I think this should be included in the guides whenever 2-pass is mentioned.

[adam]

DJRumpy: I believe bitrate is distributed per GOP. ie: this GOP needs this amount more than this GOP, and within this GOP this frame needs this amount more bitrate than this frame etc..

As far as the averaging, I think its just matter of fact. The encoder starts with your avg and raises or lowers bitrate as necessary, but it won't add what it doesnt also take away from somewhere else, and vice versa. So there is really nothing to calculate, its not like it constantly checks itself saying "a/b must equal c by the time I reach point d, it just works out that way. Basically its like a well balanced checkbook, it doesn't spend what it doesn't have, and since it scans the entire file before encoding, it never runs into any deficits or surpluses.

[Anonymoususer]

Thanks for the info, and yes I have been meaning to get an upgrade for a while now, but don't have the bread right now. Well anyway, before I move on and leave this terrible ordeal behind me, I would like to thank everyone that took the time to post their suggestions, tips, etc, everything was very useful and very informative and I learned a lot this whole week of postsings, and hopefully this could benefit some more people in the future who might be having the same problem.

Thanks again to all the people and moderators for their help, keep up the good work !

Best Regrads to everyone for the help from,

DivXerouS

[banjazzer]

Please don't leave without telling us what type of CPU you have!

[Anonymoususer]

CPU Authentic AMD Processor
450 MHZ
20 GIG Hardrive
Windows 98 SE
56.0 MB Ram
AMD K-6 3D Processor
Vitual Memory = 32 Bit
Manufacturer = Compaq

oh by the way I can't play any high quality 3D Enahnced games like, Hitman, Need for Speed hot pursuit etc.


Hope it helps.

[Betamax]

Too much text to read all through, but what I read in the first posts of this topic I'm guessing what actually was asked : how to get best results with 2-pass and still have no more than the entire disk used.

Well, you must have an AVI that's no more than 55minutes long to still have some quality.
Then you should use the bitrate calculator which is somewhere in the tools-section of this site (use the calculator of vcdhelp/dvdrhelp - easiest).

The result of this calculator, you put in the 'average' field of the 2-pass settings. Leave all other settings 'as is' and don't enable the 'padd'-option.
Then do the encoding (which might take several hours). You'll endup with a MPeG which exactly fits onto a CD-R (size as which you entered in the calculator : 74min. or 80min. or higher).

[RabidDog]

NOT wanting to stoke the fires but

A low avg does have a detrimental effect on VBR encoding but this happens regardless of what mode you encode in. With a low avg, you have a much smaller interval from which the encoder can gather bitrate. If your avg is 500kbits the encoder is not going to be able to go much lower than 300kbits on any given scene, so your not going to be able to go much higher than that either

That doesnt make sense... in vbr you could have some nice big spikes in bitrate far greater than say 200kbs, but rightly if these bits are used in hi action scenes then theyare being taken away from somewhere else!
basically I am saying just because you set the min and avg very low it doesnt mean that the encoder wont (or cant ) use a high max, eg 3000kbs . . I am presuming that all the bits are spread across the whole movie (not averaged across single GOPS
example 200 min 600 avg 3500 max. But thats just my understanding

[adam]

Sure it makes sense. You only go above your avg what you first take away somewhere else. I'm not saying that, since you can only go 200k less you can only go 200k more. I'm saying that all the bitrate saved on flat scenes, those requiring less than your avg, will be added up and spread out over all the complex scenes, those that require more than your avgerage. So you will still have bitrate spikes, but there is far less bitrate to go around. You will save 50kbits here, 200kbits there, and add all these up and you will be able to allocate maybe 2mbits during one scene, but there are only so many scenes that will even be watchable below 500kbits, that there just isn't enough bitrate saved up to do anything with.

An encoder is always going to try to get an even level of quality. Left to its own devices, an encoder will almost never allocate less than 500kbits because the quality quickly goes to hell. So with an avg near 500kbits, the encoder is not going to be able to take much more than that away. Its not going to let most scenes go to hell just for the sake of some, instead it will just let all scenes moderately suffer.

The proof is in the practice. Take your example and decrease the min all the way to 0 and essentially give yourself an unlimited max; 9mbits. So you got 0, 600, 9000. You will see that your bitrate spikes will probably not be higher than 2mbits and overall your bitrate curve will be relatively flat. Its really just simple math, the smaller the interval the less variation.

[DJRumpy]

Adam, using a low average that is relatively close to your minimum brings the entire quantization scale down to try to meet your settings? Is the inverse is also true, using a high average, relatively close to your maximum, will it give you very little variance in your bitrate again, and an MPEG that has more simularities to CBR due to the small variance in the bitrate?

I'm trying to get a grasp on the mechanics. If I'm understanding what your saying, the encoder will vary by the smallest difference between the AVG, and either the Min or Max? Take this example, using:

MIN: 300
AVG: 2000
MAX: 9000

Is it an average of all three values?
( 300 + 2000 + 9000 ) / 3 = 3766 Kb/s variance from AVG
or
Avg: 2000 ( +/- 3766 ) (BUT: honoring MIN/MAX values)

Obviously we cannot go into the negative here, or below MIN, so it would effectively 'cap' what it could borrow from MIN to give to AVG at 1700 (max difference between 2000-300)?

This sounds like it applies here, and if that is the case, an average closest to the exact AVG of ( Min + Max / 2) would give you the best 'bitrange' for lack of a better word. In the example above, this setting would give you good range in the low bitrate area, but very little swing into the high bitrange.

If that's the case (lot's of assumptions here), then CQ Mode could simply be taking your Min and Max values, using an average of both values as the AVG setting internally, and motion prediction, instead of multipass data? This would also explain why Multipass VBR exists, and Multipass CQ doesn't.

[DJRumpy]

Adam, the initial avg,max,min values used for the first pass must affect the results stored in the VAF file. I was looking over the docs for CCE. They suggest that if you change any of these values significantly, that you should discard your VAF file, and create a new one. Can you think of any reason as to why?

[Coop]

DJR - I'm curious,

Have you encoding MPEG for DVD and VCD with CCE or TMPGenc? Given the higher quality output that DVD can handle, I'm wondering how your conversion and authoring have turned out.

I'm on my third project now, and using TMPG 2 pass vbr with optomized settings. I used CBR for the first project via the wizard, and noticed a few scenes with slight blockiness, I wasn't aware of how much better VBR encodes at the time.

Also - do you convert audio and video seperately or do you leave them combines?

[aaronkow123]

You can cut your 2-pass VBR encoding time in half by enabling the Cache setting under Environment. After the first pass, all data is pulled from cache instead of analysing again.

Thanks for the tip, I've been using 2-pass for several months and didn't know about it 'til now. I think this should be included in the guides whenever 2-pass is mentioned.

Any suggestions on what would be a good setting? How many mb???
thanks

[ZippyP.]

Stonemann said:

C:\Documents and Settings\**YOURLOGINHERE**\Local Settings\Temp

is the location of the temporary file created and used during a 2 pass encode when the cache feature is enabled.

For VCD/SVCD/XVCD/XSVCD setting 4096 ( 4 gig ) for your cache file , and ensuring that you have that much space actualy free , will carry up to a 2 hour encode very safely. Typicaly landing around the 3.5 mark at the finish, so there is extra playroom left. Hour encodes typicaly take 1.7 or so gigabyte. If you are doing 3 hour encodes in one section encodes then I could possibly see bumping it up to 7 or 8 gig.

Using the newer 2-pass mode, with the data from the 1st pass cached, I am more than happy again using tmpgenc exclusively. As well as exclusively using 2-pass mode again as well. Speed and quality both are more than acceptable to me when for a while they were not. But hey , maybe that's just me ....

Hope this info helps someone.

This was lifted from another thread, sounds like he actually tested for file size.

[DJRumpy]

Coop, I use CCE. The quality is excellent, for both VCD, and DVD. Make sure if you use CCE, that you set your Quantization Matrix settings according to the format your creating. For VCD, use the 'Ultra Low Bitrate' setting. For SVCD, use the 'Low Bitrate Setting'. For DVD, use the Standard, or 'MPEG Standard'. I believe the MPEG Standard setting is for high motion mpegs, like Star Wars for example. The 'Smooth' setting is for computer generated, or animation type films.

If I'm in a hurry, I'll use 1-Pass VBR, with the Quantization scale set to 1 for best quality. Otherwise, I use 2-Pass VBR. After the research I've done with this one, I've started changing my VBR settings, so my AVG is exactly in between my min and my max setting. The quality seems to be excellent. I was expecting a slight drop in quality, but the encoder seems to have no problem

For CVD/SVCD I use min:300, avg:1412,max:2524.
For VCD, which I rarely ever encode in, I use CBR: 1150
For DVD, I use min:300, Avg:5000, Max 9000

For any typical project, the first thing I do is extract the audio. I usually leave it out until I'm ready to author. The only exception is when I convert an SVCD to DVD. They are usually split into multiple parts, requiring fine splicing to get rid of the common 2 second overlap between disks. Audio sync issues are nearly impossible to avoid, if you don't mux your audio back in before splicing them together. Almost all of my projects are straight to DVD now, so I usually just extract the audio, and drop it right into the authoring software, or convert it to MP2/AC3 if necessary.

[C53248]

DJRumpy,

For CVD/SVCD I use min:300, avg:1412,max:2524.
For VCD, which I rarely ever encode in, I use CBR: 1150
For DVD, I use min:300, Avg:5000, Max 9000

I usually use 2-pass so that I can control the resulting file size. If you don't adjust the average bitrate setting according to each video's length, why use 2-pass rather than a single pass constant quality setting?

[DJRumpy]

Because the average doesn't seem to have much bearing on the final size, although the max does. I do not have a problem fitting an 1 1/2 hours of video on DVD, so I don't need to mess with the avg setting. The only time I ever decrease either it, or the max, is when I have a 2+ hour movie, where space becomes more of a problem. You cannot accurately predict the final size of a VBR file. You can come close using an average, but again, the average setting seems to be for first pass reference only.

MPEGs encoded with various avg settings and then run through bitrate viewer still gives me an mpeg with an average bitrate according the the video sources complexity rather than my setting for avg. I've verfied this with CCE, not with TMPGenc (I don't have the patience for that).

That said, I use multi-pass, when a movie's size is borderline, possibly to large to fit easily on a DVD, or if the movie contains complex scenes to encode (eg: motion/fades). The additional passes provide better qualtiy than CQ, and a smaller file for each additional pass.

[C53248]

DJRumpy,

This forum topic is well named. That is just not correct. It is meaningless to talk about an average without having a certain file size. The average bitrate correlates directly to the final file size:

(Number of seconds) x (Average Kilobits per second) = Number of Kilobits

The number of kilobits produced is your file size.

Likewise:
Average Kilobits per second = (Number of Kilobits) / (Number of seconds)

E.g.:
60 seconds x avg. 1000 kilobits per second = 60000 total kilobits.

(60000 kilobits = 7.32 KB file)

or...
5 hours x avg. speed of 60 miles per hour = 300 miles traveled.

Try it out. This is 2-pass VBR's reason for being, to give you the bitrate allocation of VBR with as much control as you have with CBR. The maximum and minimum values are just boundaries for the encoder not to cross. The max. and min. values are primarily to keep the video within limits compatible with your player, e.g. no 12000 Kbps scenes. Secondarily they may protect you from the imperfections of the encoding algorithm, if the encoder estimates that it needs less or more bitrate for a scene than it really does. In the second case, you are second guessing the encoder and risk imposing your own ham-handed imperfection on it, so it's best to keep the minimum low and the maximum high. My observation is that it mostly doesn't stray too far anyway, especially towards the low end. There is no magic about the max. and min. being equally far from the average value. However, if either one of them is too close to the average value, you will get VBR that approximates CBR, because it has no room to vary.

My goal is usually to make use of the full capacity of the disc. I divide the size of disc in kilobits by the length of the movie in seconds to determine the average bitrate needed to do this. With this method I always fill the disc, and the file always fits. It's very predictable.

Many people think that 2-pass VBR is a process whereby the video is encoded in the first pass and then polished with successive passes. That is a misconception. What is improved in successive passes is the approximation of CQ bitrate allocation. The first pass estimates a CQ level and tries it out to see what the average bitrate comes out to be at that level. On the second pass it then extrapolates or interpolates the values of the first pass to determine what values to use to achieve the chosen average bitrate. That extrapolation or interpolation is an imperfect technique, because the appropriate variation for a lower quality level is different than for a higher quality level. With more than two passes, the additional passes are like the first pass, but they use the previous pass to better guess an appropriate CQ level to test and use as a basis.

If you are not controlling file size and just want to achieve a certain level of quality, by all means choose your percent of quality and use CQ. You will then have your bitrate allocated exactly right for the bitrate used rather than approximately. This is why CQ should look better than multi-pass if the file sizes are the same. If file size is really no limit, use CBR and set it to the maximum DVD bitrate of 9800 Kbps.

[DEmberton]

I've always done 2 pass VBR, figured the better results are worth waiting for. Question though: is there any point in setting the minimum data rate? I've always left it as zero.

Dave