!Maximum! quality Xvid settings, no compromises.

[The.Yield]

The title says it all.

I've been wading through video codec jargon for 3 days now and I cannot seem to find laymen info on 90% of Xvids settings.

So...

Off a straight VOB opened in VirtualDubMod. Cropped and resized to 640x360 (the resolution I wanna use). Using native frame rate (and 24-bit color).

Using the latest complied Xvid codec.

What settings should be used for MAXIMUM quality?

I don't care about filesize, I don't care about effient use of bitrate, etc.

Just want as close to full bleed quality as I can get.

Thanks in advance.

[greymalkin]

use CQ setting of 1. That's constant quality at the best quality.

[The.Yield]

What about some of the other advanced setting?

One thing that has really been annoying me is a thrashing of quality in high action scenes (particularly in Anime material).

Will Target Quan' 1 remedy that all by itself?

[greymalkin]

can you post an example screenshot of the issue you are experiencing? I would try a minute or so of an action scene at cq1 and see if you are pleased with it. From what I have heard that is all that is really needed when encoding to xvid. If you are watching it on your computer monitor and it is an interlaced source make sure you de-interlace it as well.

[KBeee]

If you don't want to lose any quality at all, and there's no problem with file size, why are you converting?

[The.Yield]

Video is going on a Cowon iAudio A2. Has to be a Divx or Xvid.

[manono]

You've already lost a ton of quality by opening the vob in VDubMod. No IVTC. That probably explains the "thrashing of quality in high action scenes" that you mentioned. You've further screwed it up by cropping and resizing in VDubMod rather than AviSynth.

My suggestion is to use AutoGK with Target Quality set for 100%. For nearly as good a quality and a smaller size, set it for 75%. It'll turn out way better than what you're doing.

[The.Yield]

Interesting, I'll have to try that.

Note I just found out that the Cowon A2 cannot play Qpel or GMC encoded material, whatever that means.

Can someone explain that?

EDIT ^2: I've found AudoGK to suck pretty badly. I am getting MUCH better results in VirtualDubMod directly (Goldwave for sound).

Anyone have any insight into the advanced Xvid settings?

[manono]

Suck pretty badly? You have a log for that sucky encode? Did you take my advice and set it up for a 1-pass Target Quality as I suggested? AutoGK doesn't use Q-PEL or GMC (XviD's 3 warppoint GMC anyway).

You might should install it with the ESS chipset option enabled. Some standalones require that. If you made an AVI, and it plays in that player of yours with blotchy, blocky, weird colors and smearing, then that's the reason.

[The.Yield]

Suck pretty badly? You have a log for that sucky encode? Did you take my advice and set it up for a 1-pass Target Quality as I suggested? AutoGK doesn't use Q-PEL or GMC (XviD's 3 warppoint GMC anyway).

You might should install it with the ESS chipset option enabled. Some standalones require that. If you made an AVI, and it plays in that player of yours with blotchy, blocky, weird colors and smearing, then that's the reason.

Yes I did that and yes that what it looked like when played. I'll give that ESS option a go when I get home from work.

But in general I didn't like how I had no control over things (black boarders, I found, were automatically cropped but not accurately). Also, while I could tell AutoGK to use a certain width I could not tell it to use a certain hight. What resulted was a non-perfect aspect ratio (I wanted 640x360 and got 640x352ish).

P.S. Your original comments about VdubMod ruining quality seem baseless. AutoGK actually used VdubMod to do the video rendering lol (or so it appeared). And why would I need inverse telecine? I want the video at 23.97 framerate.

[jagabo]

You're quest for the best possible quality Xvid file is probably ill conceived. In all likelihood your Cowan A2 will not play it. But here's some settings to try:

As mentioned earlier, encode with Single Pass Target Quantizer mode at Q=1. And don't use B-VOP's. You'll get a very large file that probably won't play on your portable player. Dropping down to Q=2 might play on your player and will hardly degrade the picture. You might have to go as low as Q=3 but at that there will be noticeable macroblocking if you look at enlarged still frames. You probably won't notice it on the small screen.

Don't enable the Quarter Pixel (QPEL) and Global Motion Compensation (GMC) options. Use the MPEG quantization type.



There's no problem using VirtualDubMod on a 23.976 fps VOB file. It doesn't need to IVTC the video because it's already stored as progressive frames in the VOB file. VirtualDubMod does not perform the pulldown that VirtualDubMPEG2 or your DVD player would.

If your video was 29.97 fps hard telelcined you would want to IVTC back to the original 23.976 film frames. But that is true whether you use AVISynth, VirtualDubMPEG2 or VirtualDubMod.

AVISynth will give very slightly better and faster results because it can work within the native YUV 4:2:0 colorspace of your source. VirtualDubMod has to convert everything to RGB if you use any filtering.

[manono]

A little knowledge is a dangerous thing.

What resulted was a non-perfect aspect ratio (I wanted 640x360 and got 640x352ish).

You got a perfect aspect ratio. AutoGK resizes by Mod16, as should you. it doesn't let you choose the height so you don't screw up and wind up with bad AR. Are you saying that it left behind some of the black border? If so, then that shouldn't happen, but the crop can be tweaked in the Hidden Options, and the autocrop can also be tweaked to look at more frames, to make sure all the black is cropped away.. It's all explained in the included tutorial. If it's a movie with an original framerate of 23.976fps, then it either uses Force Film in DGIndex, or IVTC's it to return the movie to 23.976fps. That's what you got, isn't it?

Your original comments about VdubMod ruining quality seem baseless. AutoGK actually used VdubMod to do the video rendering lol (or so it appeared). And why would I need inverse telecine? I want the video at 23.97 framerate.

AutoGK uses AviSynth for frameserving (yes, to VDubMod, but not the way you did it), as should you. It stays in the YV12 colorspace, as should you, but by using VDubMod's crop and resize filters, you're forced to use the much slower Full Processing, and convert to RGB and then back again, thus degrading the quality.

However, you were part right about one thing. I forgot that VDubMod ignores pulldown flags, and if the DVD is film, it opens at and encodes it at 23.976fps. There are plenty of films on DVD where it'll be opened at 29.97fps. IVTC will then be required though, and VDubMod's isn't so good. In addition, there are also plenty of DVDs that are mostly film (anime is a good example), that may be opened at and encoded at 23.976fps, but will then leave you with interlacing and jerky playback in many places. That kind should also be IVTC'd. I apologize for making that blanket statement, when your DVD could very well be encoded properly at 23.976fps. But your statement about it looking bad in motion scenes and about it being anime, which is notorious for switching back and forth between film and video, requiring IVTC, is what lead me to believe that perhaps IVTC was needed in your case.

I noticed that while writing, jagabo said some of the same things.

[The.Yield]

Thanks jagabo, I'll give those settings a shot.

FYI I fixed the quality thrashing in high action by setting the Reaction Delay Factor from 16 to 1.

Those of you who are interested, (I know no one is) the piece of video in question is from the animated movie Nausicaa: of the Valley of the Wind. The part in the begining where the giant Oam busts out of the woods sending logs and such flying.

On my most recent rendering, it's flawless.

Again, jagabo, I'll try those settings you suggest.

P.S. I SOOOO want to use Avisynth but that is very beyond me.

Manono, I get what you're saying about the AS. But I need perfect 1.77 AS for my player. And 640x352 is not perfect 1.77 =/

[jagabo]

I don't think Reaction Delay factor is used in Target Quantizer mode. It think you were using Constant Bitrate mode.

[manono]

Oh, Nausicaa. Yes, that's film, 23.976fps. Not like many of the anime TV series. I like your taste in movies. But to get the best out of them, you should learn better methods.

And I don't understand the importance of 640x360. Lemme guess. You have a widescreen TV set and you like it to fill the whole thing. Even if it gives bad AR. Now, it's anime, and the bad AR won't be noticeable. Actually, most DVD/MPEG-4 players add black bars to keep AR, which for this AR will be hidden by the overscan. But the ESS chipset players just stretch it to fill the screen, at least the ones I'm familiar with, like the Philips DVP-642. IMDB says the movie is 1.85:1. But a careful analysis of both the R1 US DVD of it, and the R2 Japanese DVD of it, say it's 1.82:1. And what's 640x352? 1.82:1:

http://www.dvdbeaver.com/film/DVDCompare9/nausica.htm

[jagabo]

Maybe he should convert to 480x272:

http://www.pcmag.com/article2/0,1759,1902278,00.asp?kc=PCRSS02129TX1K0000530

[manono]

Nice find jagabo. At first I thought it might be some kind of a portable since I'd never heard of it, but when he described the video problems after coming out of AutoGK, I decided it must be a standalone unit. Not sure why now. Expensive, eh?

[lordsmurf]

XVID + MAX QUALITY = DOES NOT COMPUTE

The nature of the format "is" compromises. Although, you can get quite decent results for web content. But don't expect it to be "as good as" an MPEG-2 file on DVD, nor the source for a DVD. It will simply be a nice quality web format, nothing more.

[The.Yield]

Ya hehe, it's a great little unit, worth every penny (Got mine for $410 with free 2day shipping).

The reason I am not encoding directly into it's native resolution is that I want to be able to get the best quality from the units Video out mode. The Cowon a2 has a maximum output resolution of 640x480. So you can see the compromise i'm making. Plus, i've tested the unit with material that is 480x272 and 640x360, it is nearly impossible to tell the difference when viewing on it's LCD. But makes a world of difference when piping it out to a TV (as i'm sure you could imagine).

And yes manono, I use the perfect 1.77 to light every pixel with image on the a2. When screen size is only 4 inches you really notice those "true" AS sacrifices.

And doing what i'm doing really isn't a bad thing, considering that when a DVD is encoded, the black bars are not part of pictures true AS; they are added lines (as I understand it). So when you go in and crop said lines precisely, and then force the resulting image into a proper resolution. The difference the image will have from it's true AS will be negligible/impercievable (at least to me).

P.S. Yes Jagabo, for that particular encoding I was using constant bitrate. I'm toying around with your settings now.

[guns1inger]

And doing what i'm doing really isn't a bad thing, considering that when a DVD is encoded, the black bars are not part of pictures true AS; they are added lines (as I understand it). So when you go in and crop said lines precisely, and then force the resulting image into a proper resolution. The difference the image will have from it's true AS will be negligible/impercievable (at least to me).

Incorrect, I'm afraid. Leaving letterboxed 4:3 out of the discussion for the moment, any 16:9 enhanced material on DVD that has an aspect ration greater than 1.78 (so 1.85:1 - 2.40:1 or wider) has black bars embedded in the video. It must have to fill the frame out to the full resolution - a fixed value. Once you get past 1.85 material, you will see definate stretching of the image from the distorted AR, unless you start to zoom in and crop off the ends.

[The.Yield]

And doing what i'm doing really isn't a bad thing, considering that when a DVD is encoded, the black bars are not part of pictures true AS; they are added lines (as I understand it). So when you go in and crop said lines precisely, and then force the resulting image into a proper resolution. The difference the image will have from it's true AS will be negligible/impercievable (at least to me).

Incorrect, I'm afraid. Leaving letterboxed 4:3 out of the discussion for the moment, any 16:9 enhanced material on DVD that has an aspect ration greater than 1.78 (so 1.85:1 - 2.40:1 or wider) has black bars embedded in the video. It must have to fill the frame out to the full resolution - a fixed value. Once you get past 1.85 material, you will see definate stretching of the image from the distorted AR, unless you start to zoom in and crop off the ends.

Yes, when you get wider than natural 1.77 AS... normally.

In my situation however, the player will simply touch the video to the frame left+ right and add enough padding to maintain correct AS of the source.

Example: I just put Wedding Crashers on my a2, which is 2.35 AS. Obviously I cannot fill the entire screen with picture in that instance. I'm not one of those people who will cut the sides off an extra wide picture to make it fill a 1.77 screen lol. I'll never be one of THOSE people!

[jagabo]

The.Yield,

At very high quality Xvid settings you may find that your AVI file turns out larger than the the source VOB.

For example, I took a 90 second, 16:9, movie trailer VOB file and compressed to Xvid AVI. It's a bit of an unusual case since, as a trailer, it's full of very short cuts and lots of action that doesn't compress well. The VOB has an average video bitrate of about 4000 kbps, a file size of 50 MB. After resizing to 640x360 (lanczos3) in VirtualDubMod I compressed to Xvid in Target Quantizer mode and got the following:

VOB --> 50 MB
Q=1 --> 93 MB
Q=2 --> 40 MB
Q=3 --> 25 MB
Q=4 --> 20 MB

About 4 MB of each of those was the AC3 audio.

So "compressing" the video with Xvid at Q=1 was a waste of time. The result is nearly twice the size of the source VOB.

[The.Yield]

The.Yield,

At very high quality Xvid settings you may find that your AVI file turns out larger than the the source VOB.

For example, I took a 90 second, 16:9, movie trailer VOB file and compressed to Xvid AVI. It's a bit of an unusual case since, as a trailer, it's full of very short cuts and lots of action that doesn't compress well. The VOB has an average video bitrate of about 4000 kbps, a file size of 50 MB. After resizing to 640x360 (lanczos3) in VirtualDubMod I compressed to Xvid in Target Quantizer mode and got the following:

VOB --> 50 MB
Q=1 --> 93 MB
Q=2 --> 40 MB
Q=3 --> 25 MB
Q=4 --> 20 MB

About 4 MB of each of those was the AC3 audio.

So "compressing" the video with Xvid at Q=1 was a waste of time. The result is nearly twice the size of the source VOB.

Yes, that's what I've noticed as well. I had a 2 hour movie come out to something like 5.66GB lol.

Sadly, I've run into another issue. I think a2 has a maximum file size limitation. It would not let my transfer the 5.66GB file to unit (gave me a not enough space on disk error, dispite having something like 20GB free).

I'm researching the issue now.

Interesting note. When recording video with the a2, you are limitied to a file size of 1.8GB max. My line of thinking is that this is the limit for ANY individual file on the a2. Thought, if that were the case, I'm not sure how the a2 would know to stop me from putting it on the unit and give me a lack-o-space error (when simply using windows explorer to paste the file in).

Anyway, I'm looking into it.

EDIT: It's a limitation of the FAT32 File system. Only allows for 4GB files heh. So I'll try at Q2 (can I use a fraction? Like 1.5? I'm at work and cant check).

[mats.hogberg]

Surely, it has its HDD (or whatever it uses for storage) FAT32 formatted. You'd get the same effect trying to copy that file from a NTFS drive to a FAT32 drive on the same computer.

/Mats

[jagabo]

Yes, you can use decimal values. VirtualDubMod will alternate between the nearest two integers to get the average you specify. For example 2.5 will result in half the frames being compressed at 2 an half at 3.

[The.Yield]

Yes, you can use decimal values. VirtualDubMod will alternate between the nearest two integers to get the average you specify. For example 2.5 will result in half the frames being compressed at 2 an half at 3.

So far I've been getting some very good results. Gotta get more testing in.

One thing I've noticed is sort of annoying however. With certain material that has very bright neon colors (like The Penguin Caper from Madagascar). The boarders of those colors, when played on the LCD of the a2 (and PSP, and Ipod Video), become very blocky and distorted. I think it to be a limitation of the screens as I see no such problem when viewing on the PC.

Aside from that, cartoon/anime material is giving me a lot of color stepping or breaking down of gradients. Interestingly enough, the a2 has a dithering option which seems to help a little bit.

Is there anything I can do to help reduce the color stepping?

[jagabo]

One thing I've noticed is sort of annoying however. With certain material that has very bright neon colors (like The Penguin Caper from Madagascar). The boarders of those colors, when played on the LCD of the a2 (and PSP, and Ipod Video), become very blocky and distorted. I think it to be a limitation of the screens as I see no such problem when viewing on the PC.

That's probably happening because the player is resizing the video for the display. The display is only 480x272. Try encoding at that size. Use the Lanczos3 filter in VirtualDub.

Aside from that, cartoon/anime material is giving me a lot of color stepping or breaking down of gradients. Interestingly enough, the a2 has a dithering option which seems to help a little bit. Is there anything I can do to help reduce the color stepping?

Do you see that on the computer as well as on the A2? If it's only visible on the A2 it's probably because the LCD isn't capable of full 24 bit color. 12 or 15 bit color isn't unusual for cheap LCDs Dithering is one method of hiding that problem -- as you've found.

You can add noise (eg. MSU Noise Generation filter) to the video with VirtualDub but that will make it much less compressible.

[The.Yield]

As to the messed up ultra-bright colors, it doesn't seem to matter what resolution it's at. I've encoded stuff at native and of course much larger, never makes a difference when viewed on the player(s). Looks fine on the PC.

The color stepping is the far more annoying issue. It is fairly viewable (the stepping) on the PC and gets far worse on the player. Namely things like sky scenes (and other large flat planes of color). The dithering does help, but not as much as I'd like.

Would it have anything to do with Vdub coverting the color to RGB?

P.S. The player definately shows 24-Bit color, or at least it claims to.

If I take stills of the video on the player (with it's screenshot function) it's hard to see the stepping. But when the video is in playback, sweet lord lol.

[jagabo]

If it looks worse on the A2 than on the computer then the A2 probably doesn't have a full 24 bit display.

Are you doing much filtering? Some filters can result in banding. Are you using Target Quantizer mode? What Quantizer value are you using? Are you using B-VOPs? B-VOPs are encoded with a larger Quantizer value (lower quality) -- that's why I recommended against them earlier.

Use VirtualDubMod to export a problematic frame and post it here.

[The.Yield]

If it looks worse on the A2 than on the computer then the A2 probably doesn't have a full 24 bit display.

Are you doing much filtering? Some filters can result in banding. Are you using Target Quantizer mode? What Quantizer value are you using? Are you using B-VOPs? B-VOPs are encoded with a larger Quantizer value (lower quality) -- that's why I recommended against them earlier.

Use VirtualDubMod to export a problematic frame and post it here.

Now that I think about it, I don't think I encoded the material I'm talking about in Target Quantizer (think I did constant bitrate). I'll have to test TQ overnight and see how it comes out.