Getting larger files than expected using Xvid to compress AVI

[jagabo]

Okay, I just did the tests, and my Sony LCD monitor displays all of the different white and black blocks. However, I can clearly see the black and white lines inside the 50% grey box at the end of the test on http://www.imaging-resource.com/ARTS/MONCAL/CALIBRATE.HTM

I have a hard time believing that this should look like a uniformly grey box. Am I misinterpreting something?

Get far away from the monitor so that you can't see the horizontal lines any more -- maybe 10 feet. Then compare the brightness of the inner box to the outer box. They should be the same brightness. If not, adjust the gamma setting in your Desktop brightness/contrast controls.

[pronco]

Okay, I'm beginning to see what they mean. I was not using native resolution before, but now I am. I also was not as far away as you suggested. I see the colors beginning to blend, but the center square is still darker than the surrounding grey. So I opened up the adapter settings and tried to adjust the gamma, but it was already set as low as it could go by default. I changed my monitor's hardware display settings (Picture Adjust) from "high" to "medium" and I can get it close, but not match exactly. I tried moving brightness and contrast but that had little effect, so I am leaving it at the default settings for the adapter.

Moving on... I just did the clock/phase calibration test on the Lagom page but it's not giving me instructions as to how the screen should look. At native resolution of 1280x1024, I'm getting a LOT of perceived movement on the screen. My display adapter will only display at 16-bit or 32-bit. I am currently set to 32-bit. However, the specs for this monitor are 16.7 million colors, or 24-bit. Not sure if this is going to be a problem.

Clearly this starts to get a lot more complicated. If this is going to turn into a whole new thread, e.g. if I have to forgo this aspect of my video source being accurate, I will...

[jagabo]

Okay, I'm beginning to see what they mean. I was not using native resolution before, but now I am. I also was not as far away as you suggested. I see the colors beginning to blend, but the center square is still darker than the surrounding grey. So I opened up the adapter settings and tried to adjust the gamma, but it was already set as low as it could go by default. I changed my monitor's hardware display settings (Picture Adjust) from "high" to "medium" and I can get it close, but not match exactly. I tried moving brightness and contrast but that had little effect, so I am leaving it at the default settings for the adapter.

It sounds like you are close enough for non-professional use.

Moving on... I just did the clock/phase calibration test on the Lagom page but it's not giving me instructions as to how the screen should look. At native resolution of 1280x1024, I'm getting a LOT of perceived movement on the screen. My display adapter will only display at 16-bit or 32-bit. I am currently set to 32-bit. However, the specs for this monitor are 16.7 million colors, or 24-bit. Not sure if this is going to be a problem.

That's not an issue. 32 bits is the same as 24 bits (8 bits each of red, green, and blue) plus a dummy 8 bit alpha channel.

You don't need to worry about the clock and phase calibration tests and most of the others. Those don't really come into play with video. The important thing is to make sure the black level and white level are good (especially make sure the blackest blacks are as black as your monitor can get) and that you can see all or almost all of the color patches (the very darkest blues may be a little hard to make out) and that the colors don't appear to change across the bars -- ie, they just look like different shades of the same color.

Now view the DV AVI file I linked to earlier and make sure it looks like the top image on that post. Adjust the video proc amp settings (not the Desktop proc amp settings) to match.

[pronco]

Adjust the video proc amp settings (not the Desktop proc amp settings) to match.

Sorry to sound like a total noob, but what are these, and how do I access them? I don't even know what "video proc amp" is an abbreviation for. In Windows XP Professional, for the tests you mentioned, I right-clicked on Desktop, went to Graphics Options -> Graphics Properties, and clicked the "Color" tab. I think this may be the "desktop proc amp" that you speak of. But, not sure.

[jagabo]

Adjust the video proc amp settings (not the Desktop proc amp settings) to match.

Sorry to sound like a total noob, but what are these, and how do I access them? I don't even know what "video proc amp" is an abbreviation for. In Windows XP Professional, for the tests you mentioned, I right-clicked on Desktop, went to Graphics Options -> Graphics Properties, and clicked the "Color" tab. I think this may be the "desktop proc amp" that you speak of. But, not sure.

Video Processing Amp -- brightness, contrast, hue, saturation, gamma, sharpness, edge enhancement, noise reduction... Your graphic card's setup should have separate controls for the Desktop and video. For example, the computer I'm on right now has Intel graphics:

Desktop proc amp:


Video proc amp:


Not all graphics cards have all the features I listed. Also be aware that media player may have their own proc amp settings. And even codecs can have proc amp settings. So be sure you've checked them all.

Windows Media Player video proc amp dialog:

[pronco]

Alright, I appear to have just discovered why my blacks are so off. I think it's a hardware limitation of the monitor I am working on, a Gateway VX1120 22" flatscreen CRT. This monitor cannot distinguish ANY of the black squares in the imaging-resource website test. All of them display as just one color-- solid black. My brightness and contrast are turned up as high as the monitor will allow, and I still cannot distinguish any of them. I can make out all the lights, no problem. But absolutely no ability to display any of the darker blacks. I downloaded a free tool, Calibrize, and it was of no help either. What a pain in the

[pronco]

I'm using an ASUS V9520 Video Suite graphic card. I can't find anything on my computer that will display anything similar to what you've uploaded. Even going into display properties only pulls up generic Windows dialog boxes. Perhaps there is an applet that came with this card that I haven't installed. I will continue looking.

[pronco]

Okay, I found the Windows Media Player video proc amp dialog... not particularly easy to find. However, when my source video is loaded, I cannot change any of the settings. They just keep bouncing back to the center of the dial. It does this whether the video is playing or paused. I have no idea why this would be.

Sorry for all the responses. I'm having to look into a lot of things at once with this one. But I think we've definitely discovered a serious problem with my blacks, so that is progress. If push comes to shove, I'll just pull the hard drive with the source video file and install it into the machine that has the better monitor. I can't switch the monitors unfortunately... it's physically unfeasible.

What a drag. I can't imagine the V9520 doesn't have a color control applet. As far as I know at this point in my investigation, it's the color settings on the graphic card that are causing my problem, and not the monitor.

[pronco]

Alright, getting back to the other stuff... I have compressed the audio by first converting the 96000 PCM stream to 44100, and then compressing it in full processing mode to MP3 192 stereo 44100 in Virtualdub. I have also IVTC'd with the Vdub native filter. Compression ratio with XVid is MUCH better now... Vdub is estimating around 2.8 GB for the final product as opposed to 4.7 GB. The combing artifacts are gone; there is some slight blurring but nothing too bad. I will try to get better results using the avisynth IVTC filter as well as the x264 compression codec at the lower bitrates suggested by DB83... but what I am seeing now is promising. THANKS EVERYONE. It's starting to look like the last major hurdle for optimizing this project's quality is the blacks issue with my monitor/graphics card.

[DB83]

Allow me to make a suggestion with the compression.

Firstly I do not use Vdub and I do find avidemux quite easy to use. Also for xVID h263 is perfectly ok rather than x264. Yesterday, since I wanted to do this I took an mpeg2 source of 63 minutes and loaded it into avidemux. I set the video compression to xVID and selected a 2-pass encoding of 700 mb. For the audio, I selected a 96kbps mp3. The resultant avi was 742 mb - 700mb for the video @ 1538kbps and just a little higher than my suggested figure. I tend to work on between 10 and 11kbps for each minute of video. Using this forumula, your final video should be just over 1 gig.

I would mention that I only do these encodes for myself so am not so fussy about final quality - they look fine for me so that is all that matters.

[jagabo]

Alright, I appear to have just discovered why my blacks are so off. I think it's a hardware limitation of the monitor I am working on, a Gateway VX1120 22" flatscreen CRT. This monitor cannot distinguish ANY of the black squares in the imaging-resource website test. All of them display as just one color-- solid black.

That's really bad. Even the 30 and 25 patches?

My brightness and contrast are turned up as high as the monitor will allow, and I still cannot distinguish any of them. I can make out all the lights, no problem. But absolutely no ability to display any of the darker blacks.

Can you differentiate all of the bars in this image:



It's not uncommon that the two darkest patches are hard to tell apart.

I start by setting the graphics cards proc amp settings to the default -- that usually gives an unmolested signal. Then adjust the monitor so that RGB test patterns are as good as they can get.

[pronco]

That's really bad. Even the 30 and 25 patches?

Completely indistinguishable, all of them.

Can you differentiate all of the bars in this image:

"W" is barely possible to make out. X Y and Z are completely indistinguishable.

I start by setting the graphics cards proc amp settings to the default -- that usually gives an unmolested signal. Then adjust the monitor so that RGB test patterns are as good as they can get.

This is probably where I need to start. I am suspicious that this monitor should be performing so poorly on its own. Now that I think of it, when I installed XP, Windows automatically detected the card and installed its own native driver. It seemed to work fine, so I kept it. I'm beginning to wonder if that could be what is causing the problem. The driver in device manager says "NVIDIA GeForce FX 5200 (Microsoft Corporation)." I am going to blow it out and install the Asus drivers. Maybe then I will be able to access the graphics card proc amp. Again this is a V9520 Video Suite.

I set the video compression to xVID and selected a 2-pass encoding of 700 mb.

I keep seeing references to doing 2 passes. I will research how and why to do 2-pass encoding now. By the way, I encoded the entire movie using VDUB's IVTC filter. It helped greatly with combing but in scenes with very fast motion I can still make out the interlace lines. I'm surprised they're there. Am I missing something?

[jagabo]

Can you differentiate all of the bars in this image:

"W" is barely possible to make out. X Y and Z are completely indistinguishable.

That explains (partially) why the Youtube video you linked to earlier is extremely washed out. The darkest blacks in that video (after rec.601 conversion to RGB) are between the T and U bars. Nowhere near zero, where they should be.

I am suspicious that this monitor should be performing so poorly on its own.

Aren't you using an old CRT? Performance like that from an aging CRT wouldn't surprise me.

Now that I think of it, when I installed XP, Windows automatically detected the card and installed its own native driver. It seemed to work fine, so I kept it. I'm beginning to wonder if that could be what is causing the problem. The driver in device manager says "NVIDIA GeForce FX 5200 (Microsoft Corporation)." I am going to blow it out and install the Asus drivers. Maybe then I will be able to access the graphics card proc amp. Again this is a V9520 Video Suite.

I would try the latest WHQL certified driver dirictly from Nvidia. I doubt it will help though.

I set the video compression to xVID and selected a 2-pass encoding of 700 mb.

I keep seeing references to doing 2 passes. I will research how and why to do 2-pass encoding now. By the way, I encoded the entire movie using VDUB's IVTC filter. It helped greatly with combing but in scenes with very fast motion I can still make out the interlace lines. I'm surprised they're there. Am I missing something?

VirtualDub's IVTC filter isn't the greatest. It only handles straight 3:2 pulldown. And even at that it doesn't do especially well. There are a lot of PAL/NTSC conversions that involve blended fields, odd pulldown patterns, and other things that VirtualDub won't be able to fix. AviSynth can deal with most of those issues.

Regarding 2-pass encoding: it's better than single pass encoding when using bitrate based encoding. With bitrate based encoding you specify the bitrate (and hence the file size since file size = bitrate * running time) but you don't know what the final quality will be. Xvid also has constant quality (Target Quantizer) encoding where you specify the quality and the encoder uses whatever bitrate is necessary to deliver that quality. Basically, you use bitrate based encoding when you need a particular file size, quality based encoding when you want to assure quality.

Bitrate based encoding requires two (or more) passes to assure the best results. During the first pass it examines the video. During the second (or later) pass it uses the information from the first pass to allocate bitrate to difference shots -- shots that need little bitrate get less than shots that need a lot.

[pronco]

Excellent. As you can imagine, this is a lot of information for a newbie to process. It might be a little while before I can reply back, but I will try to do so soon. I am going for quality, so I assume I would be using bitrate encoding, 2-pass. Also, I believe it's time I started looking into AviSynth.

[jagabo]

I will try to do so soon. I am going for quality, so I assume I would be using bitrate encoding, 2-pass.

If you need a fixed file size you use bitrate based encoding. Using 2-pass encoding will give you better quality than 1-pass encoding for that size. But you don't really know what that quality will be for any particular video. If you are going for quality you should use quality based encoding. You specify what quality you want and let the bitrate fall where it will.

[pronco]

If you are going for quality you should use quality based encoding. You specify what quality you want and let the bitrate fall where it will.

Okay, yes. I incorrectly said I should use "bitrate" encoding. I am going for quality. Bitrate encoding is not what I want to use, then. Thanks for clarifying.

[jagabo]

With Xvid a target quantizer of 3 is a pretty good compromise of size vs quality. The video will have a little macroblocking if you examine still frames closely but you won't notice them at normal playback speed. Drop down to 2 and there will be almost no macroblocking but the files start getting pretty big. 4 is the default but is lower quality than I care for. You can use non-integer values too. Possible values range from 1 to 31.

I use x264 most of the time now. It has two constant quality modes: QP and CRF. QP is much like Xvid's Target Quantizer -- constant quality in a mathematical sense. CRF is similar but takes into account the human eye is less sensitive to some losses than others. CRF gives smaller files than QP but with similar visual quality. I usually use CRF=18. Again, smaller values give higher quality and bigger files, larger values give less quality and smaller files. Possible values range from 0 (lossless) to 51.

[DB83]

In the sample I refered to above, if memory serves me correctly, the 700 meg/63 min file was encoded with a quantiser of 3.

I would agree that I have less control over the final result by selecting a target file size rather than a bitrate but this has not let me down yet.

[jagabo]

In the sample I refered to above, if memory serves me correctly, the 700 meg/63 min file was encoded with a quantiser of 3.

The Target Quantizer values isn't used when encoding in bitrate mode. The quantizer is varied for different shots. An hour long video at 700 MB probably has an average quantizer in the 4 to 5 range.

[DB83]

In the sample I refered to above, if memory serves me correctly, the 700 meg/63 min file was encoded with a quantiser of 3.

The Target Quantizer values isn't used when encoding in bitrate mode. The quantizer is varied for different shots. An hour long video at 700 MB probably has an average quantizer in the 4 to 5 range.

I'll take your word for that. I only report what avidemux was reporting to me as it encoded the file. The bit rate it reported to me was varying quite a bit.

I certainly do not note any blocking in the final video but then the source did not exactly challenge the encoding.

[jagabo]

I only report what avidemux was reporting to me as it encoded the file.

I haven't used AviDemux in a long time. If it reports an average quantizer at the end then it's probably accurate. Don't count on all your videos to be as compressible.

<edit>

I just ran a test and encoded a video with AviDemux. I see that it shows what quantizer it is using as it encodes the video. I was seeing it flip back and forth between 2 and 3 with my test video (a very clean DVD rip). Keep in mind that different videos will compress differently. Another video may have to resort to much higher quantizer values at the same bitrate.

See the videos in this post for example:
https://forum.videohelp.com/threads/295672-A-problem-for-video-experts?p=1811057&viewfu...=1#post1811057

</edit?