x264 can't handle fast motion no matter what I try

[Moonside]

In short, when I encode with x264 it looks great on anything but scenes with a lot of motion. In those scenes it's either horribly blurry and blocky, or depending on the filters I use in VirtualDub, it stutters a LOT or appears to have bright highlights as a sort of ghost image.

Is there a way to handle this? I'm using VirtualDub and working with recorded TV shows from a DVD recorder, so the source is interlaced. I've tried numerous deinterlacers, and the built in one in VirtualDub works fine except on scenes with motion. The one called "Asvzzz Deinterlace" is the only one that works for motion scenes, but then the other problems I mentioned start showing up. Most often the video stutters in a very noticeable way.

The codec settings for x264 are basically the defaults. The bitrate I use is usually 700 to 900. My computer is a Pentium D, 3 GHz, 900 MB RAM. The video card, from what I know, isn't a very good one, but I don't know if that's a problem. I have memory to spare when I encode, and I've tried putting the priority on higher in case that would help (but thread priority doesn't affect encoding, right?)

For those of you who do encode a lot, have you noticed any problems with x264 and motion? I find it odd I've never heard or read about it, so I hope it's just some setting I can change so it handles it correctly. My encoding has basically come to a standstill because of this, those scenes are absolutely poor quality compared to the rest of the video.

[ggouweloos]

What's the resolution you are encoding in?
If you are encoding fullsize I think the bitrate is too low for scenes with a lot of motion.

[guns1inger]

I would suggest the issue is more with the deinterlacing than the encoding, given that the problem changes with the deinterlacer. A simple test would be encode the same sample footage with the same deinterlacers and Xvid to see what happens. If you still get image problems you know it is the deinterlacing methods at fault, and you can concentrating on getting that right. If not, then you know it is the encoder and can concentrate your efforts there.

[ggouweloos]

700-900kbps for a 720x480 (or 720x576) resolution is too low for high motion video. It will result in blocky video. I suggest trying a higher bitrate or a constant quality setting (but I do not know if that is possible with X264).

[Moonside]

Sorry I forgot to mention the resolution. It's 720x480, but I can resize it to 640x480 as well, right?

I'll try a higher bitrate to test that scene, and give XviD a try and more deinterlace settings. Thanks for the suggestions so far.

[Moonside]

XviD seems to work, but it's very grainy compared to x264, and on top of that there seems to be even less detail in the picture. That's why I preferred x264. I'm not sure, but I'm guessing there's no way to get rid of grain while increasing detail in XviD to match what I'd get with x264.

About the stuttering, is it just not able to handle the information while it's encoding? Or is it the end bitrate that's the only problem?

I tried 1200 and 1500 and it seemed to flicker or stutter less often. By 1800 it finally went away. 1800 doesn't really increase the picture quality at all, for the amount of space it takes up. And I've seen fairly good videos without such huge bitrates.

So if a much higher bitrate is needed, there a setting I can use to allow it to go as high as necessary when the situation calls for it? I'm not that familiar with customizing codec settings, but I've heard of something like this. Something about the bitrate variance or zones?

[jagabo]

It sounds like you are performing a contant bitrate encode with x264. This means that low motion scenes will be getting as much bitrate as high motion scenes so you files will turn out very large. You need to use a multipass VBR encode to get good quality with a small file size.

Or instead of picking a bitrate put x264 in Constant Quality mode, pick a quality you're happy with, and encode. Each frame will get as much bitrate as necessary to achieve the desired quality. Try somewhere around 200 to 250.

To get smooth results without interlace artifacts, most NTSC movies should be inverse telecined, not deinterlaced.

[ggouweloos]

You could try Xvid with an alternative encoding matrix. De-interlacing is not a good idea if you want to have best quality.
A deinterlace will cut your framerate in half. For NTSC you will cut down from 60fps to 30fps, in PAL from 50fps to 25fps. When watching on a TV you will definitely notice this when compared to the original. What is your goal? Best quality or smallest filesize?

[Moonside]

In Constant Quality Mode it's basically just upping the bitrate, right? I checked using GSpot for the test clips I made and it's reporting high rates, but if I bring it down to around 1000 (using 300 Constant Quality) it still jitters around. I managed to get the picture quality to look alright, but it still keeps stuttering.

I found out what the stutter is at least. It actually skips back a few frames. I found this by going frame by frame. Randomly one appears that's from half a second ago.
It usually happens after a lot of motion but not always. Does knowing that help to figure out what the problem is? Is something falling behind during the encoding process?

I tried the inverse telecine option (Video > Frame Rate > reconstruct adaptive) but it gives me an "error -100, cannot start video compression."

As for the VBR I don't see a way to encode with that in VirtualDub. Do I have to use another program?

A deinterlace will cut your framerate in half.

It will? Yikes. It seems to make it look alright on the computer. So I guess I'll avoid it if I can get the telecining thing to work.

I'd like to avoid XviD if possible, since I've tried and I can't get it to look good at all compared to x264. And about quality vs. filesize, it seems like I have to go as high as 1800 to get that scene to stop jittering, but I can't encode entire episodes or movies like that. It's much too large. In this episode for example the scene with a lot of detail and motion is only 30 seconds long. The rest looks fine when I encoded it.

Don't most people use 900 or so for movies and TV shows? I've read that in a few guides.

[jagabo]

For film sources detinterlacing won't cut your frame rate in half. But you will be left with one duplicate frame out of every 5. for example: 1 2 3 4 4 5 6 7 8 8...

A successful inverse telecine will eliminate the repeat frames and leave you with the original film frames. 1 2 3 4 5 6 7 8....

VirtualDub's inverse telecine isn't very good -- unless your video has few pattern breaks and you can use the manual method.

But your random-old-frame problem is related to MPEG parsing, not encoding. When I've had this happen I try using DGIndex and AviSynth. Or virtualDubMPEG2 or VirtualDubMod.

Another possibility: h.264 in an AVI container can be hard to decode properly. Try setting the Max Consecutive B-Frames to 1 or 0. That will make decoding easier (though reducing the efficiency of the codec, especially at 0).

Constant quality mode is what the name implies. Every frame is encoded with the same visual quality. If a frame needs a lot of bits it gets it. If a few bits will do it only gets a few. If you specify a high quality the average bitrate will be high (each frame gets realtively a lot of bits). If you specify a low quality the average bitrate will be low (each frame gets relatively few bits). But no matter what quality you choose, frames that need more bitrate will get more bits and frames that need less will get less.

VBR mode in VirtualDub consists of picking MultiPass First Pass, encoding the video, then going back and picking MultiPass N'th Pass and encoding again. During the first pass x264 is examining each frame and determining how much bitrate it needs. During the second pass if uses that information to allocate variable numbers of bits to different frames, while meeting the average bitrate requirement (and hence the desired file size).

The "right" bitrate with CBR or multipass VBR will vary depending on many factors including frame size, frame rate, noise levels, motion, sharpness, complexity, etc. If you don't need files of a certain size (say, to put a 2 hour movie on a CD) Constant quality mode is much simpler.

[Moonside]

Thank you very much for your help, everyone. I'll have a look over the details again tomorrow when I have time. Hopefully one of these will take care of the problem.

[Moonside]

It looks like deinterlacing was the main problem. I noticed I had the interlace setting checked on x264 or XviD, and so when I used that on top of a filter by accident it caused many of the problems. Since I got the telecining option to work it's been okay. Even at 800 or 900 the quality is acceptable and it's not jumping around anymore.

Again, thanks for the help.

[Moonside]

Just when I had things working, it looks like using the inverse telecine option changes my frame rate. Is there anything I can do about this? It leaves it at 23.98 fps instead of 29. I went to look at some guides about this subject but it's rather difficult to understand.

Is there a simple solution or am I back to deinterlacing again? If I do that I might be right back where I started.

[Soopafresh]

It's supposed to be 23.976 after Inverse Telecine and Decimation

[Moonside]

Deinterlacing made it look just about as good and kept the frame rate. So I'm not sure what I should be doing now.

I've tried to read up on it but I'm still stuck. Unfortunately I need to ask for help. What exactly is the best way to handle interlaced video (from DVD recordings)? Deinterlacing gives me stuttering problems, I haven't seen a video of this size/quality needing such a high bitrate, XviD is grainy, IVTC has a lower frame rate, is there really no other option?

[jagabo]

First you need to figure out what your source is. Is it fully interlaced video or is it telecined film. A film source can be inverse telecined back to the original 23.976 progressive film frames per second, interlaced video is 59.94 different half-pictures per second. Interlaced video can either be deinterlaced to 29.97 frames per second, or bob'd to 59.94 frames per second (the latter isn't recommended because computers can have problems keeping up with the frame rate).

If you single step through a telecined film source you will see a pattern of three progressive frames followed by two interlaced frames. This pattern repeats over and over. Of course, the pattern is only noticable when there is motion. If you use VirtualDub's drop-field or dup-field deinterlace options on this type of video you will see a pattern of 1 frame repeated every 4 frames: 1 2 3 4 4 5 6 7 8 8...

Fully interlaced video will show all interlaced frames whenever there is motion. A drop/dup field deinterlace will have no duplicate frames.

Another possibility is bad a PAL/NTSC conversion which has blending and interlacing throughout.

I recommend you export 5 or 10 seconds of the video and upload it for us to look at.

Once you know what you have you can work on a strategy to convert it.

[Moonside]

Thanks again jagabo. I'll save this page and keep it in mind for future endeavors, but for the time being I'm going to have to stick with whatever works. Right now I'm just trying to record a show that's been on TV for a while since they'll probably stop playing it soon. So time is a little limited. Once I'm done with this show I'll get back to figuring more of this stuff out. For now the quality is acceptable and the motion problem is taken care of.

I appreciate all the help and information you guys have given me.

[pcspeak]

Hi Moonside.

I notice that although you're not happy with the quality, your encodes with xvid DO work.
Try this method and let us know if the quality is satisfactory.

Download xvid_encraw.exe & xvidcore.dll and put them in the same folder as your video.
http://forum.mediacoderhq.com/viewtopic.php?p=6618&sid=ee810052f766225c8bc876638bfd2fe9

If you don't already have it, download and install AviSynth. I use V2.5.7.

Create an AviSynth script (text file) in the same folder as the video. Yours will probably be different to mine.
The easiest way to test the validity of a script file is to open it in Virtualdub. If it opens, then it's OK.
Any help needed is at hand. Mine looks like this. (vid.avs)

avisource("myvideo.avi", audio=false)
converttoyv12()

Create a batch command containing the following TWO lines, in the same folder. Mine's called convert.cmd. (No wordwrap)

xvid_encraw.exe -i "vid.avs" -pass1 "vid.stats" -bitrate 900 -chigh 15 -clow 15 -overhead 0 -max_key_interval 300 -nopacked -vhqmode 4 -qpel -qtype 1 -closed_gop -lumimasking -bmin 3 -bmax 20 -imin 3 -imax 20 -pmin 3 -pmax 20 -bvhq -bquant_ratio 162 -bquant_offset 0 -threads 1

xvid_encraw.exe -i "vid.avs" -pass2 "vid.stats" -bitrate 900 -chigh 15 -clow 15 -overhead 0 -max_key_interval 300 -nopacked -vhqmode 4 -qpel -qtype 1 -closed_gop -lumimasking -bmin 3 -bmax 20 -imin 3 -imax 20 -pmin 3 -pmax 20 -bvhq -bquant_ratio 162 -bquant_offset 0 -threads 1 -avi "newvideo.avi"

Double click on convert.cmd.
Have a look at the output. You'll note there is no audio. We'll sort that out if the video quality is satisfactory. Maybe raise or lower the -bitrate in convert.cmd. I use 1000 and am happy with the size/quality tradeoff. Of course if I owned a 50" LCD TV.....