Can't get good quality x264 encode

[silverwolf0]

I am using avidemux and x264, trying out various settings on a small vob clip I cut out from a full movie. I want the best quality possible at maybe half the filesize of the original vob. However, I can't seem to get it to look as crisp or as smooth as the original. Even with cqf set to 1 and a filesize 8 times bigger than the original (just to test it out), its easy to separate the two since the encoded one is more drab and blurry not as sharp. What could be wrong?

[smitbret]

Sounds like bad source material. What are you cutting from?

[jagabo]

Are you comparing by viewing both videos at the same time in media players? If so, that's the source of your problem. The first media player gets the graphics card's video overlay feature. The second has to use the Desktop GDI. The two have different proc amp settings so they will usually look different. Start playing the two videos in the opposite order and you'll see the opposite result.

Other possibilities:

Is the source progressive or interlaced? If you're deinterlacing you'll probably get less sharp results.

Are you maintaining the original frame size and using SAR flags for the aspect ratio? If not, the video is being resized before being encoded. That can make the video less sharp.

[silverwolf0]

I have tried videos both with and without the yadif deinterlacer filter. The original vob is interlaced, but plays fine in media player classic. However, once encoded, you can clearly see the combs, so I apply the filter beforehand. In either case, the output still looks a bit drab and less contrasty/slightly darker. It doesn't matter what order I play the video files when comparing them. Media player classic does tend to crash when I try to close it and open the other video to compare. I have also tried vlc player. It looks the same.

I can make the video look a bit better by playing it in windows media player (xp sp3 version) and moving the contrast slider slightly higher, which helps with colors and contrast, but the main problem is the motion, which still has a subtle blur and lack of smoothness. Frame by frame comparisons also show less perceived detail and more blurriness. I'm an amateur at encoding, especially x264, and the results now (and before when I had used xvid4psp) have been less than perfect so there must be something I'm missing...

Do you think it might have something to do with the loop filter? Avidemux has it set to 0 for both strength and threshold by default. I've tried disabling it. The video still looks the same. I don't know anything about SAR flags and whatnot. The original vob is 720x480 but plays automatically in a nice 4:3 when opened in media player classic. The encoded file(s) are also 720x480 but remain at that aspect, so they look wider. Setting the aspect to 4:3 within the video player so it looks like the original (or conversely stretching the original wider to match the encoded file) doesn't seem to make a difference in quality. I also definitely haven't ruled out the possibility of playback differences in video players/decoders/gpu enhancements (x264 vs mpeg2), but yeah I really don't know since vlc player still doesn't display anything different and its suppose to be self-contained?

[jagabo]

Are you using different players for each type of video? Some players perform more processing on videos before displaying them. Cyberlink's DVD player, for example, cay apply contrast enhancement and sharpening filters.

VLC is not completely self contained. It has its own file reader, splitter, and codecs but, by default, it uses DirectShow to put the video on the screen. You can select GDI output to eliminate that (at the cost of more processing).

Instead of comparing with media players use an editor like AviDemux or VirtualDub.

Use the Aspect Ratio flag to get the correct aspect ratio on playback. The setting is on the Ouput1 tab in AviDemux.

If you are converting 30i (60 fields per second) to 30p then you will get less smooth motion. Convert to 60p for smoother motion (eg, Yadif in bob mode). Yadif usually leaves behind some artifacts. A little combing, aliased edges, blurred motion, etc.

As an experiment, encode a short section with HuffYUV (lossless) instead of AVC. That will tell you whether or not the problems are occurring in the processing or in the AVC encoder.

[silverwolf0]

Thanks a bunch. I set it to bob in yadif and bottom field first and it is smooth now. Been messing with this for hours. I have a few more questions though. What is this spatial and temporal check in yadif and should I use it with bob or just use bob by itself? Bob doubles the framerate. Is this typically what people do to convert interlaced video? Another question is, most guides don't talk about deinterlacing when converting from DVD to x264. I am wondering, is this the typical procedure for ripped vob files, or is the vob I'm using special in some way? When I view the vob with mediainfo, it says 29.97fps / 720x480p / NTSC / interlaced / bottom frame first. Isn't this pretty normal for a DVD vob?

[jagabo]

You should usually leave the temporal and spacial checks enabled. With the spacial check Yadif looks at scanlines above and below each scanline to detect parts of the frame with comb artifacts that need to be deinterlaced. With the temporal check it also looks at scanlines from the previous and next frame to determine what needs to be deinterlaced. It then only deinterlaces parts of the frame that needs it. The more it looks at the better it works.

If your video is telecined film rather than fully interlaced video you should inverse telecine rather than deinterlace. That way you'll be encoding 24 fps rather than 60 fps.

Most people perform a simple deinterlace and leave the video at 30 fps. With telecined film that leaves a duplicate frame every 5th frame. That shows up at several little jerks every second. With fully interlaced video (usually camcorder video, live sporting events, news) you reduce 60 motion increments every second to 30 increments. That creates a very fast jerkiness, more like a flicker. Some devices will have problems with 60 fps playback. If you upload 60 fps to Youtube or similar video site they will reencode to 30 fps.

[silverwolf0]

Thanks again for your very thorough explanations. The source video is pretty much a cheapo handheld cam that shakes alot and not some big budget movie deal so I am guessing it is regular 29.97 and not 24fps. Since it has a lot of shake the lack of frames makes it look real jerky instead of smooth. But I think its more than just a 6 fps jump. Do interlaced vob/mpeg2 files play automatically with doubled framerates in MPC and other players? I am also wondering if there's a way to tell if a movie was originally 24fps or not, besides just guessing based on the type of movie content? I've tried what you said and used telecide 3:2 pulldown + decimate to reduce it to 24fps but it looks the same, so am guessing it is not originally 24fps. Since most people converting DVDs are ripping feature film movies, which were shot in cinema 24fps and are naturally blurry during motion, is that why most people don't notice or don't care about deinterlacing? I am also wondering, since all the encoded videos end up being progressive, wouldn't it have been better to just leave it interlaced and have the player double the frames? I might wanna go see if there's a keep interlace option...

I am also a bit confused about the 24fps and 29.97fps. Is a real feature film orignally only 24 full frames meaning 24 actual sampled images, and regular cams instead are 29.97fps interlaced so it is really 60 actual sampled (half) images? So 24 vs 60 would make the 60 one a helluva lot smoother since it actually records 60 unique (temporally) half images? Why can't I just push a button and the hamsters in my computer automatically do everything with immaculate results?

[jagabo]

Handheld camcorder video is normally 30 fps interlaced. The exception would be if you used your camcorder to record a TV broad of a movie (some camcorders have s-video or composite inputs).

Each frame of interlaced video contains two separate pictures. One is in all the even numbered scanlines, one in all the odd numbered scanlines. Each of these half pictures is called a field. The fields are intended to be viewed sequentially, not simultaneously. So a 30 fps interlaced video is displayed as 60 different half-pictures every second (note that marketing speak has changed from calling this 30i to 60i -- but the two terms mean the same thing). 60 fields per second is visually smoother than viewing only 30 different pictures every second.

Film is shot at 24 fps and telecined to 60 fields per second. There are many places on the internet that discuss this process, called 3:2 or 2:3 pulldown:

http://en.wikipedia.org/wiki/Telecine#2:3_pulldown

That adds a judder (shakiness) that is often visible, especially in high contrast shots with smooth, medium speed motions (especially pans). See the sample video in this post (you need a 60 Hz monitor to see it properly):

https://forum.videohelp.com/threads/307004-Best-framerate-conversion-%28eg-23-97-to-30-...=1#post1888926

You can tell what you have by opening one of your videos in AviDemux and add the Yadif deinterlace filter in bob mode. Use the preview feature and find a spot in your film with a lot of motion. Step through the video field by field. If there is motion with each field you have fully interlaced video. If you see motion in a 3:2 repeat pattern you have telecined film.

[silverwolf0]

Is there a way to make the encoded x264 video interlaced (like the original file) so I don't have to use a deinterlacing filter and instead rely on the video player?

Edit: Found an option in the frames tab. Checked interlace box and bottom-field-first. The resulting video shows up as MBAFF for the scan type in MediaInfo. However, using MPC to play it, same old same old. Doesn't seem to trigger the built-in deinterlacer. Now that I think about it, its better to just deinterlace it when encoding so that the file will play nicely, barring any extra effort a 60fps video might give to the cpu, so that no matter what player I use it will always look nice...

[jagabo]

using MPC to play it, same old same old. Doesn't seem to trigger the built-in deinterlacer.

Yes, that's the problem. Many players don't detect the interlaced video and deinterlace it for display. You have to manually force the player to deinterlace.

[jagabo]

If you want better deinterlacing look into using AviSynth and QTGMC() (previously TempGaussMC_beta()).

https://forum.videohelp.com/threads/292642-Deinterlacing-Tips-and-Good-news-that-i-foun...=1#post1784755