Help improving anime encoding quality

[Jas47]

I've recently tried converting some anime I have on DVD to Xvid clips. I've was able to get a clip with lower quality than I was trying to get.

First, I copied the DVD to my Hard Dive using DVD decrypter the used DGindex and VFAPI to get it to an avi. I used IFO mode on DVD decrypter and left the other programs at defalt. After that I tried encoding in vdubmod and I only used two filters. I resized to 640 x 480 (Lanczos) and deinterlaced using discard field 1 or 2 ( I don't remember which). I tried several modes of deinterlacing and that worked the best. The final picture I got was not as clear as the original DVD and had a lot of noticable pixils. I also noticed that all the black lines around the characters were like stair steps and not straight. Any suggestions on how to encode anime to look its best would be appretiated.

Included a picture. It's really noticable when on full screen.

eva.bmp

[jagabo]

The stair-steps were a result of the drop field deinterlace. Try opening the VOB file directly with VirtualDubMod. Then use File -> File Information. If you're lucky the frame rate will be 23.976 fps and you wont have interlace problems. Try using one of the Smart Deinterlace filters. None of VirtualDub's built in deinterlace methods is any good (except maybe Blend under some circumstances -- not anime though).

Use a 2-pass variable bitrate encoding to get better results (compared to constant bitrate) at any particular bitrate.

If that doesn't do it use a higher bitrate.

A slight Temporal Smoother often works well with anime. It will eliminate the slight frame-to-frame noise (MPEG DCT artifacts) and make the video more compressible.

[manono]

using DVD decrypter the used DGindex and VFAPI to get it to an avi.

Mistake number 1 was doing it through a VFAPI. If you want quality, frameserve via AviSynth.

deinterlaced using discard field 1 or 2

A huge mistake, as you instantly tossed out half the resolution, accounting for the aliasing/shimmering/stairstepping. You shouldn't be deinterlacing at all. IVTC using an AviSynth IVTC filter. Also, you wound up with the wrong framerate of 29.97 (if you used "Honor Pulldown Flags" in DGIndex).

Any suggestions on how to encode anime to look its best would be appretiated.

Use AutoGK until you learn how to do it right.

[jagabo]

Anime usually doesn't appear as 23.976 fps progressive with 3:2 pulldown. And it usually can't be IVTC'd because it's a mix film, CG (29.97 fps interlaced), and other pulled down progressive (12 fps, 20 fps, etc) frames.

[manono]

I've done my share of anime series for AVI. While sometimes a mix of 23.976fps and 29.97fps, I'd say most aren't hybrid, and even when they are, the percentage that's 29.97 (sometimes credits, sometimes parts of opening songs, and sometimes a very little bit of the episodes) is so small that if I have to make it one or the other, I'd definitely go with the 23.976fps, and so would most other people. So I repeat, IVTC is the way to go, and deinterlacing (especially by dropping one of the fields) it is not the way to go.

While a lot of it is drawn as 8, 12, or 16fps, dupe frames are added to bring it up to 23.976fps. You're right, it's not usually on the discs as 23.976fps progressive. It's usually hard telecined, or a mix of soft telecine and hard telecine (with the odd pure video). In my opinion, it can and should be IVTC'd. TIVTC has a pretty good hybrid mode that brings interlaced 29.97fps down to 23.976fps nicely.