Benny Hill on DVD - how to properly convert it to XviD?

[Jeff7]

I've attempted so many techniques to do this, but I'm up against a few potential problems:
Benny Hill was recorded in the UK, so I assume that video was filmed at 25fps, though I'm not sure of this.
The DVD is region 1, NTSC, 29.97fps, and it is interlaced. I've tried Deinterlace:Blend in Virtualdub, but I don't like the ghosting. I've tried Smart Deinterlacer 2.8 beta 1, with or without Smart Bob 1.1 beta 2. In those cases, I either get jumping, or else a shimmering stair-step effect on lines. I've tried inverse telecine with TMPGEnc and Virtualdub, and then also tried AVIFrate. The result of that was jumpy video, in that the motions were not smooth.

Is there any good way of sifting through the various recording and encoding methods employed, and get a clear, deinterlaced video?

[Wile_E]

Use ripit4me and AutoGK. Simple and easy to convert to Xvid/Divx. You are probably better off leaving it interlaced and using a proper xvid/divx software player to play the file.

[manono]

Hi-

Is there any good way of sifting through the various recording and encoding methods employed, and get a clear, deinterlaced video?

I haven't seen the Benny Hills, but from your description, they used a PAL master for the NTSC DVDs. A sample would help prove (or disprove) that. If it is PAL2NTSC, then one good way to make it progressive again is with the use of the RePAL filter in AviSynth:

LeakKernelBob(Order=1)
#TDeint(mode=1, full=false, tryweave=true)#slightly better, but slower
RePAL()

AviSynth is the only way to get it progressive again and remove the blends at the same time. IVTC is no good at all to you. No deinterlacer is going to help.

[jagabo]

If they used 3:2:3:2:2 pulldown to convert PAL to NTSC (without blending, and without breaks in the pattern) you could use TMPGEnc's "deal after this frame with selected pattern" IVTC option to restore it to 25 fps.

[Jeff7]

Hi-

Is there any good way of sifting through the various recording and encoding methods employed, and get a clear, deinterlaced video?

I haven't seen the Benny Hills, but from your description, they used a PAL master for the NTSC DVDs. A sample would help prove (or disprove) that. If it is PAL2NTSC, then one good way to make it progressive again is with the use of the RePAL filter in AviSynth:

LeakKernelBob(Order=1)
#TDeint(mode=1, full=false, tryweave=true)#slightly better, but slower
RePAL()

AviSynth is the only way to get it progressive again and remove the blends at the same time. IVTC is no good at all to you. No deinterlacer is going to help.

I guess I'm finally going to have to figure out how to use AviSynth.
I've got a load of questions on using it, but I suppose I should save them until after I've read FAQ's and how-to's on the program.

Now I'm not sure about how PAL works - is PAL ever interlaced at any time? This was shot for TV originally, so I don't know if they'd interlace it for broadcast or not in the original format.

You say "a sample would help" - what exactly do you need in terms of a sample? How should it be prepared for posting? Thus far I've used DVD Decrypter to rip the episodes, and I've tried DGIndex and VFAPI to serve up the VOB files for use in Virtualdub or TMPGEnc.

If they used 3:2:3:2:2 pulldown to convert PAL to NTSC (without blending, and without breaks in the pattern) you could use TMPGEnc's "deal after this frame with selected pattern" IVTC option to restore it to 25 fps.

What pattern would I use though?


I don't know if this helps any, but I just looked at a scene change transition. There are 2 frames where elements of the next scene begin to appear slightly, then there is a frame where half the fields are from the previous scene, and the other half are from the next scene. Following that are 2 frames that show elements of the previous scene. So it's a total of 5 frames just to fully complete a scene change.

[jagabo]

If they used 3:2:3:2:2 pulldown to convert PAL to NTSC (without blending, and without breaks in the pattern) you could use TMPGEnc's "deal after this frame with selected pattern" IVTC option to restore it to 25 fps.

What pattern would I use though?

If I remember correctly it would be a rotation of this pattern:

101010010100

But it sounds like you are seeing some field blending so the AVISynth RePAL filter will work better -- it will remove most of the blending.

[kush]

You say "a sample would help" - what exactly do you need in terms of a sample?

A short sample cut from the DVD itself..It can easily be done in any number of tools..I use VideoRedo for my HD caps..Well, it can input (and output) .VOBs. Set a short in/out sequence, say 10-20 secs tops (remember not everyone has DSL/cable), cut, invert cut so everything but your sample will be gone..And output to either .MPG or .VOB.

[edit]
You don't want to use any tools where your video will be processed (other than the cutting of course) prior to the sample output..The whole point of the sample suggestion was to verify the source.

[manono]

To get a sample, open a vob in DGIndex. Scroll to a place with motion and use the "[" and "]" buttons to isolate a 15-20 second piece. Then File->Save Project And Demux Video. Take the M2V and upload it to http://rapidshare.de/ When done they'll give you a link which you then post here.

[manono]

Now I'm not sure about how PAL works - is PAL ever interlaced at any time?

Yes, PAL can be interlaced. More often though, it's a progressive source which is encoded as interlaced. 2 different things. Not having seen the Benny Hills, I don't know if they were originally interlaced or progressive. I've seen a ton of BBC TV shows converted to NTSC, and almost all were from progressive PAL sources (some maybe deinterlaced, not sure) before being converted to NTSC. The RePAL filter works well on them.

Since you have DGIndex, make sure and read the documentation that comes with it. A simple script using RePAL goes something like this (adjust for the paths and names):

LoadPlugin("C:\Path\To\DGDecode.dll")
LoadPlugin("C:\Path\To\LeakKernelDeint.dll")
LoadPlugin("C:\Path\To\RePAL.dll")
MPEG2Source("C:\Path\To\Video.d2v")
LeakKernelBob(Order=1)
RePAL()

Both LeakKernelDeint and RePAL can be found here:

http://www.avisynth.org/warpenterprises/

First you make the D2V Project file using DGIndex with Video->Honor Pulldown Flags set. Then you create a new .txt file, but rename it with the .avs extension, something like Video.avs. Inside it you stick that script, modified for your paths and names.

Test your scripts in VDub(Mod) before encoding. It'll give you an error message to help pinpoint any problems. You can also scroll around and advance frame by frame so if you can see if it's working. It won't get rid of all the blending (you'll sometimes see some right after scene changes, before it locks onto a new pattern, or in dark places where it can't "see" what's going on), but as long as it plays smoothly, that's the main thing. You should have very little jerkiness if it's working correctly.

If making a DVD, it'll give you a progressive MPV at 24.975fps. I encode those for 25fps. Then run it through DGPulldown with the custom box checked, for 24.975->29.97. That way you can keep the audio and subs untouched. If making an AVI, just keep it at 24.975fps, but you may need to add lines for the crop and resize.

By the way, forget VFAPI and/or frameserving via VDub. It's slow, inferior, and was obsolete 5 years ago. Forget IVTCing using TMPGEnc. Even when possible, it takes so long to set up (if the pattern changes at all, as it nearly always does), and so long to encode, that it's not worth it. Any IVTCing necessary can be done just as well and a whole lot more easily and much more quickly in AviSynth.

[RabidDog]

Smart Bob ? he was the Slaphead who got his pate slapped in every episode... WAITER! this chricken is rubbery.......cue music..

[Jeff7]

To get a sample, open a vob in DGIndex. Scroll to a place with motion and use the "[" and "]" buttons to isolate a 15-20 second piece. Then File->Save Project And Demux Video. Take the M2V and upload it to http://rapidshare.de/ When done they'll give you a link which you then post here.

I never actually noticed that function.

Sampling, 1.8MB (Original crappy sample link removed, please use this one instead
# It's a short section, but it's a problematic one.

Note the top of the doorway. I just tried the very helpful script you posted, and just about everything looks great, except the doorway and horizontal lines. They aren't really jagged....maybe "distorted" is the right word.

Another sample, XviD this time.
This is from a scene close to the one above, this is a cropped portion of the door, encoded as losslessly as I can get with XviD. Note the odd shimmering effect around the edges.
This is the output from the AVISynth script file.

I'm trying to tweak the threshold= parameter of the LeakKernelBob setting, but I've never had much luck with adaptive deinterlacers like that. They always seem to cause artifacts on nearly horizontal lines, or else they leave ghosted comb-teeth effects around motion areas.


FYI, these are being encoded to XviD for playback in WMP.

Maybe if I get AVISynth figured out, I can re-do my Futurama discs. DGIndex almost has them done right when set to Film, but occasionally some interlaced frames still appear for some reason. But again, that's through VFAPI, so maybe that has something to do with it, I'm not sure.
Edit: I've tried various filters now on the Futurama thing, including Tivtc, fdecimate, and decomb522. They do improve slightly upon DGIndex's Film output, but there are still interlaced frames appearing. Yeah well, that's for another thread. Benny Hill is the Priority Right now.

[manono]

Hi-

About the distorted door. When you notice artifacts, you have to check to make sure they're not in the source. They are in this case. I can't tell if it's a trick of light or if that top steel bar is really bent, but it's that way in the MPEG, so I wouldn't worry about it being in the RePAL result.

Ordinarily RePAL is meant for PAL2NTSC conversions where the fields have been blended. I can't find any evidence of that having happened in that clip. Maybe elsewhere in the shows. You check by running LeakKernelBob by itself, with RePAL commented out:

LeakKernelBob(Order=1)
#RePAL()

That gives you bobbed fields and a 59.94fps framerate. You're supposed to see a whole lot of blends when stepping through it, and I see none. When stepping through the fields I see every field being different. When you have one of these blended PAL2NTSC conversions, 1 frame in 6 is progressive, and I see none being progressive. If all the shows are like that, and you're doing this for AVI, I couldn't fault you for making it 29.97fps. If RePAL works to give you smooth playing video, though, then it must be different elsewhere in the video. I have seen small parts of videos where they looked like that briefly and had the more normal blends elsewhere, and RePAL worked fine.

The reason you get some shimmer or aliasing of thin lines is because RePAL and all unblenders use a lot of bobbed fields, and thin lines tend to do that when being deinterlaced that way. It normally looks OK when being watched on TV, but that's probably the main drawback of unblenders such as RePAL, Restore24, and CRestore. To me though, it's a lot less objectionable than blends, which drive me crazy. And there's usually little enough of it that it doesn't become too annoying. But it's up to you to decide what looks best. Instead of using LeakKernelBob for the bobbing, slower but slightly better would be TDeint:

TDeint(mode=1, full=false, tryweave=true)

It's not going to work wonders though.

They do improve slightly upon DGIndex's Film output, but there are still interlaced frames appearing.

About the Futuramas. If Force Film works pretty well except for some interlaced frames every once in a while, and if they play smoothly, then one way to fix it might be to add a conditional deinterlacer into the mix. I like TDent for that because it's such a good deinterlacer:

TDeint(Full=False)

It's supposed to leave it alone when progressive, and only kick in when it "sees" interlacing trying to sneak through. I haven't worked with Futurama either, but maybe it drops to video around edit points/some scene changes. If so, either a full IVTC might work, or what I suggested just above. By the way, TIVTC has a nice mode where it skips the IVTC when it encounters Film, and only IVTCs when it sees video. It reads the D2V:

TFM(D2V="C:\Path\To\Futurama.d2v")
TDecimate(Mode=1)#for animation

[jagabo]

The MPEG video shows blending. Here's frame 7 after VirtualDub's "duplicate field 1" deinterlace filter:



RePAL will work better for this video than a TMPGEnc. But I suspect the show was video taped so there are no progressive frames to reconstruct.

[Jeff7]

Hi-

About the distorted door. When you notice artifacts, you have to check to make sure they're not in the source. They are in this case. I can't tell if it's a trick of light or if that top steel bar is really bent, but it's that way in the MPEG, so I wouldn't worry about it being in the RePAL result.

It's not the bending, I know that it's bent. Open the AVI file in Virtualdub, and go through a frame at a time. Maybe set the Zoom to 200% or 300%.
Along the lines, distortion appears.
Go from frame 0 to frame 1. The line on the top becomes fuzzy. Frame 2, it's normal again. Frame 3 suddenly shows a few odd pixels below the line, then frame 4 shows more toward the left of the bulge. Or switch back and forth between frames 14 and 15. Odd shimmering effects are visible.
That's the distortion I meant, not the bending of the steel bar.

I don't know much about PAL's history, but this episode was recorded in 1972. I don't know that "progressive" referred to anything but perhaps a political movement at that time.

I guess jagabo is right then. This is an instance where the source just either doesn't convert well to NTSC, or else the people making the DVD didn't put enough time into doing the transfer properly, such as what happened with the NTSC -> PAL Futurama DVD conversion. In that case, the PAL version featured interlacing effects and ghosting in the progressive PAL output. A lot of people were quite disappointed in the PAL version.

[manono]

You're talking about the PAL Futuramas? It says you're in the US. Why on Earth would you want the PAL Futuramas? They're a famous screwup. Get the NTSC versions.

[Jeff7]

You're talking about the PAL Futuramas? It says you're in the US. Why on Earth would you want the PAL Futuramas? They're a famous screwup. Get the NTSC versions.

I do have the NTSC Futuramas. I was merely using the PAL Futurama DVDs as a comparison of what happens when the corporations get sloppy about switching between standards. It looks like that may have happened with the Benny Hill DVD's too.

[vhelp]

Its been my uderstanding that older generation tv programs were shot in
interlace. Be it NTSC[29.970] of PAL[25] (and especially for these type
of shows back in those days) they were always Interlace. But, todays
shows of this type may be different now. I don't know. Anyway.

But, it is quite possible to get away with the RePal() filter on this and
get a 25 fps. But, at the expense of interlace jaggies. I've been messing
around with the demo clip you left behind, but its not enough to fully
test the fluidness of the conversion with such a short number of frames
to work with.

I wouldn't mind seeing a longer sample clip. BH was one of my favorite shows
back in the day. I had many favorites. Anyways.

-vhelp 4092

[Jeff7]

Longer clip, a bit over 6MB. 345 frames, 11.512s long. (Crappy sample removed, please use this sample instead.)
# It features some more motion, and additional scene changes.
I don't know how much is allowable to be posted on this forum, so I don't want to post a file that's too awfully big. What is considered "acceptable" here? Maybe one of his short skits in its entirety? (Some were less than 30 seconds.)

[manono]

You haven't quite gotten the knack of choosing video to upload. You want to choose video with some actual movement, and not a couple of people talking. A camera panning, or someone walking, are the kinds of things you want when studying a video. I repeat that there's no sign of blending, as in blends/ghost images/double images in the fields. Scene changes don't count. I think I'd leave it at 29.97fps, and if you have to deinterlace for AVI, then here's a comparison of different AviSynth deinterlacers, some very slow:

http://heptium.sh.cvut.cz/~integra/deint/

There are certain kinds of images that just don't deinterlace very well (yet). You have them in parts of the Benny Hills, and the pics from that link show some of the hardest possible images to deinterlace. I chose LeakKernelDeint earlier because it's the fastest decent deinterlacer and smart bobber, but there are some much better ones, some excruciatingly slow.

[Jeff7]

Sorry, I thought those clips would be ok. They do show someone moving side to side at least.
I'll find another clip with more motion later today and edit it into this post. I've got some other obligations to take care of first.


Speed isn't much of an issue here. I just queue up lots of work for Virtualdub to do and set it to work in the background. Granted, there is some kind of a limit, but our ideas of "excruciatingly slow" may differ. 2-5fps processing rate is acceptable. 0.25fps is not. I'm running an Athlon XP2400 right now. I had wanted to upgrade, but college fees have dibs on my money right now.

[manono]

As an example, I tried MVBob and RePAL on the 2nd clip and it took 11 minutes on an Intel 3 GHz. Something like half a frame a second. And it still didn't get rid of those weird artifacts on the black lines above and below the bent steel plate above the door.

[Jeff7]

Crazy. It just seems somewhat amusing to me - interlacing is such a simplistic method that TVs can figure it out, yet it just doesn't work with computers. (Yes, I understand the whole difference in how they render images.)
Heh, what about a TV emulator that displays the image as a TV would, and it then takes a picture of that rendered image. TV-style deinterlacing!

Thanks for the comparison of the AviSynth deinterlacers. I'll look over them and figure on which seems to give the fewest artifacts. I'm a bit of a perfectionist when it comes to video and audio, such to the point that I notice visual noise on professionally made DVDs, such as the Futurama DVDs. There are little artifacts on some of them that look like noise in a poorly compressed JPEG.



Edit: MVBob - definitely out of the question. Slow doesn't even begin to describe it. Encoding to XviD at the same time, it registers as 0.0fps in Virtualdub, with quick bursts of 3fps, averaging at 0.15fps. Maybe if I had a quad-core Xeon 3.2GHz it might be alright. But it does have nice quality, and a fairly low bitrate in the resulting encoded file.
Though it seems fussy. Now it's doing 1fps.


Edit #2: I do believe I said I'd post another sample. Hopefully this sample might be better?
Panning, motion, scene changes, trees (bare branches = loads of fun for a deinterlacer), and changes in camera speed. Will that suffice?
There's even hair on the lens or film in frames 336 and 337.


And I may have found something that's not perfect, but it doesn't leave shimmering, and it's at a framerate that doesn't make my encoder scream for mercy:

MPEG2Source("z:\sample.d2v")
separatefields
RePAL()