Mr. Bean Animated series avisynth de-interlacing issues

[rickydavao]

Been on Videohelp for a while, mostly learning, although trying to help out when I can ... I guess it was only a matter of time before something stumped me and it was my turn to ask for help!
I've been trying to convert my NTSC DVD (Region 1) of the Mr. Bean animated TV show to XviD so that I can stream it off my PC to the kids' bedroom on their XBox 360. I'm guessing that the show was made in England, so was originally in PAL format, and then "modified" for NTSC (yay ...). Now I've had great success with "Agatha Christie's Marple" TV series (also British) using both srestore and repal in avisynth, but neither seem to work in this instance. I've tried just about every combination of de-interlacing filters available in both megui and the experimental version of XviD4PSP to no avail. I've also searched through the forum for other possible filter chains and have found several, but none of them seemed to do the trick (either left interlacing artifacts, sever jaggies or a lot of line and/or color ghosting).
Basically, all I'm looking for is an avisynth script that will let me input the VOB(s) via dgdecode (that much I can do), and give me a progressive output at either 23.976 or 29.97 f/s, so that I can feed this script to Virtualdub and encode to XviD. Been banging my head for a few days over this one, so thought that it might be time to check if any of the avisynth gurus here can help me out.
I have attached a small clip from one of the shows (taken from the very beginning of the VOB so as not to disrupt frame order) to show the kind of interlacing I'm talking about (used mpg2cut2 to cut from the original VOB). Playing the clip in VLC will easily show the interlacing issues that I'm dealing with. If a larger clip is required to work with, please let me know, and I will upload something a bit bigger to MegaUpload or a similar site.
Thanks in advance for your help ... always muchly appreciated

bean.mpg

[manono]

Either:

Yadif(Mode=1,Order=1)#or your favorite bobber
SRestore()

or:

Yadif(Mode=1,Order=1)#or your favorite bobber
RePAL()

Forget about cleaning up all the lines-outside-the-lines and ghosting.

...and give me a progressive output at either 23.976 or 29.97 f/s, so that I can feed this script to Virtualdub and encode to XviD.

Since it's to be an XviD, and since the native framerate is neither 23.976fps nor 29.97fps, what difference does it make if it's not one of those? If you change it to something other than what it comes out of the unblender as (with AssumeFPS(xx.yyy)), you'll also have to stretch the audio.

[poisondeathray]

I don't get how can they release crap like that (and charge money for it)?

[rickydavao]

@ poisondeathray - Tell me about it!

@ manono - Yep, been there, done that. I'm beginning to wonder if the NTSC release of these shows is completely unsalvageable with regards to getting a "relatively clean" progressive file (regardless of final frame rate, anyway).

Using manono's script suggestions, there is still, unfortunately, a lot of ghosting like in the first 2 images below, and some blurring and artifacts as in the 3rd image. I'm beginning to think there may be no way to fix this ... *sigh*


(@ manono ... finished this posting after your 2nd post ... was having trouble uploading the images ... )


EDIT: On further exploration, it appears that the issues in the screenshots below are being generated by the Yadif(Mode=1,Order=1) line in the script (I commented out the srestore() line), as they are easily visible in a frame-by-frame viewing in Virtualdub. Is it possible that the yadif parameters need to be tweaked, or is another bobber in order? I will try experimenting in the meantime ...

[manono]

I'm beginning to wonder if the NTSC release of these shows is completely unsalvageable with regards to getting a progressive file (regardless of final frame rate, anyway).

There's no interlacing left. Therefore it is progressive. The unblenders by definition deinterlace it. You may not like the results in this particular case, but don't blame the unblenders for that. Just put on a bobber alone and go through it frame by frame. The unblenders depend on the majority of the fields to be "clean". With this thing a great majority are blended.

[jagabo]

Maybe you can just perform a simple Bob() and encode it at 59.94 fps. Will the Xbox 360 play that? It will still look like crap but shouldn't look much worse than watching the DVD directly.

[poisondeathray]

The blends aren't caused by yadif, those are in the source

But the little dots and jaggie artifacts are remenants caused by deinterlacing with yadif, and you could use a better deinterlacer to get rid of those (e.g. yadifmod+nnedi2) , but it may make blends worse in some frames. eg. (look at mr.beans' face)

yadif(bobbed) + srestore


yadifmod+nnedi2 (bobbed) + srestore

[rickydavao]

Thanks everyone for all of your help so far. I've been having a little bit of success with a simpler script, as follows:

AssumeTFF()
Yadif(Mode=3,Order=1)#or your favorite bobber
blendbob()

... after which I put a denoiser and moderate sharpener for a bit of cleanup. Didn't use srestore() at all ... is it really necessary? Found that this script gave me a relatively clean 29.97 f/s output compared to what I was seeing before. However, poisondeathray, I would be interested in trying your "yadifmod+nnedi2 (bobbed) + srestore " script as well to see if I can improve my result at all ... as I'm not as familiar with those avisynth commands as I'd like to be, could you possibly post the script you used to obtain the results in your post? Thanks!

Here's the frame that you posted using the yadif and blendbob script from above:

[manono]

Didn't use srestore() at all ... is it really necessary?

To help unblend it, some kind of unblender is necessary. All you've done there is increase the amount of blending. You've also increased the framerate, making the video less compressible for the same file size.

However, some people like the look of blended videos. I don't happen to be one of them. I spend my life unblending fieldblended garbage and wouldn't purposely ever create more blending.

[lowellriggsiam]

Have you considered a different method, like dvddecrypter ->ISO -> Fairusewizard->XVID?

[rickydavao]

With apologies to manono, after having done 2 encodes at the same bitrate - the first with my yadif-blendbob script, and the second with the yadif-srestore script - while the yadif-blendbob script does produce a higher framerate file (29.97 f/s), the fact that there appear to be fewer "interlace-type" artifacts in the final encode with this method (especially given how prominent the black outlining is in this cartoon) simply looks a bit better, to me anyway. I have attached a 40-second clip of the final result, encoded to XviD VAQ build (total average bitrate for the complete file was 1200kb/s) to give you an idea of how it turned out. There is still a bit of shaking, especially in some horizontal lines, but looking back to the original DVD files, this shaking was evident there as well. The script has gotten rid of some of the jaggies evident in the original vob, and the denoisers and sharpeners helped to clean the image up a bit. So I'm left with a file that, at least to my eye, is a bit sharper than what I started off with, and at least no worse for the jaggies that were in the original either (I hope this makes sense). I'm thinking that this is probably the best I can do.
Further ideas always welcome, of course, and thanks for everyone's help & suggestions !

mr.bean.avi

[poisondeathray]

I just suggested a better deinterlacer than yadif in bob mode, because yadif tends to leave those jaggy dot artifacts.

I've never heard of blendbob before, but if it's giving you better results , great

LoadPlugin("PATH\yadifmod.dll")
MPEG2Source("bean.d2v")
interp=nnedi2(field=3) #double rate, keep top field
yadifmod(order=1, field=-1, mode=1, edeint=interp) #TFF, field set to order, double rate
Srestore()

[jagabo]

yadif-blendbob script... mr.bean.avi

The problem with that conversion is that every 6th frame is a repeat. That creates small jerks, 5 times a second. Watch the smooth zoom/pan shot 18 to 21 seconds into your clip. Yadif/srestore would get rid of those duplicate frames, in theory. But your source is in such poor shape it may not work well.

That shot from your clip after TDecimate(mode=2, rate=25):

25fps.avi

[lordsmurf]

Time out.... reality check time....

so that I can stream it off my PC to the kids' bedroom on their XBox 360.

... is all this work really worth it?

I'm all for quality video, but this isn't the master copy (the DVD is), it's for an audience that is often happy watching crap on an iPod or Youtube, and is usually given 50% attention at best (also playing toys, doing homework, etc).

So again ..... worth it?

Any decent quick method would work. Personally, I'd run it through VirtualDub, use the Deinterlace Area-Based filter on it, and then frameserve it over to the encoder. I have MainConcept Reference, but I know few here can afford that option. Surely some consumer app will work, however.

Just saying...

I'm a big "keep it in perspective" person.

[rickydavao]

Happy "Day after Hallowe'en" everyone! (too much candy ...)

@ lordsmurf ... yeah, I completely understand where you're coming from, but it's this darn OCD! Actually, I'm not kidding .... had it since I was a kid building plastic model kits and even model rockets (!), and it doesn't look like it's going away any time soon. Even though it's just for the kids, I'm just not happy with producing less than the best quality I can with any particular destination format. I guess I also look at it as a challenge, and its kind of fun that way, too. Also, once the puzzle is solved, that solution is there for me should I ever need it again, and also available through the forums should anyone visiting VideoHelp require the answer to a similar problem. Life would be dull if there wasn't anything new to learn :P

@ jagobo ... thanks for the tip with regards to tdecimate. I hadn't noticed those "small jerks" until you pointed them out to me, most likely because I was concentrating more on the quality of the individual frames. Good info and works a treat.

@ manono ... thanks for insisting that more blending is not necessarily a good thing (how did I not notice that "BLENDbob" would probably blend more? Duh!). Which pushed me to try ...

@ poisondeathray ... thanks so much for the avisynth script! I did a few quick runs this morning (short segments with CQ encoding) and the results look very similar to the encode I got using yadif-blendbob with the notable exception that the encoded video looked "cleaner". There were still small comb-type artifacts, but as I mentioned earlier, these are evident in the original VOBs (crappy source, I know). However, the yadif-blendbob encode just looked "smoother" (no doubt due to belndbob, as manono pointed out). Your script gives a better result overall, and no doubt closer to what it should look like. I will post a short clip from the encode I'm running now as soon as it's done so you can see the result.

Thanks again, all!

[rickydavao]

As promised, here is the final result (or at least a short segment thereof). This was encoded with XviD VAQ using an average bitrate of 1200kb/s, using the script provided by poisondeathray (I added some moderate denoisers and sharpeners specific for cartoon/anime). The encode takes a while due to the nnedi2 and srestore, but the results are pretty good. Working on the next episode now and hopefully that will turn out at least as well.

Thanks again for everyone's help, and let me know what you think of the result if you can spare a minute

Cheers!

mr.bean.avi

[jagabo]

That's pretty clean considering the source you started with. It's not clear whether the frame rate should be 24 or 25 fps. You'll need to look at a few more smooth panning shots to see which works better.

[manono]

It's not clear whether the frame rate should be 24 or 25 fps.

But it is clear which it is. It's PAL2NTSC. Another thing that's clear is that it's not 23.976fps as rickydavao made the AVI (assuming he didn't just slow it down after unblending to the correct framerate). However, since it's crude animation with lots of repeated frames in most parts, the fact that one frame is dropped every second might not be noticed. However, I believe he should encode it to the correct framerate, rather than forcing it to some framerate he wants, for whatever reason he might want that framerate.

It also means he didn't follow poisondeathray's script as he said he did since SRestore at default settings (as in pdr's script) gives 25fps.

[jagabo]

It's not clear whether the frame rate should be 24 or 25 fps.

But it is clear which it is. It's PAL2NTSC.

You're simply assuming that. And that assumption is probably correct. But what I meant was, looking at his small sample it wasn't clear if the frame rate was 24 or 25 fps. The only section with smooth motion was too short to say for sure.

[manono]

It's not clear whether the frame rate should be 24 or 25 fps.

But it is clear which it is. It's PAL2NTSC.

You're simply assuming that. And that assumption is probably correct. But what I meant was, looking at his small sample it wasn't clear if the frame rate was 24 or 25 fps.

I wasn't assuming anything. To which sample are you referring? His 23.976fps AVI? Then yes, it's difficult to say. I was going by the unprocessed bean.mpg from the first post, and with that one it's quite clear what the unblended framerate is.

[rickydavao]

@ jagobo & manono ... thank-you for your insights.

@ manono ... you are correct that I slowed it down after unblending (yes, I did follow poisondeathray's script) to 23.976 fps using assumefps. We also have an old Philips DVP642 in another bedroom that we may want to play these on at some point (we're all big Mr. Bean fans), and I had assumed (probably mistakenly) that these files would play best if they were encoded to a standard NTSC frame rate. If the Philips will handle 25 fps OK, then there is obviously no need to do this, and I will remove the assumefps from my script accordingly.

I would guess that it is PAL2NTSC since the show was made in England. But I'm curious about jagobo's comment as to whether the frame rate was 24 or 25 fps. I have attached two identical clips to this post with what I hope is a good section of panning. The first was encoded with:

interp=nnedi2(field=3) #double rate, keep top field
yadifmod(order=1, field=-1, mode=1, edeint=interp)
Srestore()

as suggested by poisondeathray. The resulting frame rate is 25fps. The second clip was encoded using:

AssumeTFF()
Yadif(Mode=3,Order=1)
blendbob()
TDecimate(mode=2, rate=25)

as partially suggested by jagobo. The resulting frame rate is also 25fps. Does looking at these clips help determine whether the frame rate was 24 or 25fps? Also, which clip to your mind looks better? Needless to say, the second clip was a lot quicker to encode. As always, your thoughts are sincerely appreciated

bean1.avi

bean2.avi

[jagabo]

I wasn't assuming anything... I was going by the unprocessed bean.mpg from the first post, and with that one it's quite clear what the unblended framerate is.

Really? So, from that short MPG sample, you're absolutely sure that a 24 fps source wasn't converted to 25 fps by duplicating every 24th frame, or via 2:2:2:2:2:2:2:2:2:2:2:3 pulldown, or through field blending? (And subsequently convert to 29.97 fps interlaced NTSC via field blending.)

Rickydavao, bean1.avi won't download. My guess is it was too close to the 6 MB limit. Bean2.avi had duplicate frames at 3,4 and 145,146. But I think you can safely assume the frame rate should be 25 fps.

[jman98]

We also have an old Philips DVP642 in another bedroom that we may want to play these on at some point (we're all big Mr. Bean fans), and I had assumed (probably mistakenly) that these files would play best if they were encoded to a standard NTSC frame rate. If the Philips will handle 25 fps OK, then there is obviously no need to do this, and I will remove the assumefps from my script accordingly.

I have the DVP642 as well and yes, it will play 25 fps Divx fine. However, please pay attention to this sticky as each of these potential problems will bring this player to a stop.
https://forum.videohelp.com/topic352457.html

Do you place no value on your spare time? I'd have just bought the kids a Western Digital media player and hooked it up to the TV rather than doing all this time consuming re-encoding just so you can stream it. I doubt you'd have to make any changes to your files with one of the Western Digital players.

[rickydavao]

@ jagobo ... sorry about that! I have edited the post above, trimmed the original file and re-uploaded a new "bean1.avi". Should be OK now .... thanks for the heads-up!

@ jman98 ... thanks for the info regarding the DVP642, as I just wasn't sure. I have read that sticky before, so am aware of the other limitations, but appreciate the reminder. And while it can be time-consuming, as you say, it is also a hobby. I have managed to answer a lot of previous questions on my own using the information provided on the wonderful forums here, and a part of it is that I simply enjoy a good puzzle/challenge ... which is what this has been. To each their own, I guess

[jagabo]

@ jagobo ... sorry about that! I have edited the post above, trimmed the original file and re-uploaded a new "bean1.avi". Should be OK now .... thanks for the heads-up!

As far as I could tell, bean1.avi had no duplicate or dropped frames in the smooth panning shot, the first ~6 seconds of the video. It also had a lot less ghosting than bean2.avi.

Regarding the DVP-642, I seem to recall it plays both 24 and 25 fps Divx AVI files by duplicating frames to make 29.97 fps. That makes playback of both of them a bit jerky.

[manono]

Really? So, from that short MPG sample, you're absolutely sure that a 24 fps source wasn't converted to 25 fps by duplicating every 24th frame, or via 2:2:2:2:2:2:2:2:2:2:2:3 pulldown, or through field blending? (And subsequently convert to 29.97 fps interlaced NTSC via field blending.)

Heck, you proved it to yourself already that 25fps PAL was converted to NTSC by fieldblending:

The problem with that conversion is that every 6th frame is a repeat.

Why it doesn't completely unblend I'm not sure. It does seem to be doubly blended, so maybe it was originally created at 24fps and field-blended to get to PAL. There's nothing we can do about that at this point, though. We can only remove one layer of blending.

[jagabo]

Heck, you proved it to yourself already that 25fps PAL was converted to NTSC by fieldblending:

The problem with that conversion is that every 6th frame is a repeat.

That was based on a very short panning sequence (less than a second) so I don't consider it definitive. The latest clip with about 6 seconds of smooth scrolling at 25 fps with no duplicate frames and no dropped frames makes it pretty clear.