IVTC'ing NTSC source with random pattern

[miggeth]

Hi there,

If anyone can help me with this it would be great. I'm primarily working with music video's. These tend to be edited after they have been telecined, resulting in an interlace pattern that can sometimes be pretty random. Is there any way of restoring them to 24fps progressive?

Gav.

[manono]

Hi-

Is there any way of restoring them to 24fps progressive?

That assumes the entire thing is 24fps to begin with, and when editing they sometimes add in progressive or interlaced 29.97fps footage and effects, and with overlays sometimes even have both framerates playing at the same time.

Have you tried to IVTC them?

You might upload a small sample so we can have a look. I'll assume you have some AviSynth skills, as that's the only way you'll be able to "fix" it. And you might read up on some work going on to improve similarly difficult anime videos:

http://forum.doom9.org/showthread.php?t=138305

[miggeth]

Yes, you're right. Occasionally music videos are a hybrid of both framerates. That's not the problem I'm having, the problem is, all the combing filters I've tried leave duplicate frames and this obviously causes missing frames later on. This happens especially at scene changes.

I've been using Donald Graft's decomb plugin with AVIsynth, apparently this is the best one, but it's so damn complicated I don't really know where to begin. If anyone know's what settings are generally best for this kind of operation, I would be grateful.

Also, I've found a few sites that discuss the problems of decombing anime, it sounds like a similar thing so maybe I'll get somewhere with that?

Thanks for that manono.

[SLK001]

...the problem is, all the combing filters I've tried leave duplicate frames and this obviously causes missing frames later on.

After you perform an IVTC, you also have to run a decimation filter, which will remove usually 1 frame out of five (and removing the frame most like one of the others).

You might want to try the fielddeinterlace filter of DG's decomb package and just skip the IVTC of the random telecining. Something like this works wonders:

mpegsource("yourvideo.d2v")
fielddeinterlace()
converttoyuv2()

This will blend the interlaced fields so that at playback, the lines will not be seen.

[miggeth]

SLK001, thanks for the advice, I didn't forget to decimate, I'm talking about the odd frame here and there, especially at scene changes. I'm pretty sure this can be done with the right settings, but decomb has a lot of settings, and I don't really know what I'm doing yet.

It can get pretty frustrating messing about with settings for hours, when maybe someone know's already.

I'm not a fan of blending, especially with 3:2 pulldown. As you probably know, if you blend such material you end up with all the lines shown at the same time, when they were meant to be shown one after the other. This causes inaccurate motion, not to mention the obvious ghosting.

I appreciate all suggestions though, keep them coming please.

[gadgetguy]

I'm curious as to why you feel you need to deinterlace. What is your goal with this video? IMO the only way to do this correctly is manually. You can use avisynth filters to restore each manually selected segment of telecine, but there will still be errors because video that has been edited after telecine is often missing frame elements depending on where the cut was made. Undoing scene transitions, (especially crossfades), of telecined material is almost impossible to restore. And if any of the original footage is true interlaced, anything you do to deinterlace will significantly reduce quality for that segment when compared to the IVTC material. Personally, I have never seen any footage worth that much effort for something that can readily be viewed in it's current state. So I come back to my original questions... Why do you need to deinterlace? What is your goal?

[SLK001]

You wouldn't apply a 3:2 pulldown if you blend your original source (since I'm assuming that it is 29.97fps).

Usually, sources with this random TCing pattern are almost impossible to get correct via an IVTCing process. Unless you are willing to break up your video into chunks, then analyze what parameters that your decomb filter will need for each chunk, then process, then encode, then rejoin, THEN apply a pulldown, I recommend that you just treat the video as a 29.97 source and simply blend the telecined frames. On a TV at 29.97, the blending is undetectable. Even on a computer display, the blending will look as a scene fade (you won't see any interlacing lines).

Although I hate having to do this (and the house is filled with my opinion of the film "butcherer") I have had to do this on occasion. I find that fielddeinterlace provides excellent results for me.

[edDV]

I'm curious as to why you feel you need to deinterlace. What is your goal with this video? IMO the only way to do this correctly is manually.

My thoughts exactly. I would start by separating clips and working on them individually. Some will IVTC, others (including transitions) may need bob, blend or other, then framerate change. Some composites may need rotoscoping and recompositing.

Until ~2005, most music videos were edited at 29.97 480i or 1080i. 25fps PAL was usually edited separately for the big labels. Others were electronically transcoded.

[vhelp]

What I don't understand is the logic behind editing after the telecine. Thats always stumped
me something fearse. You have a 24p and you need to edit the video. Why wait till you have
a telecine to edit. I'll never understand.. never. Anyway.

When you have a segment scene, say 10 seconds, its safe to assume you have 3:2 pattern
of this. The only problems to this is the Start and End points because as you arealy are aware,
there are edits throughout the video. And some of these edits are In and Around the scene
change. These edits can be anywhere inside a 3:2 pattern (ie, PPPii) and if the cut does not
start at the first "P" or ending "i" you have problems. And the next best thing with this is to
blend/deinterlace those points between those cuts. To do this requires careful analysis of the
interupted 3:2 pattern and a unique calculation of the best method to apply to those fields to
make a best frame match or 24p "stitch" inside those cuts to similate fluid scene changes at
a 24p fps perspective.

But the above is not always closely done accurately to the point of undetection. As such, I"ve
always wondered if there was way to apply a special kind of pulldown flag to just those "stitched"
areas in order to "fake" a 3:2 or other type of pulldown for the mpeg. I thought that maybe the
pulldown tool could be adjusted to feature such an option. What's one more field to check/click
on

Other than that, if these videos are for standard dvd playback, then it is best to just process
those as 29.970 and use a high bitrate strategy.

If an up-conversion is in your thoughts (for future HD type projects) keep the origianl source
near by and keep reading here and there for day-to-day new ideas.

-vhelp 4707

[edDV]

I think he is trying to reverse the production process. If he is successful doing this I will be impressed indeed.

[manono]

I think he must be trying to deinterlace for something like XviD or DivX AVI or some other format where having it progressive is, if not a must, at least preferred. And he's doing it using Decomb which is far from the best IVTC to use for problematic sources such as music videos.

If he is successful doing this I will be impressed indeed.

It can be difficult, but not impossible. That's why I suggested miggeth upload a sample, so we could have a look and suggest solutions.

[jagabo]

It can be done manually by breaking the video into short clips (a break wherever there's a break in the telecine pattern) and handling each section separately. For example:

src = Mpeg2Source("file.d2v").ConvertToYUY2(interlaced=true)

v1 = trim(src, 0, 1234).SeparateFields().SelectEvery(10, 0,1, 2,3, 4,5, 7,8).Weave()
v2 = trim(src, 1235, 2345).SeparateFields().SelectEvery(10, 0,1, 3,4, 6,7, 8,9).Weave()
v3 = trim(src, 2346, 3456).SeparateFields().SelectEvery(10, 0,1, 2,3, 5,6 8,9).Weave()

return(v1 + v2 + v3)

The Trim() arguments reflect segments of continuous 3:2 pulldown pattern. The SelectEvery() arguments are specific for the pattern of each segment. Obviously, this is very tedious for a video with lots of breaks. There are a few other little details I won't go into...

I've done a few movies with dozens of breaks this way.

[miggeth]

Well, my purpose gadgetguy is to perform video cleaning operations, deblocking, aspect ratio changes, and obviously to get progressive frames (which look much better on large display surfaces). These things are much more difficult and sometimes don't work at all on sources that contain 3:2 pulldown material.

I don't want to deinterlace unnecessarily, I think you are confusing inverse-telecine with deinterlacing, a common mistake.

Thanks for all your input, I'm currently trying to use manono's suggestion from the doom9 forum. The AnimeIVTC plugin for AVISynth. It's not going well though, this is advanced stuff and I'm still fairly new to AVISynth, plus I think some software is missing from the AnimeIVTC pack. Have you used this manono, or any one else? Can you give me some tips how to get it working?

Here's an example of the types of material I'm working with, this is not a indicative of my musical tastes, It saves me from uploading something else. If anyone likes a challenge.

http://rapidshare.com/files/119246123/tbow.part1.rar
http://rapidshare.com/files/119246307/tbow.part2.rar

[manono]

Hi-

Have you used this manono, or any one else? Can you give me some tips how to get it working?

No, but I haven't worked on stuff that might need it in a while. As for tips, did you get both the guide (linked in the first line of the post) and the required filters (linked in the second line of the post)? And did you follow the instructions about sticking the .dlls in the AviSynth plugins folder? If that doesn't work, then add them to the script using a LoadPlugin line for each. Did you read the problems others had and how they were solved?

And finally, always test your scripts in VDubMod. If something's not working it'll give you a good error message (usually). If you have any specific problems you can't solve, either post here with the error or perhaps in the Doom9 thread. If Doom9, and if you haven't registered, you'll have to first register and then wait 5 days to post. But it does seem to me that the function he developed will work well with your stuff, as well as with anime.

I got the first RAR, but now have to wait an hour to DL the second one.

[gadgetguy]

Well, my purpose gadgetguy is to perform video cleaning operations, deblocking, aspect ratio changes, and obviously to get progressive frames (which look much better on large display surfaces). These things are much more difficult and sometimes don't work at all on sources that contain 3:2 pulldown material.

I don't want to deinterlace unnecessarily, I think you are confusing inverse-telecine with deinterlacing, a common mistake.

For the record, I have not confused Interlace with Telecine. Once a progressive source has been Telecined, it is Interlaced video even though most of the frames still contain progressive information. Interlaced material that has been Telecined, can be inverse-telecined if all of the original frames are present. If cuts were made where fields from different frames were combined, part of the original progressive frame is missing and cannot be completely restored. It's not clear if all of your source material was shot on film or digitally, or CGI, therefore some of the footage could be from an interlaced digital source. (@vhelp - In my experience, Telecine before edits is used to combine progressive footage with Interlaced material.) To get that to a pseudo-progressive state it must be de-interlaced, not IVTCed. Deinterlaced footage always takes a quality hit, while Telecined material is restored via IVTC.

Not to parse words, but your purpose is not to perform video cleaning operations. If I read you correctly, your purpose is to try to improve the video for viewing on a large screen TV, (that's the answer I was looking for). Performing video cleaning operations is the means to that purpose. I'm not trying to start an argument, or even contest what you're doing, only that I didn't understand what purpose would make this much effort worthwhile and wasn't sure you understood the challenge you were looking to undertake. I now think you fully understand what it's going to take and have decided it's worth it. It looks like you're well on your way and I wish you well.

[jagabo]

That sample video had a lot of telecine pattern breaks. Yadif did an ok job with it -- except for a few shots. Here's one of the bad shots along with a manually IVTC'd example (Xvid AVI):

yadif.avi
manual.avi

[miggeth]

manono, I got all the plugins and the guide, put them all in AVISynths folder, it didn't recognise the function for one of the plugins. That AnimeIVTC uses about eight different plugins I think.

Yeah jagabo, that's the problem I'm having, and this is a relatively good one by Promo Only standards. I'll check out the videos you posted when I get home. When you say manual, does that mean you went through it frame by frame?

[jagabo]

When you say manual, does that mean you went through it frame by frame?

Not frame by frame but I use two instances of VirtualDub. One with only MPEG2Source() so I can see the raw frame numbers and the other to check progress of the IVTC. I figure out what the telecine pattern is at the beginning of the movie and use the appropriate SelectEvery() settings for the entire film:

v1 = trim(src, 0, 0).SeparateFields().SelectEvery(...).Weave()
return(v1)

Then I open that script in the second VirtualDub and scan until I start seeing comb artifacts again. I identify exactly what frame number that is in the first VirtualDub and update the script. Say the first interlaced frame appears at frame 1000:

v1 = trim(src, 0, 999).SeparateFields().SelectEvery(...).Weave()
v2 = trim(src, 1000, 0).SeparateFields().SelectEvery(...).Weave()
return(v1+v2)

I then open that script in the second VirtualDub and scan for the next pattern break. Repeat the process until the video is complete.

If there are two many pattern breaks I usually give up and just use an automated IVTC. This process isn't quite perfect. Sometimes there's an orphaned field at the start or end of a section so you'll end up with an interlaced frame at the cut. But this is only displayed for 1/24 of a second so it's not highly annoying. Certainly better than the type of artifact in the sample I posted earlier.

[manono]

Hi,

I decided not to use that AnimeIVTC. After everything's set up, it encodes at about 2fps. Slow as Christmas, and only for the real perfectionist, which I'm not.

As jagabo said, this one's kind of a mess. Widescreen 4:3, edge enhanced, shifted chroma, and the usual crap you get when you edit on video - broken cadences all over the place. I used this script:

LoadPlugin("D:\AviSynth Stuff\Dlls\DGDecode.dll")
LoadPlugin("D:\AviSynth Stuff\Dlls\TIVTC.dll")
LoadPlugin("D:\AviSynth Stuff\Dlls\VinVerse.dll")
MPEG2Source("E:\Test\Test.d2v")
TFM()
TDecimate()
VinVerse()
A=Last
B=A
Overlay(A,B,Mode="Chroma",X=-2)
crop(14,102,-14,-104)
LanczosResize(624,272)
FreezeFrame(602,603,602)
FreezeFrame(604,605,605)
FreezeFrame(2072,2073,2072)
FreezeFrame(2235,2236,2235)
FreezeFrame(2237,2239,2239)
FreezeFrame(2831,2832,2831)

Vinverse is supposed to clean up any leftover interlacing. It probably didn't do anything, but I don't think it hurt, either. The Overlay stuff shifted the colors back into place. You may have noticed blue-gray "lines" where the black met lighter colors, like along the black straps of her blouse. I didn't bother fixing the edge enhancement. The Freezeframes were to fix some leftover garbage in the car scenes and another place which had some ugly looking frames. There may still be problematic areas, and there are quite a few duplicate frames, but it's otherwise not too bad. Here's a link to a 25MB XviD AVI of it:

http://www.mediafire.com/?zxxmotlyo9l

Nice video though.

[jagabo]

The simple automated IVTC I did was:

Load_Stdcall_plugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll")
MPEG2Source("filename.d2v")
ConvertToYUY2(interlaced=true)
AssumeTFF()
Yadif(mode=0, order=1)
TDecimate()

[miggeth]

Thanks for explaining what you did manono, that's a fairly good encode too. I only know of one person who can get these video's back to progressive perfectly, and she puts a lot of work into them I suspect. Also she won't tell me how.

So, would you say TFM was better at IVTC than Donald Graft's Decomb plugin?

[jagabo]

I only know of one person who can get these video's back to progressive perfectly

Actually there is no way to do it perfectly. Consider the three frame sequence:

A B C

Where each letter is a film frame. Telecined with 3:2 pulldown this becomes (upper case is top field, lower case is bottom field):

Aa Ab Bc Cc

Now suppose we cut just before the frame Bc

Aa Ab

and we join that with another segment that looks like Gh Hh

Aa Ab Gh Hh

There is no way to restore film frames B or G because only one of thier fields appears in the remaining video.

This is one of the "other little details" I didn't bother mentioning earlier.

[miggeth]

Most frames can be recovered though, and then you could cut the few incomplete ones in Vegas and stretch the audio accordingly. Much effort, you should see her output though, something tells me it can be done some other way.

[jagabo]

Most frames can be recovered though, and then you could cut the few incomplete ones in Vegas and stretch the audio accordingly.

Why not just do it in AviSynth while your making the IVTC script? You could duplicate the previous or next frame, bob one of the orphaned fields, or blend the two orphaned fields together.

[manono]

I only know of one person who can get these video's back to progressive perfectly...
So, would you say TFM was better at IVTC than Donald Graft's Decomb plugin?

I agree with jagabo; you can't do it exactly perfectly. You can make it so the average Joe doesn't spot anything out of the ordinary, but experienced encoders can spot any compromises that might have been made. For example - those broken cadences and orphaned fields - you can blend them to hide the problem, or you can bob them, or you can create duplicate frames. I go for dupe frames as they aren't quite as noticeable as are deleted or missing frames, or as blending, and bobbing some of those in the video didn't look so good. This is especially true for music videos where the cuts and edits come so fast that not-quite-smooth playback won't be noticed. And if the dupe frames are at scene changes, as they usually are, then no one will notice when watching. And yes, I believe TIVTC to be a better IVTC than Decomb for problematic sources.

Most frames can be recovered though, and then you could cut the few incomplete ones in Vegas and stretch the audio accordingly.

Again I agree with jagabo. Changing the length of the video opens a whole other can of worms as far as the audio goes. I say keep it the same length and duplicate any problematic frames with the previous or next good one. I showed you how in my script. That's perhaps the easiest way. Bobbing just a few stray fields here and there is more difficult and takes more work, but is also doable. In the case of your video, though, some of those fields looked like hell when bobbed. I'm mainly referring to some of the car scenes. That's why I chose to FreezeFrame them with the previous or next frame.

[jagabo]

Oh, by the way, the sample script I gave a few days ago wasn't specifically for your video. The Trim() values were just random number off the top of my head.

[thetoof]

Hi

I ran into this thread by accident as I was searching for a plugin and I somehow felt compelled to post. I dled your source for the fun of it and got very good results with

animeivtc(mode=1,aa=1,killcomb=1)

You can get the latest version here: http://forum.doom9.org/showthread.php?t=138305
Sorry if linking to a post I made in another forum goes against one of the rules of this one... have fun deleting my post if it does

[manono]

Sorry if linking to a post I made in another forum goes against one of the rules of this one.

Not at all. Glad to have you here. Stop by more often.

[miggeth]

Thanks for taking the time to check it out with AnimeIVTC, I had high hopes for that, but I could never get it to work for some reason. I don't think I put everything in the AVISynth folder properly. I was confused whether to copy just the .dll's or all the files and folders, or to take all the files out the folders and place them in the AVISynth folder on their own. I gave up eventually, since I'm pretty new to AVISynth and the install instructions are clearly written for experts, it was too confusing.

Anyway, if you want to see what I mean by perfect encodes from this kind of material, head over to http://www.keitex.net/forum/, and look for videos posted by Ntaty. I've never spotted blended frames or duplicates, and I've gone through some of them frame by frame, in parts. See what you think.