VC-1 Interlaced endlessly repeating a frame.

[ndjamena]

I'm attempting to re-encode the Blu Ray Version of the Doctor Who complete specials. Doctor Who is recorded as 25p, these Blu Ray's have been authored for an international audience so have been converted to 29.97i using a process similar to Telecining and then encoded into Interlaced VC-1. This is the script I'm using.

 
DIRECTSHOWSOURCE("Video 1.avi")
TFM(mode=5,slow=2)
TDecimate(mode=2, rate=Float(25.0))
ChangeFPS(25)

At first I was using multithreading but that left the output with frames out of order (sometimes by several seconds) so I removed SetMTMode. Everything seems to go well, except that now, somewhere in the video, the video will just stop and the last frame will get repeated endlessly until the frame count is reached and that happens in all five specials. As a first guess I'd assume the problem would be VC-1 related, but I'm not sure. If it matters I have to Shark 007 codecs installed at default settings. I've already wasted over a day on this and since it takes 2 hours to process each special I could waste quite a lot more time trying to figure out what's going on (especially since I'm not sure where to start). I'm using VirtualDub and the VT Video Codec. Help...

[ndjamena]

OK, I just did a version with just DirectShowSource and nothing else and the problem was still there. I know next to nothing about codecs or demuxes so at this point I'm officially lost...

[El Heggunte]

Since you're using DirectShowSource, I suggest that you first create an appropriate GRF file in Graphstudio, and only then feed DirectShowSource with it. Reason: the default VC-1 decoder may have problems when used together with the default AVI splitter (wrong timestamps, for example). MPC-HC AVI Splitter is very good, and I recommend it. Besides, the latest builds of LAV Video should also support interlaced VC-1 via an up-to-date libavcodec. But you must be sure these filters are being used by DirectShowSource, in the first place.

http://avisynth.nl/index.php/DirectShowSource#Opening_GRF_files

And oh BTW, how did you create that AVI with VC-1? I hope you DON'T say, "with vc12avi".

[ndjamena]

Your in luck, I've never heard of "VC12avi"! MakeMKV -> MKVExtract, MKVExtract puts vc1 into avi containers by default. I've installed Graphstudio, opened the AVI file and it's given me a lot of options I have no idea what to do with. I don't have time to figure it out at the moment, I have places to be. I'll try when I get back but at a quick glance I have to say there's nothing there I recognise...

[El Heggunte]

Well, there is no good reason for preferring AVI to MKV, regarding VC-1 streams,
unless you know very well what you're doing.
And no, you should never ever let MKVextract wrap VC-1 in an AVI container.

While you don't find the time to use Graphstudio,
create a Video.mkv instead of a Video.avi, and try that Avisynth script again.

[manono]

Got a sample? A short one with steady movement? Ordinarily one doesn't use DirectShowSource if it can be avoided. And if it's really telecined as described then:

TFM().TDecimate(Mode=0,Cycle=6,CycleR=1)

should do the job. And you're sure they're not field-blended like many of the crappy DVD releases?

[El Heggunte]

Ordinarily one doesn't use DirectShowSource if it can be avoided.

Is there any recent build of FFMS2 that supports interlaced VC-1?
If the OP has an nVidia card, he could use DGDecNV instead of DirectShow.

[manono]

Is there any recent build of FFMS2 that supports interlaced VC-1?

It doesn't already? That's one reason I asked for a sample - so we could play around with it to try and come up with a better solution.

[ndjamena]

Sorry, I forgot to visit the site after reading the notification e-mail so wasn't informed of all the other replies. MKV works much better but I'm still having problems I'm trying to fix. Namely: I'm getting frames out of order. Right now, I'm attempting transferring straight from DirectShowSource to UT Video codec AVI to see if that helps.

And a sample:

http://www.mediafire.com/?b41hh1r97uriccj

[ndjamena]

Got a sample? A short one with steady movement? Ordinarily one doesn't use DirectShowSource if it can be avoided. And if it's really telecined as described then:

TFM().TDecimate(Mode=0,Cycle=6,CycleR=1)

should do the job. And you're sure they're not field-blended like many of the crappy DVD releases?

I'll answer this. There's five of them (The Next Doctor, Planet of the Dead, The Water of Mars and The End of Time (1) and (2)), 1,3,4 and 5 have duplicate fields to prop up the frame-rate to NTSC. However #2 is different. It's basically like if you lined up all 25 frames, created 35 intermediaries using blending then take the resulting sixty frames and interlace them into 30i. For that one I had to run the entire thing through QTGMC and then SRestore which is why there's such a delay between posts. As for the other four I ran them through TFM then since I couldn't seem to get TDecimate to output EXACTLY 25fps I used FDecimate instead but there really was no good threshold to use since some of the actual frames were lower than some of the duplicates, especially in long range shots with CGI backgrounds. Then after running them through X264 and trying to watch them I discovered the out of order frames. I'm hoping the out of order frames come from DSS and not the UT codec AVI intermediaries. Even so, the output doesn't seem all that great, I'm still left with duplicates causing stuttering and some of the TFM frames still look partly interlaced. I'm also hoping fixing the out of order frames fixes most of the other issues too and I'm thinking of using QTGMC as clip2 in TFM.

Or I could just give up and hope the $300 Series 1-7 complete blu ray set on has these in PAL format...

[manono]

As for the other four I ran them through TFM then since I couldn't seem to get TDecimate to output EXACTLY 25fps I used FDecimate instead but there really was no good threshold to use since some of the actual frames were lower than some of the duplicates, especially in long range shots with CGI backgrounds.

Try this then, if they have to be 25fps:

TFM()
TDecimate(m2PA=True,maxndl=6,Mode=2,Rate=25.0)

For me, El Heggunte was right as I can't even get the sample to open properly, so I have to go by what you said without being able to study the sample.

[El Heggunte]

[ intentionally left blank ]

[jagabo]

TFM()
TDecimate(m2PA=True,maxndl=6,Mode=2,Rate=25.0)

That worked well with the sample, except at the start. But that's very common. There is a problem at times with the interlaced chroma. Be sure to use AssumeTFF().

Keep in mind that DirectShowSource() performance will vary depending on what DirectShow filters you have installed.

[ndjamena]

Yes, I noticed the chroma problem, I realise I know nothing about how these things work but I figured it was caused by YV12 being converted between interlaced and progressive and that there was no way to undo it.

I'm trying manono's script. I tried it on the first UT converted avi with x264 first but it just hung with my cpu use stuck at about 40%. I thought there might be something wrong with the file so I tried it on another UT avi in VDub and the same thing happened. So I went back to the original VC1 MKV with DDS2/VDub and the same thing happened. I tried it with just DDS2 and it worked, I tried it with TFM as well and it worked but when I added TDecimate it hung again. Confused about Jagabo's post I tried it on the Sample and it worked. I have no idea what's going on there...???

[ndjamena]

Would 29 Black Frames in a row kill TDecimate at those settings?

[jagabo]

Would 29 Black Frames in a row kill TDecimate at those settings?

No.

[ndjamena]

Both video's that hang have 29 black frames at the start. Both of them hang with zero frames encoded. The PC sits there using 40% of the CPU, I've waited at least 5 minutes for it to kick in but it hasn't. It's also reading quite a lot from the file. I figure somehow there's not enough memory for it to hold the entire sequence so it's just endlessly recreating the frames. Does that sound plausible and how high can I set SetMemoryMax???

[jagabo]

Maybe try converting to an all i-frame intermediate first.

Opening the AVS script in VirtualDub maxed out at around 450 MB. In the x264 CLI encoder (slow preset) around 1 GB.

Try adding convertfps=true to DirectShowSource.

[El Heggunte]

^
^

Both video's that hang have 29 black frames at the start. Both of them hang with zero frames encoded.

What about using Trim(29,0)

[ndjamena]

^
^

Both video's that hang have 29 black frames at the start. Both of them hang with zero frames encoded.

What about using Trim(29,0)

I like that! It's brilliant! Of course I'd need to add 25 black frames before encoding for audio sync.


Jagabo, I thought UT Video codec was essientially all I-frames, although I used 'optimise for compression' and I'm not sure what that does. Aaaand, I'm pretty sure I can't use DirectShowSource with other filters or I get frames out of order, unless it caused from reading one of the UT video intermediaries using AVISOURCE...

http://www.mediafire.com/download/rmmecxn1l30hpht/Sample_2.mkv

and yes, I did use AssumeTFF (Although the most obvious out of sequence frames - the scene change between outside the base and inside are two separate frames in the source)

Trim...

[ndjamena]

Nope, still not working

AVISOURCE("Doctor Who (2005) - 00x09 - The Next Doctor.avi")
Trim(30,0)
TFM()
TDecimate(m2PA=True,maxndl=6,Mode=2,Rate=25.0)

It hangs, I'm pretty sure I just copied and pasted...

That's with the UT Video AVI, the same happens with the VC1 MKV using DirectShow source. Yet the sample works. This is weird.

[ndjamena]

I forgot AssumeTFF() but I've added it now and there's no improvement.

[El Heggunte]

Just to be on the safe side:

do you have *many* files in the default Avisynth plugins folder?

[jagabo]

Jagabo, I thought UT Video codec was essientially all I-frames, although I used 'optimise for compression' and I'm not sure what that does.

I missed that you already tried UT Video Codec.

Aaaand, I'm pretty sure I can't use DirectShowSource with other filters or I get frames out of order

Sometimes you can, sometimes you can't. It depends on what DirectShow filters you have installed and are using, file readers, file splitters, codecs, etc.

unless it caused from reading one of the UT video intermediaries using AVISOURCE...

Random seeking with AviSource and UT should be fine.

[ndjamena]

m2PA -
Will override the default read-ahead maximum of 100 for mode 2. This will allow the
one pass mode (metrics not available from an input file) to produce the same results
as if the metrics were available.
**NOTE: the cycle size could very well be in the 1000's or 10000's, so there is the
possibility setting m2PA=true could mean the processing will stall for quite
some time (5-10 minutes or more) when a new cycle starts!!! Please check the
largest cycle size that will be used using debug=true before setting m2PA=true!
true = override the default maximum
false = don't
Default: false (bool)

Do I really need that?

[jagabo]

m2PA - Do I really need that?

No. But it gets you more accurate results (ie, less duplicate or missing frames).

[ndjamena]

I'll have to make a queue using the VirtualDub command line or I'll sit here forever

[ndjamena]

OK, I just got a batch file running. I'll go to bed now and hopefully by the time I get up SOMETHING will have happened

[jagabo]

You could also try TDecimate(cycle=6, cycleR=1). That will leave you with 24.975 fps. You could use ChangeFPS(25) to get exactly 25 fps. That will only give you one duplicate frame every 40 seconds.

[manono]

It hangs, I'm pretty sure I just copied and pasted...

It's supposed to hang. Depending on the length of the video and how long before it finds a repeating cycle, it could take a while (maybe 10-15 minuutes for a complete movie?). Maybe better you test with a small section.

Or try jagabo's suggestion in the post previous to mine. Dupe frames often go unnoticed when playing the film at full speed.