Weird telecine format?

[ZetaStax]

I'm in the process of ripping my Seinfeld DVDs, but noticed a very strange telecine pattern on some episodes. Most of the time a standard 3 progressive 2 interlaced pattern is visible, but usually on camera cuts the pattern is cut short and gets out of sync, so it can be 3-2-3-2-2-3-2-3-2 for instance. Needless to say the usual inverse telecine filters do not work properly and I'd really like to avoid having to resort to decomb. What would be the best way to tackle this case?

I tried cutting a sample but it resulted in a few extra duplicate frames... until I used -vsync 2 with ffmpeg. So I assume there are some weird timestamps in the source. Anyways, here is the sample https://mega.nz/#!spBR0ATR!zvZQ9VUoEcj5b_pEO1U4jByggZ6Y0MNFvsaJwCKMRfg

[Sharc]

I did not check your file, but isn't your pattern a regular 2-2-3-2-3 (or cyclic shift of it like 3-2-3-2-2) which used for 25fps to 30 (29.97, NTSC) fps telecining?
If it is soft telecined you can just ignore the flags and should get 25fps progressive video.

[ZetaStax]

Hmm not sure. It does some weird thing where there can be 1 or 3 interlaced frames back to back instead of shifting the 3-2 pattern, but I'm having trouble correctly moving forward frame by frame with vlc sometimes it moves ahead 2 frames for no apparent reason.

[jagabo]

3:2:3:2:2 pulldown is one method used to create 59.94 fields per second (29.97 frames per second) from 25 fps progressive sources. 5 progressive frames are converted to 12 fields, 25 frames to 60 fields per second. Then one duplicate is dropped every 1001 frames to create 59.94 fields per second (29.97 fps). The usual restoration method is to field match then discard duplicates, leaving 25 fps.

[JVRaines]

Try the TFM adaptive filter in Avisynth.

[jagabo]

For simple 3:2:3:2:2 pulldown it's usually TFM().TDecimate(cycle=6, cycleR=1), leaving 24.975 fps. Since it's usually a film source some people like to AssumeFPS(24) or AssumeFPS(24000,1001) to restore the original running time.

[manono]

A simple TFM().TDecimate() does the job.

As is common with these things, it was edited as video and there's nothing PAL about it. You wind up with duplicate frames at scene changes sometimes but otherwise it IVTC's well.

And next time please upload an M2V from the DVD, and not a useless MKV.

[jagabo]

Weird. Somehow I missed the link to mega.nz in the first post. Yes, the video is normal 3:2 pulldown with discontinuities at shot changes.

[ZetaStax]

Alas, none of these solutions are working. TFM + TDecimate result in some jumps in the video, in the very beginning. After that it's okay.

TFM().TDecimate() was the worst, frames looping all over. TFM().TDecimate(cycle=6, cycleR=1) was better, but a noticeable jump remains.

And next time please upload an M2V from the DVD, and not a useless MKV.

Care to elaborate? I have no tools to convert vob to m2v. How is mkv "useless".. ?

[jagabo]

The video can't be perfectly restored because it was edited as video after being hard telecined. So you will end up with occasional duplicate or missing frames at shot changes.

Mpeg2Source("sample_track1_eng.d2v", CPU2="ooooxx", Info=3)  # extracted with mkvextractgui, indexed with DgIndex
TFM(d2v="sample_track1_eng.d2v") 
TDecimate()

I suspect your problem arises from trying to work directly with the MKV file. Many tools have problems with MPEG2 video in MKV. With DVDs always work with the VOB files or an MPG file created with VOB2MPG.

[manono]

Alas, none of these solutions are working. TFM + TDecimate result in some jumps in the video, in the very beginning.

Nonsense. At the beginning of the sample or the beginning of that particular episode? All we have to go on is the sample and, as jagabo mentioned, no one in NTSC land and in their right mind works with MKVs made from DVDs. You're just inviting trouble, as perhaps you've discovered.

The only problem with the sample is orphaned fields caused by editing it as video. The result is dupe frames at scene changes sometimes and perhaps some missing frames at scene changes as well. But the result after IVTC plays smoothly.

Now, if you mean the beginning of the episode then, yes, sometimes the intros are done differently. But we'd need to see a sample to be sure.

Get and use the DGMPGDec package to learn how to work with DVDs. Read the included docs carefully. jagabo showed a possible sample script.

I have no tools to convert vob to m2v.

If you don't have DGIndex already, you should and it's included in the DGMPGDec package mentioned earlier. Open a VOB, use the [ and ] buttons to mark off a small section. Then File->Save Project and Demux Video. The video will be the M2V mentioned earlier. When creating the D2V file for future encoding, open all the VOBs at once for whatever project it is (maybe just one or two for a television episode) and File->Save Project and then you'll use the resulting D2V file in your script. It's all described thoroughly in the docs included with DGMPGDec.

[davexnet]

Can also use Avidemux to re-multiplex directly to mpeg program stream - seems to work well

[ZetaStax]

Nonsense. At the beginning of the sample or the beginning of that particular episode? All we have to go on is the sample and, as jagabo mentioned, no one in NTSC land and in their right mind works with MKVs made from DVDs. You're just inviting trouble, as perhaps you've discovered.

The beginning of the provided clip (which happens to be the beginning of the dvd title). I haven't touched the source in anyways and used MakeMKV to extract the dvd titles, this is the first time I'm seeing this issue, it seemed pretty reliable.

The only problem with the sample is orphaned fields caused by editing it as video. The result is dupe frames at scene changes sometimes and perhaps some missing frames at scene changes as well. But the result after IVTC plays smoothly.

I'm not sure I understand. How can IVTC even work if some frames have been edited? If the expected pattern is 3:2 and more than 2 interlaced frames happen back to back, how can the filter even recover? Is TIVTC working in a smart way, actually looking for interlaced frames?

Can also use Avidemux to re-multiplex directly to mpeg program stream - seems to work well

If this is possible then no information is lost in the MKV at least. But why this container causing an issue, it has always seemed to me that MKV was the de facto container.

[jagabo]

I'm not sure I understand. How can IVTC even work if some frames have been edited? If the expected pattern is 3:2 and more than 2 interlaced frames happen back to back, how can the filter even recover? Is TIVTC working in a smart way...?

Yes.

[manono]

I haven't touched the source in anyways and used MakeMKV to extract the dvd titles,

You're contradicting yourself. You repackaged the VOBs in such a way that whatever you used to perform the IVTC failed. Next time use a proper decrypter on your DVDs first followed by MPEG2Source (and DGDecode and the D2V project file made using DGIndex).

...this is the first time I'm seeing this issue...

Maybe you don't often work with hard telecine. It's time to learn how to do it correctly, since you seem to be working your way through a very long series.

But why this container causing an issue, it has always seemed to me that MKV was the de facto container.

It's fine for just playing videos. It's probably fine for reencoding PAL DVDs. But you're reencoding NTSC DVDs. I don't understand why myself, and don't much care as there's been a much better way to handle NTSC DVDs for many years. Many of the best developers are Europeans who might not fully understand telecine/IVTC and don't develop their products to be bulletproof against the problems NTSC people encounter every day.

[ZetaStax]

That's a lot to wrap your head around, but I've tried to follow jagabo's provided example but I'm getting a video that consists of a single green frame. Obviously I'm doing something wrong.

I extracted the m2v using mkvextract and indexed it using DGIndex. Now I'm running the following script and sending it to ffmpeg

Mpeg2Source(".\Seinfeld - Season 8 (Disc 4)_t02_Track01.d2v", CPU2="ooooxx", Info=3)
TFM(d2v=".\Seinfeld - Season 8 (Disc 4)_t02_Track01.d2v", slow=2)
TDecimate()
Prefetch(6)

I know it would be better using the original vob but since jagabo has had success with this method I wanted to try to replicate it.

[manono]

I know it would be better using the original vob but since jagabo has had success with this method I wanted to try to replicate it.

He couldn't use the VOB because he didn't have it. Save yourself some time and effort by using the DVD source.

Can you open the script in Virtual Dub without the green? If not, what's the error message? Did you test the script at all before doing the encoding? Are you using plain AviSynth or AviSynth +? If regular AviSynth, try leaving off the Prefetch line. Comment it out (don't use it) by putting a '#' in front of it like so:

Mpeg2Source(".\Seinfeld - Season 8 (Disc 4)_t02_Track01.d2v", CPU2="ooooxx", Info=3)
TFM(d2v=".\Seinfeld - Season 8 (Disc 4)_t02_Track01.d2v", slow=2)
TDecimate()
#Prefetch(6)

If that doesn't work, try simplifying the script even more:

Mpeg2Source("Seinfeld - Season 8 (Disc 4)_t02_Track01.d2v")
#TFM(d2v="Seinfeld - Season 8 (Disc 4)_t02_Track01.d2v")
#TDecimate()
#Prefetch(6)

Open it with just the MPEG2Source line and if that looks normal, remove the '#' one at a time, save and and reload the script (F2) until the reason for the error is discovered. You could also try simplifying the names as well, such as 'S1E2.d2v'

If the script shows as normal in VDub but only becomes green after being encoded in ffmpeg, then the problem lies there. You can open it in Virtual Dub2, choose and configure your codec there and also encode it.

[ZetaStax]

Even when I only leave the Mpeg2Source line VirtualDub gives me filter error 80040154. I don't know why it cannot see my plugins. I'm using AviSynth+ by the way. Even a working script cannot be imported, is it only compatible with regular AviSynth?

[jagabo]

You may have a 32 bit vs. 64 bit problem. Did you "install" the correct DgMpegDec.dll file?

https://forum.doom9.org/showthread.php?t=149764

32 bit programs will use 32 bit AviSynth and 32 bit filters. 64 bit programs will use 64 bit AviSynth and 64 bit filters. You can't use 32 bit filters in 64 bit AviSynth. You can't use 64 bit filters in 32 bit AviSynth.

[ZetaStax]

I'm using all x64 avisynth+ and plugins. And I used the DGMPGDec found here http://avisynth.nl/index.php/AviSynth%2B_x64_plugins so no idea why it isn't working

[jagabo]

Explicitly load the plugin in your script to see of loading the plugin succeeds. You'll get an error message if the plugin fails to load.

LoadPlugin("C:\path\to\DGDecode.dll")

Also, do not move/delete/rename the VOB files after creating the d2v. The d2v file is only an index. The VOB files must remain in place to get the actual video.

[ZetaStax]

Still no luck. I must have done something wrong when creating the d2v because the plugin loads correctly (ffmpeg does not give me an error, only VirtualDub). To be sure I also tried VirtualDub2 but same thing.

[manono]

Then have you already tried making a fresh D2V file?

Did you change anything in DGIndex, such as, perhaps, using its cropping tool? Ordinarily you'd leave everything at default except maybe changing the Field Operation when you have pure soft telecine.

[ZetaStax]

Sorry for the delay, I had some internet issues.

I loaded up the VOB directly into DGIndex and demuxed it, but the result is the same. I don't understand, the demuxed m2v looks good, so I suppose DGIndex is working correctly, but VirtualDub still cannot open the script even if simplified to the Mpeg2Source line, and encoding with ffmpeg still results in a single green frame.

I've also tried using Vob2Mpg but I'm stuck, the program is giving me error 0xD0000135 (I'm on Windows 10). I've tried compatibility mode and such but can't get it running.

[manono]

What's the complete script, again? And with just using the MPEG2Source line, the name in the line matches exactly the name of the D2V file? Attach the D2V file here.

You don't need to demux the VOB, only create the D2V project file. File->Save Project. But I suspect the problem lies either with AviSynth+ or with 32/64bit for which someone else can help. Maybe.

[ZetaStax]

The script is very simple

Mpeg2Source("VIDEO_TS.d2v")

That's it.

By the way I had to try a different DVD as the OP one was borrowed from a friend. It makes no matter, since the result is the same. The d2v file is filed with "92 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2" I have no idea how this format works but I assume it has to be related to frame data, and it does explain why the only thing I get is a green frame.

I will try a DGIndex x86. EDIT: Actually it seems there is only one version of DGIndex, welp.

[jagabo]

That looks normal for a d2v file. The file is plain text and contains no video. It's only an index to the VOB/MPG files. The VOB/MPG files must still be available because the video will be taken from there. Here's a random d2v from my computer:

DGIndexProjectFile16
5
L:\VIDEO_TS\VTS_01_1.VOB
L:\VIDEO_TS\VTS_01_2.VOB
L:\VIDEO_TS\VTS_01_3.VOB
L:\VIDEO_TS\VTS_01_4.VOB
L:\VIDEO_TS\VTS_01_5.VOB

Stream_Type=1
MPEG_Type=2
iDCT_Algorithm=6
YUVRGB_Scale=1
Luminance_Filter=0,0
Clipping=0,0,0,0
Aspect_Ratio=4:3
Picture_Size=720x576
Field_Operation=0
Frame_Rate=25000 (25/1)
Location=0,0,4,2a49a

d00 5 0 2048 0 1 1 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
d00 5 0 407552 0 1 1 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
d00 5 0 870400 0 1 1 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
d00 5 0 1331200 0 1 1 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
...
(about 10000 more lines like these)
...
d00 5 4 353662976 0 1 20 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
d00 5 4 354123776 0 1 20 b0 b0 90 b0 b0 a0 b0 b0 a0 b0 b0 a0
d00 5 4 354590720 0 1 20 b0 b0 90 a1
d00 6 4 354725888 0 2 1 d2 ff

FINISHED  100.00% VIDEO

Make sure the DgIndex and DgDecode are the same version.

[ZetaStax]

This is fairly equivalent to the d2v that I get. I believe the issue is that DgIndex is x86 only, I will need to reinstall all of my plugins + ffmpeg using x86 builds to see if that changes anything.

[jagabo]

I believe the issue is that DgIndex is x86 only

No. It only produces a text file. Both 32bit and 64 bit DgDecode can use the index files. I use both all the time.

[ZetaStax]

Well then I don't know what's the issue. I've installed avisynth+ x86, x86 plugins including DgDecode and now the script can be opened by VirtualDub, and ffmpeg x86 manages to encode it. So the issue probably lies with the DGDecode x64 build. This one http://avisynth.nl/index.php/AviSynth%2B_x64_plugins precisely.