Trying to transcode a DVD with a mix of soft- and hard-telecine

[Vostok]

I am trying to transcode a DVD that is driving me bonkers, because it has a mix of soft- and hard-telecine and NTSC video. It's driving me bonkers because in some places the content switches between these formats multiple times within a few seconds! I have never seen that before.

The DVD is 41 minutes long. According to

mplayer DVD.mpg -nosound -vo null -benchmark

there are 336 rate changes within that 40 minutes, between 24000/1001 fps progressive content and 30000/1001fps NTSC video (which is sometimes hard telecined video, and sometimes actual NTSC video).

My problem is that if I just run DVD.mpg through any transcoding program such as ffmpeg, Quicktime, MPEG Streamclip, DG Pulldown, tsMuxer or anything else, it does a completely rubbish job due to the frequent rate changes. Typically what happens is that the 24 fps soft-telecined progressive content is converted to 29.97 fps content that repeats some frames but omits others. It's not even an interlaced frame rate change. The result is a jerky mess.

I am at witt's end. Is there a set of parameters I can use with ffmpeg that can create a usable output video file, e.g. produce a 24 fps file that captures every original frame of the soft-telecined 24 fps material, and frame rate converts everything else to 24 fps?

Or vice versa, produce an interlaced 29.97 fps file that replicates exactly all the hard-telecined and NTSC video content, and converts the soft-telecined content to hard-telecined?

Thanks for any help! (and yes this is my first post!)

ffprobe reports this:

Input #0, mpeg, from 'DVD.mpg':
  Duration: 00:40:52.15, start: 1.000000, bitrate: 7911 kb/s
  Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, smpte170m, progressive), 720x480 [SAR 8:9 DAR 4:3], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc
    Side data:
      cpb: bitrate max/min/avg: 8000800/0/0 buffer size: 1835008 vbv_delay: N/A
  Stream #0:1[0xa0]: Audio: pcm_dvd, 48000 Hz, 2 channels, s16, 1536 kb/s

[Luke M]

It's unclear what your goal is. If you just want a smaller file, you can recompress while retaining the original interlacing. If you want to deinterlace, you can do that, and end up with either a constant 60p, or a variable 24p/60p video.

[Vostok]

My goal is to create a file of the same or higher quality (e.g. Apple ProRes which ffmpeg can export) which can then be edited in a non-linear video editor in order to remaster the audio and do other video processing.

As explained in my post, simply running the original MPEG through any transcoder I have used, and just letting that transcoder do its thing, produces incorrect output (e.g. repeated and missing frames in the soft telecine parts). These programs do not seem to transcode correctly when the input material is like as I described.

I am hoping that maybe ffmpeg, with a suitable set of option flags set, will do this job properly. Or some other program might. But so far nothing I have tried has worked correctly.

[jagabo]

The final ouput of NTSC DVD is always 59.94 fields per second. Use bwdif (the best deinterlacer in ffmpeg) to make a 59.94 fps video.
https://ffmpeg.org/ffmpeg-filters.html#bwdif

Something like:

ffmpeg -i input.mpg -vf bwdif=mode=1:parity=0:deint=0  -c:v prores output.mov

My goal is to create a file of the same or higher quality

Of course, using any lossy codec can only make the ouput lower quality than the source.

[Vostok]

Something like:

ffmpeg -i input.mpg -vf bwdif=mode=1:parity=0:deint=0  -c:v prores output.mov

Thanks - I tried that and it sort of worked - except one-quarter of the fields every second in the hard-telecined parts are played back in the wrong order. Is there a field dominance setting?

Of course, using any lossy codec can only make the ouput lower quality than the source.

Well, yes, I know that. I was trying to say I am not trying to make a smaller or lower-quality file.

[jagabo]

Something like:

ffmpeg -i input.mpg -vf bwdif=mode=1:parity=0:deint=0  -c:v prores output.mov

Thanks - I tried that and it sort of worked - except one-quarter of the fields every second in the hard-telecined parts are played back in the wrong order. Is there a field dominance setting?

parity:

The picture field parity assumed for the input interlaced video. It accepts one of the following values:

0, tff

Assume the top field is first.
1, bff

Assume the bottom field is first.
-1, auto

Enable automatic detection of field parity.

The default value is auto. If the interlacing is unknown or the decoder does not export this information, top field first will be assumed.

[Vostok]

Man, this still does not work. Regardless of the parity setting, some parts have the fields presented in the wrong order.

This DVD seems to have been deliberately authored in such a way that a DVD player can play it back OK, but any kind of transcoding gets screwed up real bad. Everything I have tried has failed.

The only way out of this might be to export the individual frames as an images sequence, re-import at a fixed frame rate, manually find the rate change parts, and then retime those to match the elapsed timecodes of the original.

Argh.

[jagabo]

Can you provide a sample of the source?

[Vostok]

Sure... this 45s excerpt has 11 frame rate changes in it according to mplayer excerpt.mpg -nosound -vo null -benchmark


[Attachment 66863 - Click to enlarge]

[davexnet]

After creating the d2v in DGindex, the following seems to work OK in Avisynth:

mpeg2source("C:\Users\davex\Desktop\avs_test\exc1.d2v")
assumetff()
tfm().tdecimate() 
# mctemporaldenoise(settings="very low",chroma=true)  << optional noise reduction

[Vostok]

After creating the d2v in DGindex, the following seems to work OK in Avisynth:

mpeg2source("C:\Users\davex\Desktop\avs_test\exc1.d2v")
assumetff()
tfm().tdecimate() 
# mctemporaldenoise(settings="very low",chroma=true)  << optional noise reduction

Hi - not sure how I managed to miss your reply but I did…

Yes that output looks fine… well done!

However I am in a Mac world and both DGindex and Avisynth appear to be Windows only?

Are there comparable tools that you know of that work in the command line that can be installed via homebrew? That would be fine for me.

I have been unable to solve this using ffmpeg, although there normally is some extremely obscure solution in that world, if one only knows how.

I’ve never come across MPEG content before that switches between hard- and soft- telecine multiple times a minute. Truly bizarre.

My preference would be to get the whole thing back to 24 fps progressive, i.e. leave the soft-telecined frames alone since they are already 24 fps (they are soft telecined to 29.97 for TV presentation only), and inverse telecine the hard-telecined frames back to 24 FPS.

[Sharc]

Where is your source?

[Vostok]

Where is your source?

Linked above in Post #9. The thumbnail isn’t there any more for some reason, but the MPEG file is still there if you click on that link and download it.

[Sharc]

Not perfect, but you can try with ffmpeg
http://ffmpeg.org/ffmpeg-filters.html#fieldmatch

ffmpeg -i "excerpt.mpg" -c:a aac -c:v libx264 -preset slow -crf 18 -pix_fmt yuv420p -vf fps=30000/1001,fieldmatch=order=tff:combmatch=full,yadif=deint=interlaced,decimate "excerpt IVTC_.mp4"

Edit: Attachment for combmatch=full added.

[Vostok]

Not perfect, but you can try with ffmpeg

Thanks, I’ll closely examine the outputs.

In what way is it “not perfect”?

Just about anything I tried with ffmpeg either resulted in occasional dropped or duplicated frames.

[Sharc]

In what way is it “not perfect”?

Take a look. Residual combed frames. The 2nd attachment is better though, but still some.
Maybe you can tweak the parameters (mode, cthresh ...) such that it behaves more like the Avisynth AssumeTFF().TFM().TDecimate().

[Vostok]

Take a look. Residual combed frames. The 2nd attachment is better though, but still some.
Maybe you can tweak the parameters (mode, cthresh ...) such that it behaves more like the Avisynth AssumeTFF().TFM().TDecimate().

I see what you mean.

What I don’t understand is why this cannot be algorithmically undone perfectly.

Clearly some GOPs are tagged as 24 fps and the DVD player soft-telecines those segments. Other GOPs are hard-telecined and tagged as 29.97fps and the DVD player just pushes those out unchanged.

It must be possible, surely, for ffmpeg to identify the 24fps GOPs, apply 2:3 pulldown, and spit out 29.97 fps exactly for those GOPs, but leave the 29.97 GOPs alone.

Then the whole thing could be reverse telecined, again perfectly, back to 24 FPS.

There should not be any guesswork required 🤔

[Sharc]

NTSC is 59.94 fields per second.
The vast majority of DVD players are flag readers. They play the 59.94 fields per second for native interlaced material, and follow the pulldown flags for outputting 59.94 fields per second for telecined material. Some high end "progressive" DVD players detected the 2:3 pattern of telecined material (cadence analysis instead of flag reading). The problem there was that for mixed interlaced and telecined stuff the player required a couple of frames to adapt to the changing situation.

https://hometheaterhifi.com/technical/technical-reviews/dvd-benchmark-part-5-progressive-scan-dvd/

[Vostok]

The vast majority of DVD players are flag readers. They play the 59.94 fields per second for native interlaced material, and follow the pulldown flags for outputting 59.94 fields per second for telecined material.

Indeed… so is there a way to get ffmpeg to do that? The flags are unambiguous… so I don’t understand why there seems to be so much “guesswork” with the solutions proposed to date, resulting in suboptimal outputs where it seems to take a while for ffmpeg to realize something has changed.

Maybe due to random pulldown cadence it isn’t possible for ffmpeg to accurately IVTC the hard-telecined segments, while leaving the native 24 fps progressive content alone, and produce a nice un-combed 24 fps progressive output file in one go.

But it should be possible for ffmpeg to apply 3:2 pulldown on the 24 fps segments, and do nothing to the hard-telecined segments, to at least create a single output file with a constant 29.97 fps / 59.94 field per second. Then I can do the IVTC on that output file to restore the original 24 fps progressive, manually adjusting cadence as and when needed.

[jagabo]

Sure... this 45s excerpt has 11 frame rate changes in it according to mplayer excerpt.mpg -nosound -vo null -benchmark


[Attachment 66863 - Click to enlarge]

The problem with that clip is the overlay and the background were telecined separately, then overlaid as 29.97 fps interlaced video. Sometimes the two were synchronized (the pulldown patterns match) and sometimes they weren't. You can see this easily with a simple Bob() deinterlace. Using TFM().TDecimate() will result in either the background or the overlay being jerky (duplicate/missing frames) when the pulldown patterns aren't synchronized.

This type of issue is very common with music videos. It didn't matter with interlaced TVs because they displayed one field at a time.

[Sharc]

It didn't matter with interlaced TVs because they displayed one field at a time.

If today's progressive monitors had a setting like "interlaced mode" for emulating legacy CRT TVs - it would solve such issues. We will never see this I guess.

[jagabo]

Pretty much all media players and TVs have smart bob (something like Yadif) deinterlacing. But some don't handle mixed hard/soft telecine properly. Some don't understand interlaced video in some containers and/or codecs.

[Vostok]

The problem with that clip is the overlay and the background were telecined separately, then overlaid as 29.97 fps interlaced video. Sometimes the two were synchronized (the pulldown patterns match) and sometimes they weren't. You can see this easily with a simple Bob() deinterlace. Using TFM().TDecimate() will result in either the background or the overlay being jerky (duplicate/missing frames) when the pulldown patterns aren't synchronized

I see. Closer inspection of the video using bwdif to pull it apart shows a really messed up compositing and encoding. Not only are the pulldown patterns not synchronized, sometimes the field dominance of the overlay doesn’t match the field dominance of the background, leading to temporal jitter no matter what you do.

So this video is basically irredeemable in terms of ever trying to restore it 24 fps progressive without some artefacts and I’ll just have to suck it up.

Thanks for all your help.