Trying to transcode a DVD with a mix of soft- and hard-telecine

[Vostok]

I am trying to transcode a DVD that is driving me bonkers, because it has a mix of soft- and hard-telecine and NTSC video. It's driving me bonkers because in some places the content switches between these formats multiple times within a few seconds! I have never seen that before.

The DVD is 41 minutes long. According to

mplayer DVD.mpg -nosound -vo null -benchmark

there are 336 rate changes within that 40 minutes, between 24000/1001 fps progressive content and 30000/1001fps NTSC video (which is sometimes hard telecined video, and sometimes actual NTSC video).

My problem is that if I just run DVD.mpg through any transcoding program such as ffmpeg, Quicktime, MPEG Streamclip, DG Pulldown, tsMuxer or anything else, it does a completely rubbish job due to the frequent rate changes. Typically what happens is that the 24 fps soft-telecined progressive content is converted to 29.97 fps content that repeats some frames but omits others. It's not even an interlaced frame rate change. The result is a jerky mess.

I am at witt's end. Is there a set of parameters I can use with ffmpeg that can create a usable output video file, e.g. produce a 24 fps file that captures every original frame of the soft-telecined 24 fps material, and frame rate converts everything else to 24 fps?

Or vice versa, produce an interlaced 29.97 fps file that replicates exactly all the hard-telecined and NTSC video content, and converts the soft-telecined content to hard-telecined?

Thanks for any help! (and yes this is my first post!)

ffprobe reports this:

Input #0, mpeg, from 'DVD.mpg':
  Duration: 00:40:52.15, start: 1.000000, bitrate: 7911 kb/s
  Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, smpte170m, progressive), 720x480 [SAR 8:9 DAR 4:3], 29.97 fps, 59.94 tbr, 90k tbn, 59.94 tbc
    Side data:
      cpb: bitrate max/min/avg: 8000800/0/0 buffer size: 1835008 vbv_delay: N/A
  Stream #0:1[0xa0]: Audio: pcm_dvd, 48000 Hz, 2 channels, s16, 1536 kb/s

[Luke M]

It's unclear what your goal is. If you just want a smaller file, you can recompress while retaining the original interlacing. If you want to deinterlace, you can do that, and end up with either a constant 60p, or a variable 24p/60p video.

[Vostok]

My goal is to create a file of the same or higher quality (e.g. Apple ProRes which ffmpeg can export) which can then be edited in a non-linear video editor in order to remaster the audio and do other video processing.

As explained in my post, simply running the original MPEG through any transcoder I have used, and just letting that transcoder do its thing, produces incorrect output (e.g. repeated and missing frames in the soft telecine parts). These programs do not seem to transcode correctly when the input material is like as I described.

I am hoping that maybe ffmpeg, with a suitable set of option flags set, will do this job properly. Or some other program might. But so far nothing I have tried has worked correctly.

[jagabo]

The final ouput of NTSC DVD is always 59.94 fields per second. Use bwdif (the best deinterlacer in ffmpeg) to make a 59.94 fps video.
https://ffmpeg.org/ffmpeg-filters.html#bwdif

Something like:

ffmpeg -i input.mpg -vf bwdif=mode=1:parity=0:deint=0  -c:v prores output.mov

My goal is to create a file of the same or higher quality

Of course, using any lossy codec can only make the ouput lower quality than the source.

[Vostok]

Something like:

ffmpeg -i input.mpg -vf bwdif=mode=1:parity=0:deint=0  -c:v prores output.mov

Thanks - I tried that and it sort of worked - except one-quarter of the fields every second in the hard-telecined parts are played back in the wrong order. Is there a field dominance setting?

Of course, using any lossy codec can only make the ouput lower quality than the source.

Well, yes, I know that. I was trying to say I am not trying to make a smaller or lower-quality file.

[jagabo]

Something like:

ffmpeg -i input.mpg -vf bwdif=mode=1:parity=0:deint=0  -c:v prores output.mov

Thanks - I tried that and it sort of worked - except one-quarter of the fields every second in the hard-telecined parts are played back in the wrong order. Is there a field dominance setting?

parity:

The picture field parity assumed for the input interlaced video. It accepts one of the following values:

0, tff

Assume the top field is first.
1, bff

Assume the bottom field is first.
-1, auto

Enable automatic detection of field parity.

The default value is auto. If the interlacing is unknown or the decoder does not export this information, top field first will be assumed.

[Vostok]

Man, this still does not work. Regardless of the parity setting, some parts have the fields presented in the wrong order.

This DVD seems to have been deliberately authored in such a way that a DVD player can play it back OK, but any kind of transcoding gets screwed up real bad. Everything I have tried has failed.

The only way out of this might be to export the individual frames as an images sequence, re-import at a fixed frame rate, manually find the rate change parts, and then retime those to match the elapsed timecodes of the original.

Argh.

[jagabo]

Can you provide a sample of the source?

[Vostok]

Sure... this 45s excerpt has 11 frame rate changes in it according to mplayer excerpt.mpg -nosound -vo null -benchmark


[Attachment 66863 - Click to enlarge]

[davexnet]

After creating the d2v in DGindex, the following seems to work OK in Avisynth:

mpeg2source("C:\Users\davex\Desktop\avs_test\exc1.d2v")
assumetff()
tfm().tdecimate() 
# mctemporaldenoise(settings="very low",chroma=true)  << optional noise reduction