How to use ffmpeg to de-telecine (de-interlace) PAL to PAL 25p

[jagabo]

What's interesting to me is that the fieldmatch filter seems to be quite slow for some reason.

Given ffmpeg's low CPU usage when using it I suspect it's single threaded. If you're concerned about speed just run several at the same time.

[poisondeathray]

I'm quoting the specific part above because it shows that we do basically see eye to eye here. The term "interpolation" is also a very, very general term and easily misunderstood, I would guess not many actually WOULD understand it to mean anything even close to as complex as whatever yadif does. If you specifically mention "temporal interpolation", then - sure, I guess.

The simplest explanation for "interpolation" is an algorithm to create pixels. eg. If I upscale an image, that is an example of spatial interpolation .

Yadif creates missing pixels from dropped scanlines (all deinterlacers do) , but in a spatial manner, not temporal. There is a field check for spatial and temporal motion and differences. ie. It's not temporal interpolation, it's temporal analysis.

Mode =1 is not temporal interpolation, because on interlaced content, 50 samples /s are already represented. No new samples in time are being synthesized. Interlaced content is a spatial deficiency (single field) . Mode=0 is not temporal either - each field becomes a frame. It's a spatial interpolation . On 25p content, no new samples in time are being generated either. If you apply mode=1 you have 25p in 50p. It's still 25p just duplicates. The temporal sampling resolution is not changed using yadif

Optical flow is an example of temporal interpolation. e.g. 25p to 50p content. New in-between samples in time are generated. You actually have double the motion samples

For some reason I cannot see the test video (I'll try another browser maybe)

some embedding issue, it should be ok now

The one thing that's still not "solved" here is the performance issue that I mentioned early on. Yes, you can use many filters (algorithms) that are guaranteed to produce the correct result. However, it's possible doing that would be very slow and it is generally not critically important that the result would be guaranteed to be 'correct'. You mentioned that whether or not I can "see" the difference depends on this and that - you are exactly right!

yes, field matching is slower , it's still several times realtime for me (filter only). I get about 1/2 speed comparedd to yadif. Bwdif is aobut 1.4-1.5x faster than yadif for me

If it helps to understand, this is not my job and no one is paying me a dime to do this stuff. I have a lot of videos here that I want to process fairly quickly. Just simply put, I do NOT have time to closely analyse each of them and spend time making a tailored AviSynth filter chain for each of them. But of course in what I KNOW to be telecined progressive content, I'd like to avoid losing detail unnecessarily.

The bottleneck is encoding. Even fieldmatching gives you many times realtime. Use something like NVEnc HEVC. You need a bit more bitrate to match libx265, but it's much faster

What's interesting to me is that the fieldmatch filter seems to be quite slow for some reason. Like I mentioned, simply merging lines from adjacent fields with a known, constant pattern should be faster than ANY alternative here, especially something like yadif that performs complex image analysis.

So I guess stupid question time: If yadif can process this stuff at 10-20x realtime, why does fieldmatch not process it at a HUNDRED times realtime?

Fieldmatch is more complex in many ways than yadif

Your case is not a fixed known pattern. Otherwise the entire thing would be clean (no combing, and you apply no filter) , or the entire thing would be combed no clean sections (then you might be able to use the select filter with interleave in a pattern, it might be faster)

[jagabo]

Even fieldmatching gives you many times realtime.

I'm seeing about 3x realitime at best. For example, using the video in post #26, on an 8 core 16 thread CPU (i9 9900K):

Using fieldmatch:

ffmpeg -y -benchmark -threads N -i "24p to 25 euro-pulldown-tff.mp4" -vf fieldmatch=order=tff:mode=pc -f null -

threads	seconds
---------------------
1	25.630
2	14.766	
4	14.207
8	14.127

You can see that speed hardly increase over threads=2 (and CPU usage never gets over 15 percent), Using combmatch=full is even slower. But using yadif:

ffmpeg -y -benchmark -threads N -i "24p to 25 euro-pulldown-tff.mp4" -vf yadif=mode=0:parity=0 -f null -

threads	seconds
------------------
1	13.554	
2	 7.445
4	 4.296
8	 2.690
16	 2.140

Speed is still increasing at threads = 16 and CPU usage has maxed out. All are far faster than fieldmatch.

Of course, if one is using slow CPU encoding the low speed of fieldmatch doesn't matter much.

[poisondeathray]

You can see that speed hardly increase over threads=2 (and CPU usage never gets over 15 percent),

Yes it seems poorly optimized. Avisynth's TFM (pp=0) seems not very well optimized either ( playing with prefetch values didn't help much)

vapoursynth's TFM (PP=0) seems to utilize resources properly. vapoursynth threading is generally better than avisynth or ffmpeg

[Sharc]

Even fieldmatching gives you many times realtime.

I'm seeing about 3x realitime at best. For example, using the video in post #26, on an 8 core 16 thread CPU (i9 9900K):

Using fieldmatch:

ffmpeg -y -benchmark -threads N -i "24p to 25 euro-pulldown-tff.mp4" -vf fieldmatch=order=tff:mode=pc -f null -

threads	seconds
---------------------
1	25.630
2	14.766	
4	14.207
8	14.127

You can see that speed hardly increase over threads=2 (and CPU usage never gets over 15 percent), Using combmatch=full is even slower. But using yadif:

ffmpeg -y -benchmark -threads N -i "24p to 25 euro-pulldown-tff.mp4" -vf yadif=mode=0:parity=0 -f null -

threads	seconds
------------------
1	13.554	
2	 7.445
4	 4.296
8	 2.690
16	 2.140

Speed is still increasing at threads = 16 and CPU usage has maxed out. All are far faster than fieldmatch.

Of course, if one is using slow CPU encoding the low speed of fieldmatch doesn't matter much.

Do the frame flags (interlaced, progressive) possibly impact the speed of the fieldmatching?
The file "24p to 25 euro-pulldown-tff.mp4" of post #26 was euro-telecined by means of a script and then x264 encoded as interlaced TFF.
The file here uses x264's --euro-pulldown which flags the frames differently.

[jagabo]

Field matching speeds are about the same (per frame) with that progressive video. That video has fewer frames and proportionately shorter times.

[Sharc]

Field matching speeds are about the same (per frame) with that progressive video. That video has fewer frames and proportionately shorter times.

Fewer frames yes. It is soft-telecined with x264's --euro-pulldown option, I think. Frame number is same as the original 24fps source, hence it doesn't even require fieldmatching. Just ignore the pulldown flags.

[Atillart]

Hi, what would be the best method to deinterlace a movie 3:2 pulldown (29.976).
I suppose that it would be necessary to join together the fields which agree and to remove the 6 frames (12 fields) interlaced.

[jagabo]

Normal NTSC 3:2 pulldown:

-vf fieldmatch=combmatch=full,decimate=cycle=5

match files, then remove one of every five frames (6 of every 30). The default for decimate is 5 so you don't really need to specify the cycle:

-vf fieldmatch=combmatch=full,decimate

[davexnet]

For 29.97 with soft pulldown to IVTC 23.976, this seems to work better:

ffmpeg -i ftl_test.mpg -vf "fieldmatch=order=tff, fps=fps=24000/1001" FTL_IVTC4.mkv

Here's my source:

[jagabo]

For mixed hard/soft telecine (quite common, especially in anime and cartoons) you can use:

-vf "repeatfields,fieldmatch=combmatch=full,decimate=cycle=5"

Of course that works for sources that pure soft pulldown or pure hard pulldown too. So maybe that's the safest.

Then there are videos with occasional orphaned fields -- fields for which there is no matching field to produce a progressive frame. With those you will want to follow with a smart deinterlacer.

[davexnet]

I ended up with a result like this and didn't know what was going on

[jagabo]

Yes. By default fmpeg doesn't perform the pulldown when decoding MPEG 2 with soft pulldown flags. So the output of the decoder is 23.976 fps, not 29.97 fps. But ffmpeg still thinks the output is 29.97 fps. Hence the running time is reduced by 20 percent. And decimation leaves you with jerky video.

The sequence you used in post #40 gives the correct runtime but results in a variable frame rate video with six little jerks every second. Jerky because the duration of frames is 33 ms, 33 ms, 33 ms, 66 ms, repeating.

The sequence I gave in post #41 gives frames that are all ~41.7 ms (ie true 23.976 fps, constant frame rate).

[davexnet]

Thanks jagabo. Seems like I should look into FFprobe

[jagabo]

Here's a two line batch file onto which you can drag/drop a video and get a CSV file (suitable for importing into Excel, Libre Calc, etc.) to see timecodes.

echo entry,media_type,pkt_dts,pkt_dts_time,best_effort_Timestamp_time,duration,pkt_duration_time,ftype,coded# > "%~nx1.csv"
ffprobe -v 32 -stats -y -hide_banner -i %1 -select_streams v -print_format csv -of csv -show_entries "frame=media_type,coded_picture_number,pkt_dts,pkt_dts_time,pkt_duration,pkt_duration_time,best_effort_timestamp_time,pict_type" >> "%~nx1.csv"

[davexnet]

Here's a two line batch file onto which you can drag/drop a video and get a CSV file (suitable for importing into Excel, Libre Calc, etc.) to see timecodes.

echo entry,media_type,pkt_dts,pkt_dts_time,best_effort_Timestamp_time,duration,pkt_duration_time,ftype,coded# > "%~nx1.csv"
ffprobe -v 32 -stats -y -hide_banner -i %1 -select_streams v -print_format csv -of csv -show_entries "frame=media_type,coded_picture_number,pkt_dts,pkt_dts_time,pkt_duration,pkt_duration_time,best_effort_timestamp_time,pict_type" >> "%~nx1.csv"

I expected to see the uneven codes to explain the stutter, but they're all 41ms:

frame,video,10469,10.469000,10.469000,41,0.041000,I,250
frame,video,10511,10.511000,10.511000,41,0.041000,B,253
frame,video,10552,10.552000,10.552000,41,0.041000,B,252
frame,video,10594,10.594000,10.594000,41,0.041000,B,254
frame,video,10636,10.636000,10.636000,41,0.041000,P,251
frame,video,10677,10.677000,10.677000,41,0.041000,B,257
frame,video,10719,10.719000,10.719000,41,0.041000,B,256
frame,video,10761,10.761000,10.761000,41,0.041000,B,258
frame,video,10802,10.802000,10.802000,41,0.041000,P,255
frame,video,10844,10.844000,10.844000,41,0.041000,B,261
frame,video,10886,10.886000,10.886000,41,0.041000,B,260
frame,video,10928,10.928000,10.928000,41,0.041000,B,262
frame,video,10969,10.969000,10.969000,41,0.041000,P,259
frame,video,11011,11.011000,11.011000,41,0.041000,B,265
frame,video,11053,11.053000,11.053000,41,0.041000,B,264
frame,video,11094,11.094000,11.094000,41,0.041000,B,266
frame,video,11136,11.136000,11.136000,41,0.041000,P,263
frame,video,11178,11.178000,11.178000,41,0.041000,B,268
frame,video,11220,11.220000,11.220000,41,0.041000,P,267
frame,video,11261,11.261000,11.261000,41,0.041000,P,269
frame,video,11303,11.303000,11.303000,41,0.041000,B,271
frame,video,11345,11.345000,11.345000,41,0.041000,P,270
frame,video,11386,11.386000,11.386000,41,0.041000,B,273
frame,video,11428,11.428000,11.428000,41,0.041000,P,272
frame,video,11470,11.470000,11.470000,41,0.041000,P,274
frame,video,11512,11.512000,11.512000,41,0.041000,B,276
frame,video,11553,11.553000,11.553000,41,0.041000,P,275
frame,video,11595,11.595000,11.595000,41,0.041000,B,278
frame,video,11637,11.637000,11.637000,41,0.041000,B,279
frame,video,11678,11.678000,11.678000,41,0.041000,P,277
frame,video,11720,11.720000,11.720000,41,0.041000,B,281
frame,video,11762,11.762000,11.762000,41,0.041000,P,280
frame,video,11803,11.803000,11.803000,41,0.041000,B,283
frame,video,11845,11.845000,11.845000,41,0.041000,P,282
frame,video,11887,11.887000,11.887000,41,0.041000,B,285
frame,video,11929,11.929000,11.929000,41,0.041000,P,284
frame,video,11970,11.970000,11.970000,41,0.041000,B,287
frame,video,12012,12.012000,12.012000,41,0.041000,B,288
frame,video,12054,12.054000,12.054000,41,0.041000,P,286
frame,video,12095,12.095000,12.095000,41,0.041000,B,291
frame,video,12137,12.137000,12.137000,41,0.041000,B,290
frame,video,12179,12.179000,12.179000,41,0.041000,B,292
frame,video,12221,12.221000,12.221000,41,0.041000,P,289
frame,video,12262,12.262000,12.262000,41,0.041000,B,295
frame,video,12304,12.304000,12.304000,41,0.041000,B,294
frame,video,12346,12.346000,12.346000,41,0.041000,B,296
frame,video,12387,12.387000,12.387000,41,0.041000,P,293
frame,video,12429,12.429000,12.429000,41,0.041000,B,298
frame,video,12471,12.471000,12.471000,41,0.041000,P,297
frame,video,12513,12.513000,12.513000,41,0.041000,B,300
frame,video,12554,12.554000,12.554000,41,0.041000,B,301
frame,video,12596,12.596000,12.596000,41,0.041000,P,299
frame,video,12638,12.638000,12.638000,41,0.041000,B,303

[jagabo]

Sorry, my mistake. I had been playing around with variations of your command line. It looks like the video I checked with ffprobe was not the one from your original command line but rather an alternate. I went back and rebuilt the video with your command line and checked it. It is fine, with 41 ms durations. Both int the durations column and by subtracting sequential best effort timestamps.

[davexnet]

Sorry, my mistake. I had been playing around with variations of your command line. It looks like the video I checked with ffprobe was not the one from your original command line but rather an alternate. I went back and rebuilt the video with your command line and checked it. It is fine, with 41 ms durations. Both int the durations column and by subtracting sequential best effort timestamps.

I agree with you, the report is fine, but the file doesn't play properly.
The video is speeded up, finishes in 1:20 while the audio continues for another 20 seconds