Can ffmpeg produce NTSC FILM with 3:2 pulldown?

[diprotic]

Goal: make an NTSC DVD of several h.264/m4v sources, using only free software.

Background:
The last few days I have fine-tuned a 2-pass encode using ffmpeg. I'm generally very happy with the encoding quality, and I'd like to stay with ffmpeg if I can.

...Then, I remembered my sources are shot at 23.976fps.

The -target ntsc-dvd flag in ffmpeg forces 29.97fps, effectively performing a hard telecine, which wastes a huge amount of data on duped frames for film sources.

What I tried:
I forced NTSC FILM framerate (with -r 24000/1001 after -target ntsc-dvd), only to see that DVDStyler wants to re-encode when authoring. My guess is that ffmpeg is not inserting pulldown flags into the video.

I found some rather dated forum / mailing list posts that seem to indicate that ffmpeg cannot (yet) insert the pulldown flags. Nothing in the ffmpeg documentation hints that this is possible, other than an mplayer filter called softpulldown, which produced horribly buggy output for me.

Questions:

1. Is it reasonable to assume that DVDStyler's insistence on re-encoding my 23.976 stuff is due to the lack of pulldown flags? It takes the 29.97 stuff just fine, with all other settings the same.
2. If so, can ffmpeg, via any means, properly output NTSC FILM with 3:2 pulldown flags? I don't care if it's native or via a filter / plugin / whatever. I'm using the latest from git, btw.

Thanks in advance for any input / correction / help!

-diprotic

P.S.

I understand mpeg2enc is a very different approach which definitely does support 3:2 pulldown, and produces arguably better quality than lavc/ffmpeg (albeit at a performance loss). It would introduce a lot of intermediate steps that ffmpeg avoids. If this is the only way to produce NTSC FILM with free software, I'm open to trying it, but I would really appreciate a point in the right direction, because that's a totally new workflow for me.

[poisondeathray]

there are some windows tools like dgpulldown that can insert the flags to a 23.976 stream

according to the changelog ffmbc can encode mpeg2 with 3:2 pulldown since 0.6.1

FFmbc-0.6.1:
- MPEG-2 3:2 pulldown encoding is now supported.

from ffmbc -h

mpeg2video encoder options:
-pulldown <string> E.V.. Encode with pulldown: 3:2

I haven't tested it in ffmbc, don't know how well it works

[diprotic]

there are some windows tools like dgpulldown that can insert the flags to a 23.976 stream

according to the changelog ffmbc can encode mpeg2 with 3:2 pulldown since 0.6.1

FFmbc-0.6.1:
- MPEG-2 3:2 pulldown encoding is now supported.

from ffmbc -h

mpeg2video encoder options:
-pulldown <string> E.V.. Encode with pulldown: 3:2

I haven't tested it in ffmbc, don't know how well it works

That's crazy! I can't believe how many features of ffmpeg are completely undocumented. Sure enough, the -pulldown flag is there in my windows binary. Any idea of the correct usage? Strange that the flag takes a string as an argument.

Edit: Nevermind, I misread both your comment and my ffmpeg output. Looks like ffmbc is the way to go.

[poisondeathray]

not sure what the usage/values are . In x264, its --pulldown 32 . I don' t know if ffmbc/ffmpeg will take the same
syntax

I've also noted sometimes ffmbc and ffmpeg syntax are slightly different

[poisondeathray]

Hmmm ....I was just playing with ffmbc ; I'm guessing might be hard pulldown , not soft . -pulldown 3:2 seems to be the correct switch but it seems to expect interlaced input - anyways I can't figure it out to do it properly 23.976 with soft ... were you able to figure it out ?

Since you have a windows ffmpeg binary, that means you have access to a PC.... you might try DGPulldown (this does work for sure)

Also, HCEnc - much better quality than ffmpeg or mpeg2enc ; but requires avisynth or dgindex and pc only

you can also try x262 (that's x262, not x264) as an encoder - looks promising but some bugs still . Some parts of x264 code is "borrowed" . There are some binaries in this thread
http://forum.doom9.org/showthread.php?t=157675
https://github.com/kierank/x262

[diprotic]

Hmmm ....I was just playing with ffmbc ; I'm guessing might be hard pulldown , not soft . -pulldown 3:2 seems to be the correct switch but it seems to expect interlaced input - anyways I can't figure it out to do it properly 23.976 with soft ... were you able to figure it out ?

Since you have a windows ffmpeg binary, that means you have access to a PC.... you might try DGPulldown (this does work for sure)

Also, HCEnc - much better quality than ffmpeg or mpeg2enc ; but requires avisynth or dgindex and pc only

you can also try x262 (that's x262, not x264) as an encoder - looks promising but some bugs still . Some parts of x264 code is "borrowed" . There are some binaries in this thread
http://forum.doom9.org/showthread.php?t=157675
https://github.com/kierank/x262

A pulldown switch expecting interlaced input? That seems really strange. I thought 3:2 pulldown was to take film source (usually progressive 24fps) and adapt it to NTSC video (which then may become interlaced).

I'll have to play with it when I get back to my production machine and get to the bottom of it.

I'm posting from a windows machine, but I produce on a Mac or Linux. I've read of a few standalone pulldown programs for *nix, but I'd really like to avoid adding a whole extra step (or extra steps) to my process. With my current ffmpeg process, there's no (de)muxing or intermediate files, although it looks like an intermediate step may be required if ffmbc -pulldown turns out to be hard telecine.

I've considered HCEnc in a VM, wine, or similar, because it gets very good feedback from those who have used it, but I'm a bit of a shell nerd and again, I've already achieved 99% of my desired results with ffmpeg. It's really sad that it would lack (what I think) would be such a fairly basic, yet critical feature of an MPEG2 encoder.

As I was posting here I stumbled upon this post, which looks like a way to use ffmpeg to decode the h.264 input and pipe the output to mpeg2enc. This approach would probably work, but require a) completely retooling my encoding settings for mpeg2enc, and b) require separate audio/video processing, muxing, etc. I guess a few more lines in my transcoding shellscript wouldn't kill me

I'll goof off when I get the chance and report back.

[poisondeathray]

A pulldown switch expecting interlaced input? That seems really strange. I thought 3:2 pulldown was to take film source (usually progressive 24fps) and adapt it to NTSC video (which then may become interlaced).

yes, that's what soft pulldown rff/tff flags are for - to output a 59.94i signal with 23.976 progressive content. When played back, the pulldown is removed and you get your 23.976p stream back.

Not sure what this exactly does in ffmbc... I only tried a few things , didn't have time to explore much. ffmpeg is weird - sometimes it's as simple as the order in which you place the switches . You can ask on the ffmbc board as well

[diprotic]

OK, wow.

Many, many hours of research has taught me that ffmpeg and ffmbc can not produce DVD-Video compliant NTSC FILM with proper 3:2 pulldown. At least, not yet—to the very best of my understanding. Please, correct me if I'm wrong. I ripped through C source and tried hundreds of permutations on the command line.

ffmpeg, up to git on 12/18/2012, does not have any native pulldown filter. The closest it offers is mplayer filters, accessible with -vf mp=<filter>. I tried several filters including pulldown, but ffmpeg dropped almost every frame; very buggy.

ffmbc does feature the -pulldown 3:2 flag, but errors out if the source is progressive, stating that the pulldown flag only works with interlaced source. I can't make heads or tails of this; I have no idea why it wouldn't be able to telecine progressive source. Since my source is progressive, ffmbc is out.

Here's what worked:

mencoder 1.1-1.4.2 finally put all the pieces together for me. Actually, mencoder exceeded all my expectations.

First, mencoder supports N-pass encodes, as opposed to ffmpeg's 2 pass limit. And yes, there is a significant improvement from 2-pass to 4-pass, especially with bitrate constraints. I even ran one video through 7 passes just for laughs, with a minor (albeit not worthwhile) improvement over 4.

Second, telecining actually worked exactly as expected. I'll post my entire encoding invocation later, but for now, the telecine bits are:

-of mpeg -mpegopts telecine -ofps 24000/1001

Third, mencoder uses ffmpeg's lavc, so most of my meticulously assembled high-quality ffmpeg settings directly translate to mencoder.

The visual quality difference between 4-pass, soft telecine from mencoder is significantly better than the 2-pass, hard telecine that ffmpeg gave me.

Best of all, DVDStyler accepts the output as compliant, without need to re-encode!

Until ffmpeg gets N-pass encodes and proper telecine, looks like I'll be switching to mencoder.

Merry Christmas!

[poisondeathray]

Thanks for reporting back . The "ffmpeg folks" should merge that working code

I think I read somewhere that ffmpeg/lavc mpeg2 encoding quality has improved somewhat recently, I wonder how it stacks up to other encoders now?

[diprotic]

Thanks for reporting back . The "ffmpeg folks" should merge that working code

I think I read somewhere that ffmpeg/lavc mpeg2 encoding quality has improved somewhat recently, I wonder how it stacks up to other encoders now?

I don't have a big frame of reference, and it's been a while since I used commercial encoders like TMPGenc and CCE. All I can say is that the final 4-pass encode using mpeg2video from lavc/mencoder resulted in a very pleasing result, with surprisingly little or no apparent blocking, noise, ringing, or other typical MPEG2 artifacts, even at 1700k VBR.

IMHO lavc mpeg2video from mencoder leaves very little to be desired.

P.S.

The fourth advantage mencoder has over ffmpeg is fairly decent documentation. I was able to understand most of the advanced features and how they may affect the encode. FFmpeg on the other hand doesnt even document the -target film-dvd option, which does output FILM framerate (24000/1001), but apparently without pulldown flags. Gahh!

[poisondeathray]

LOL the ffmpeg documentation is utter garbage . It' s so confusing (at least to me) with obscure examples (if they even have examples). You literally have to learn by trial & error and testing like you did earlier

[rezsbc]

OK, wow.

Many, many hours of research has taught me that ffmpeg and ffmbc can not produce DVD-Video compliant NTSC FILM with proper 3:2 pulldown. At least, not yet—to the very best of my understanding. Please, correct me if I'm wrong. I ripped through C source and tried hundreds of permutations on the command line.

Here's what worked:

mencoder 1.1-1.4.2 finally put all the pieces together for me. Actually, mencoder exceeded all my expectations.

Thanks so much for this post! Appreciate the effort you went to I will check out mencoder now, you've saved me a lot of frustration I was looking at all those options you tried!

[diprotic]

Glad it helped someone. It was a nightmare, glad to save you the trouble!

[electron.rotoscope]

I'm trying to do something similar, and my quest has brought me here. Thanks for the post!! Very helpful. I don't know how to use mencoder very well, but it seems like the "-mpegopts telecine" magic only works when writing out to a program stream and not to a raw m2v, which sadly means that it's of limited use to me.
Earlier I found a program DGPulldown (from 2006 but still available that will take unflagged .m2v from ffmpeg and add the soft pulldown flag, but it requires rewriting the whole video to a new file so it's not a perfect solution either.
Imagine my surprise when I download the mplayer source looking for information (a trick I had used previously to figure out what -target film-dvd did in FFMpeg due to the aforementioned sparse documentation) and find in mplayer-export-2015-03-26\libmpdemux\muxer_mpeg.c the section about soft telecine prefaced with "/* excerpt from DGPulldown Copyright (C) 2005-2006, Donald Graft */"
Now to try to get someone who can actually code in C to add the stuff from libmpdemux\muxer_mpeg.c either to ffmpeg or mencoder's raw m2v export. Or maybe just live with a secondary windows-based rewrite of all my streams...
EDIT: Seeing on videohelp.com's auto-generated links that there's also a mac binary hmmm

[manono]

Earlier I found a program DGPulldown (from 2006 but still available that will take unflagged .m2v from ffmpeg and add the soft pulldown flag, but it requires rewriting the whole video to a new file so it's not a perfect solution either.

Don't know why not, but if you check the "Modify file in place" box, it'll add the flags to the original MPV/M2V without creating a second file.

[electron.rotoscope]

Earlier I found a program DGPulldown (from 2006 but still available that will take unflagged .m2v from ffmpeg and add the soft pulldown flag, but it requires rewriting the whole video to a new file so it's not a perfect solution either.

Don't know why not, but if you check the "Modify file in place" box, it'll add the flags to the original MPV/M2V without creating a second file.

Oh my goodness I didn't even realize. Thanks!!