Why is ffmpeg dropping frames when encoding interlaced mpeg2

[wild-mouse]

I'm archiving some old VHS and ffmpeg's behavior has be confused. My setup looks like this: VCR -> OSSC -> Inogeni DVI2USB. The OSSC is in passthru mode and sends a 525i signal via HDMI to the capture card. I encode realtime using gstreamer because it's fairly easy to set up a working v4l2/pulseaudio pipeline and capture lossless h264 and pcm. The "raw" result of capture is a video file which is 720x480, 29.97fps, progressive. The OSSC's passthru does *not* perform any deinterlacing, so there are 2 interlaced fields in every frame of this "raw" file that I can clearly see when playing it in VLC. My goal is to convert this file to DVD and preserve the interlacing so that I can play it on a non-upscaling DVD player (480i via component video). I thought I had it all figured out with this command:

ffmpeg -i raw.mkv -vf setfield=tff,separatefields -target ntsc-dvd -flags +ildct+ilme -aspect 4:3 dvd.mpg

However, ffmpeg appears to drop frames during the conversion:

frame= 3283 fps=345 q=2.5 Lsize=  111846kB time=00:01:49.84 bitrate=8341.0kbits/s dup=0 drop=3280 speed=11.5x

I don't understand why that would happen. If ffpmeg is truly encoding interlaced video, then it would be using every frame of the raw file, correct? The separatefields filter would be converting the stream into a top,bottom,top,bottom(etc) list of fields that is effectively 59.94 fps. If I'm assuming that ffmpeg is dropping "frames" from the separated fields, I am still confused because every separated field would be needed in order to reproduce the original interlaced information.

Is there any way that I can know for certain that the resulting mpeg includes all the information from the "raw" video, only properly encoded as fields to be interlaced by the player? The encoded video looks like progressive frames in VLC, but I'm not sure if what I'm seeing are interlaced frames at 59.94 fps or just the even frames at 29.97 fps (or just the odd frames for that matter). Is it possible that this is all just poor messaging from ffmpeg?

Here's the stream info if that helps any:

Input #0, matroska,webm, from 'raw.mkv':
  Metadata:
    ENCODER         : Lavf58.76.100
  Duration: 01:15:52.68, start: 0.000000, bitrate: 51018 kb/s
  Stream #0:0(eng): Video: h264 (High 4:4:4 Predictive), yuv422p(tv, bt709/bt709/smpte170m, progressive), 720x480 [SAR 8:9 DAR 4:3], 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
    Metadata:
      title           : Video
      ENCODER         : Lavc58.134.100 libx264
      DURATION        : 01:15:52.681000000
  Stream #0:1(eng): Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s (default)
    Metadata:
      title           : Audio
      DURATION        : 01:15:52.670000000
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mpeg2video (native))
  Stream #0:1 -> #0:1 (pcm_s16le (native) -> ac3 (native))
Press [q] to stop, [?] for help
Output #0, dvd, to 'dvd.mpg':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0(eng): Video: mpeg2video (Main), yuv420p(tv, bt709/bt709/smpte170m, top coded first (swapped)), 720x480 [SAR 8:9 DAR 4:3], q=2-31, 6000 kb/s, 29.97 fps, 90k tbn (default)
    Metadata:
      title           : Video
      DURATION        : 01:15:52.681000000
      encoder         : Lavc58.134.100 mpeg2video
    Side data:
      cpb: bitrate max/min/avg: 9000000/0/6000000 buffer size: 1835008 vbv_delay: N/A
  Stream #0:1(eng): Audio: ac3, 48000 Hz, stereo, fltp, 448 kb/s (default)
    Metadata:
      title           : Audio
      DURATION        : 01:15:52.670000000
      encoder         : Lavc58.134.100 ac3

[wild-mouse]

To (maybe) answer my own question: I think what's happening is that my command is producing a video with half the temporal resolution. The separatefields filter is deinterlacing the fields from my raw video and producing a stream of 720x240p frames at 59.94 fps. ffmpeg is then dropping every odd frame and resizing the remainder to 720x480. The result is a 720x480p stream that is flagged as interlaced even though the extra temporal resolution is lost. My previous understanding was that mpeg2video would encode interlaced video as a series of fields and it was up to the player to interlace them for the display, hence my use of the separatefields filter. My new understanding is that mpeg2 does (or at least can) encode interlaced video as a series of fields, however, the decoding process produces interlaced frames directly. Hinting to ffmpeg that the input video is interlaced using the ilme flag may indeed trigger the more efficient encoding mechanism for interlaced video where each field is encoded separately, thus not requiring the separatefields filter. I hope that's right at least.

My new command for transcoding looks like this:

ffmpeg -i raw.mkv -flags +ildct+ilme -vf setfield=tff -top 1 -target ntsc-dvd -aspect 4:3 dvd.mpg

Do I need both setfield and -top 1? That is the only thing I've found where ffmpeg's stream info agrees with ffprobe's analysis of the output video. Using setfield alone, ffmpeg's stream info reports "top coded first (swapped)" while encoding, while ffprobe says the resulting output is simply "top first". Using -top 1 alone, ffmpeg's stream info reports "top first" when encoding, however, ffprobe says the resulting output is "bottom first". Only when using both does ffmpeg's encoding stream info and ffprobe agree on "top first". But maybe I'm just not understanding what "top coded first (swapped)" actually means.

[pandy]

To (maybe) answer my own question:

To keep it short - encoder will encode frames made from fields - all you need to do is configure field order and tell encoder it is interlace (so configure some aspects of encoder).

My new command for transcoding looks like this:

ffmpeg -i raw.mkv -flags +ildct+ilme -vf setfield=tff -top 1 -target ntsc-dvd -aspect 4:3 dvd.mpg

Do I need both setfield and -top 1? That is the only thing I've found where ffmpeg's stream info agrees with ffprobe's analysis of the output video. Using setfield alone, ffmpeg's stream info reports "top coded first (swapped)" while encoding, while ffprobe says the resulting output is simply "top first". Using -top 1 alone, ffmpeg's stream info reports "top first" when encoding, however, ffprobe says the resulting output is "bottom first". Only when using both does ffmpeg's encoding stream info and ffprobe agree on "top first". But maybe I'm just not understanding what "top coded first (swapped)" actually means.

-vf is filter part i.e. processing - if you not process video then you can remove vf part completely.

I would add '-alternate_scan true' if your content is interlaced.

[wild-mouse]

-vf is filter part i.e. processing - if you not process video then you can remove vf part completely.

As I stated, when i remove the setfield filter, ffprobe and mediainfo report the output video as bottom first, despite the source material being top first. According to ffmpeg's documentation, the setfield filter "does not change the input frame, but only sets the corresponding property", so can you still consider it processing the video? It seems to be doing exactly what the documentation says, setting the field order property on my output file to the correct "top first" value and nothing else. But I could just be misunderstanding ffmpeg's mysterious ways. However, my intuition tells me that the best workflow for video is to preserve the field ordering from source to destination.

Edit: It appears that this top first / bottom first discrepancy is a bug: Incorrect field order indication when encoding interlace top field first using h264_nvenc

I would add '-alternate_scan true' if your content is interlaced.

Thanks for the tip!

[pandy]

-vf is filter part i.e. processing - if you not process video then you can remove vf part completely.

As I stated, when i remove the setfield filter, ffprobe and mediainfo report the output video as bottom first, despite the source material being top first. According to ffmpeg's documentation, the setfield filter "does not change the input frame, but only sets the corresponding property", so can you still consider it processing the video? It seems to be doing exactly what the documentation says, setting the field order property on my output file to the correct "top first" value and nothing else. But I could just be misunderstanding ffmpeg's mysterious ways. However, my intuition tells me that the best workflow for video is to preserve the field ordering from source to destination.

Edit: It appears that this top first / bottom first discrepancy is a bug: Incorrect field order indication when encoding interlace top field first using h264_nvenc

It is not entirely clear to me what is your goal i.e. why you are changing BFF to TFF - is there any reason for this? This is explicit requirement?
If your goal is to change field dominance i.e. change from BFF to TFF or from TFF to BFF then this thread should be interesting for you https://forum.videohelp.com/threads/365238-how-to-invert-field-order-with-ffmpeg

https://ffmpeg.org/ffmpeg-filters.html#fieldorder
https://ffmpeg.org/ffmpeg-filters.html#phase

In first place i would not alter field dominance if correctly signaled - any valid decoder will follow signaled order flag - reverting field dominance may be important for signal flows where encoder expect particular field dominance type - such situation may occurs in broadcast but smart encoder should detect improper field dominance before encoding.

[wild-mouse]

I'm not trying to change field order, I'm trying to preserve the field order from my source which is VHS. VHS interlaced field order is top first. But ffmpeg is creating a video with the field order set to bottom first. Why ffmpeg is doing this is beyond me, and the only way I can seem to correct it is to include the setfield filter and the -top 1 options.