Mark/Force MBAFF Encoded Video as Progressive

[ajingo]

I have some videos which are MBAFF encoded are actually progressive. These are MKVs.
Normally if not MBAFF it is just a case of marking as progressive and this is fine.
I have tried setting FlagInterlaced=2, FieldOrder=0 to no luck. After setting these MediaInfo still shows MBAFF.

My main issue is playback on android devices using MeidaCodec.
Both a Fire TV and Tablet (S5e) attempt to deinterlace this video, even with these flags set.
I have confirmed this on the fireTV as the MediaCodec framerate shows as 50 in the advanced information, and on the Tablet there is obvious quality loss, (the deinterlacer there is really poor, but on testing with actually interlaced content, does attempt something).

I would like to stop MediaCodec from attempting to deinterlace without reencoding.
Below is the MediaInfo output.

Thanks in advance.

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Bit rate                                 : 3 947 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 25.000 FPS
Standard                                 : Component
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Interleaved fields
Scan order                               : Top Field First
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

[Mr Whippy]

I've got a similar problem here.

All my files recorded from DVB-S come in almost exactly the same as yours in media info after they've been remuxed to MKV.
50hz for telly, but actually 25fps and most frames are progressive in most cases (turn off dei-nterlacing in VLC or watch with ffplay etc). I've only noticed stuff like The Simpsons and things like that have used interlacing, but it's not proper 50hz interlacing, you just end up with 25fps with ghosted frames... or film credits can sometimes look better with the de-interlacing on.



My main issue a little like yours in that YADIF is invoked when transcoding on Jellyfin because these files are detected as interlaced, and yadif seems to cause some softness to these 25p 'interlaced' images which aren't, when transcoded like that.


Re-encoding this type of footage using a de-interlacing method does obviously resolve this detection/flagging issue, but it isn't really correct I don't think, because it does detriment the quality.

What I used for a test was setting the MKV flag in MKVToolnix to 2 (Video Interlaced Flag), which was then detected by FFmpeg in the stream info, which then made FFMPEG happy to transcode the video stream and *not* set the new stream as MBAFF in any way, resulting in a clean progressive detectable file, and without having any de-interlacing process run on it.



It's deeply annoying that this is such a problem because it seems MBAFF encoding does specify on a per-frame basis if things are interlaced or not.

In practice, your file may be MBAFF encoded but all frames are marked as non-interlaced. Thus a de-interlacing shouldn't run if that is the case.

But in my case, Jellyfin is invoking the YADIF filter (ffmpeg) yadif=1:-1:0 which assumes all frames are interlaced and runs the deinterlacing. If it were set to yadif=1:-1:1, then it'd only deinterlace frames set as interlaced.



I'm going to dig into an mkv in mkvtoolnix and see where these mbaff metadata are set. I assume for your footage and my footage these flags are set to indicate the footage isn't interlaced, and poorly implemented players are just assuming mbaff being present means it's interlaced footage.


Cheers

Dave

[ajingo]

I didn't think to try my modified files via Jellyfin (already setup).
Setting the flags does override interlacing for jellyfin, completely skipping de interlacing on the server side.
And the Android client also seems to skip the deinterlacing through jellyfin - so that is my issue almost worked around.

My main issue is android clients as it is necessary to use the hardware decoders (MediaCodec), so I am stuck with trying to trick it into not deinterlacing.
Playback on PC is fine as I just stick with mpv, allowing me to toggle deinterlacing.

I'm going to dig into an mkv in mkvtoolnix and see where these mbaff metadata are set. I assume for your footage and my footage these flags are set to indicate the footage isn't interlaced, and poorly implemented players are just assuming mbaff being present means it's interlaced footage.

I have a feeling the MBAFF info that MediaInfo shows is set on the stream, not the container. I'll see if I can find what I initially read that suggested this.

(Edit: I did find this post https://forum.doom9.org/showthread.php?p=1836724)


Otherwise last resort would be do reencode, but force no deinterlacing (or, funilly enough, what my last question about - only deinterlacing interlaced frames.)

Thanks for the suggestions.

[Mr Whippy]

Yes MBAFF related info seems to be inside stream, not mkv container, from having a poke around in MKVToolnix etc.

At my end, just setting that flag to 2 (to force ffmpeg to see progressive) just crashes my Jellyfin transcode request, it just hangs with the swirling circle. However JF does rightly show the video as 1080p not 1080i, and the info shows progressive not interlaced.


I read an interesting thread on jellyfin forums about mkv and frame rates/interlacing detection. It seems mkv see 25fps 'interlaced' as 50 'events' per second, despite the frames duration being 40ms = 25hz... the fields are counted as frames for frame rate detection in ffmpeg.
Obviously not an issue with other containers, or progressive in mkv, but interlaced in mkv is causing issues there.

So ffmpeg within jellyfin is/wasn't playing nice on mkv framerate detection and yadif deinterlacing (not sure if that tweak is in 10.7 when it arrives or made it into 10.6.4)

In any case, mkv being the relatively unadopted container it is, and it's specific way of holding info, may be struggling just because frame rate detection methods may be using the fields as frames, rather than the frame duration as frame rate (mkv doesn't specify actual frame rate anywhere, it's just inferred from metadata in each block iirc)




Since my main concern with these files is DVB-S .ts streams, I've been using videoredo to tidy them up and put to mkv.

I did a small test and transcoded to hevc, using no de-interlacing, and smart de-interlacing.

Both files appear identical in mediainfo and ffmpeg stream readouts etc. However one was much smaller than the other.

Only upon reviewing in VLC with interlacing off was it clear that the larger file with interlacing off during transcoding, showed actual interlaced frames.

The smart de-interlaced transcode (rightly detected as non-interlaced by ffmpeg) appeared to show a frame of superior visual quality, to the former file with interlacing active.

So I'm not sure what is happening there.

All I can assume is that videoredo is striping each half of the 25p frames as told to by being interlaced, and is mixing the 2nd half of one frame with the first half of the next, giving 50 fields of actual data, but still at 25 full frames per second.

Whether there is some actual interlacing in raw HD .ts streams I can't be sure.



I'm now going to do some tests with videoredo and check how the quality looks with the smart deinterlacing transcoding, vs the .ts stream from VLC with interlacing off (which looks sharp)

If videoredo is managing to transcode the footage and look 'correct' and strip out this 50hz information and put it to true 25p then I'll be happy doing that.




It's all very confusing though. It'd be good to know what the actual raw AVC content looks like. Is it actually 50 fields a second, and they line up perfectly to make 25p when the base content was 25hz or so (ie, films)?
It does feel to me like you don't even need yadif or deinterlacing for this task. It feels like it's just a case of taking both fields and overlapping them if the source was 25fps (it'd give the usual 'interlaced' effect if the source was 50fps)

But then I expect if the AVC is encoded with fields then you need to re-encode to get from 50 fields to 25 fields. Indeed maybe that is why my test gave me such a different file size despite the same video information. Both 25fps etc, just one with split fields and the other not.

Which then makes you think that transcoding is probably best with this footage any way (assuming it's 25fps source), because it's pretty inefficient to store 25p as 50i!

[Mr Whippy]

I've been reading a bit more to understand properly what is going on.

For me the 50i is essentially 50 fields per second, stored in 25 full frames per second of interlaced data. All good.

Actual 50hz recordings obviously then show up in those 25 full frames with half the data from the 1/50th second later, the "interlaced look"
Actual 25hz recordings then show up with both fields being from that full 25 frames source, so there is no half data 1/50th second later, the 'progressive look'

But I assume because it's MBAFF and the streamer shouldn't care as it's all going to 50hz TV, there are 50 fields per second even with 25p content, and frames 'look' like 25p, even though they're 25i.


I assume AVC does 25fps header inside TS (well possibly even just using timestamps), and then another flag probably tells what interlace method it is, and then stores the fields.
Inside MKV the information is stored in such a manner that some players are dealing with it differently, using inferred specifications, or different flags, and ultimately not dealing with the data accordingly.

Technically, 25p source in a 25i file doesn't need de-interlacing, as the pair of fields per 1 frame are aligned.
MBAFF flag might be used to set interlaced or not, but then I assume from the streamers end, they can just stream it interlaced with the flag any way as the TV will run fine with it any way. Ie, if they set MBAFF to non interlaced, then technically they'd have to send non-interlaced frames. They are still sending interlaced frames, they just happen to look ok because the source was 25p.


It's the progressive viewers that struggle, potentially, and MKV format and low adoption/support/mature standards means random behaviour.



So I think the best thing to do initially is try a different container.

MKV is nice for Jellyfin because of the support for several audio and subs streams. However .ts is too. Why not try put the video in .ts or .mp4 to check how things look?


Also re-encoding is an option. But I'm currently a bit iffy on the quality of de-interlacing 25p stored at 25i because technically it doesn't need de-interlacing, just the raw 'on screen' pair of fields can be sent straight to a new progressive field.
Ie, if you dumped all the field pairs and then overlaid them all and re-encoded, you'd need no de-interlacing pass.

I'm just unsure if the de-interlacing is a bit bad for image quality. With 25i streams you're already re-encoding from a fairly lossy format so the last thing you need is an unnecessary step softening the video.

I think I'm going to test here by comparing the .ts source and then doing some lossless outputs with yadif and videoredo smart deinterlace, and overlay them with a difference layer in After Effects and see if there are any notable issues.



Going back to what you initially said, it'd be great if we could just flag it as 25p... but I think the issue is it's encoded (assuming yours is an AVC stream from broadcast) at 50 fields per second, or 25i... so it's not progressive, it just looks progressive.

[badyu17]

Couple of points that may or may not help you: mkvtoolnix stores 25fps interlaced streams as 50fps aka one field as one matroska block. VideoReDo uses ffmpeg for muxing but is doing things differently - one pair of fields as one matroska block so it shows in mediaInfo as 25fps. The reason for this is that mkvmerge is following matroska spec precisely (judging by earlier statement from mosu, the mkvtoolnix author) while ffmpeg is not following this spec precisely. The more confusing thing is that mbaff doesn't separate fields in the stream, they are coded as full frames, so which muxing method is correct is open to debate. On the other hand picaff is coding as two separate fields in the stream but keep in mind both are adaptive so the only thing you shoud look at is the type of footage: if it is film then you don't need to deinterlace at all because mbaff is adaptive. It will code progressive footage as progressive. If it is sports and news then it will code interlaced macroblocks as interlaced and some macroblocks as progressive because it works on a macroblock basis. The problem is that players see mbaff encoded flag and apply deinterlace on all frames.

[ajingo]

Nice to know how Mastroska works internally. I'll have to try ffmpeg or even changing container to see if anything helps out.
For now, setting the interlaced flags as I described before prevents Jellyfin from transcoding.

For my original problem, mpv for android allows toggling HW/SW decoding which will disable the automatic de interlacing.
Surprisingly software decoding performs well enough on a Snapdragon 670 for 1080p and for 1080p50 I find software decoding performs better than HW.
No idea what is going on there - but it works. Hw shows as (mediacodec-copy), so maybe a issue copying frames back to ram?
Here is the difference.
Another oddity, my phone which has the same Snapdragon 670, "tries" to deinterlace but utterly fails, producing a green and pink mess.
FireTV also deinterlaced but the deinterlacer is actually of a (somewhat) decent quality (one frame per field, small quality loss)

Anyway summary - If anyone else has issues with media-codec automatically deinterlacing progressive material - Try MPV.

[Mr Whippy]

Great post Badyu17, that's helped a lot... I think.


Just for reference here ajingo, if I set the progressive flag in mkvtoolnix (so when media is snooped with ffmpeg -i it shows as progressive and 25fps), then Jellyfin does then flag it as progressive and 1080p rather than interlaced 1080i... however it just breaks transcoding at my end.


I've also been testing transcoding the 1080i ts files > 1080p via VideoReDo which is mostly successful, but the deinterlacing thing can be a pain there as there is no explicit "none" setting. Just auto, which then chooses "none" for you if the footage is detected as progressive, or should I say, having no combing or actual temporal interlacing.

However I've noticed lots of TV films I record are 25p, then somehow speed up the end credit visuals and interlace them at 50 fields, so 'none' isn't ideal as the end credits look all combed, and 'smart' is far superior... seemingly giving 'none' during the 25p parts, and then doing a really nice deinterlacing for the credits into true 25fps.

But it's all so very hard to check what is going on there.


So I tried ffmpeg to do this task. But seemingly with my 1080i ts > 1080i mkv, then ffmpeg mkv in > out mkv, the interlacing remains but is broken (looks like swapped fields). Documentation suggests ffmpeg won't interlace anything unless told to, and should be defaulting to progressive coding and 'none' deinterlacing.


I'm also pondering how 1080i is handled by Jellfin > Infuse > Apple TV 4k > 1080p TV...
In theory the TV is taking even fields, interpolating (deinterlacing) data, showing it 1/50th sec, then grabbing the odd fields, interpolating (deinterlacing) data, showing it 1/50th sec... so half the data you're seeing is interpolated over two otherwise identical looking frames.
It does seem that you're going to get a much better quality from 1080i recorded content if you transcode appropriately, and have your TV display 1080p and just frame double the content... so you get two identical frames but made from no interpolated data.


On my mobile phone Infuse is pretty battery heavy, but it's a shame there that Jellyfin is doing a sub-standard transcode with yadif too, so having the source files in true progressive would be useful for that reason too.


If Jellyfin could just transcode with 'none' rather than running yadif (or try smart detect the combing and run it when needed, or even mwdif), and if I knew the TV/ATV was handling the stream as a progressive frame and doubling it up, it'd be ok.


For now it's not a big issue as it all 'works' and looks pretty good. Just if it looks it's best is another question.


The main issue I think with doing anything with the TV streamed HD AVC footage is that it seems very very good quality for the size. I've done some HVEC on slow and the file sizes have ended up bigger than the AVC, and ultimately they have generational loss in the quality so have probably degraded a bit.



Random other observations, for those who like the DVBSUB that come with streamed TV content, I'm sure I've noticed those getting a bit broken the more files are processed... I know they shouldn't as they're a separate stream.
I've had no issues with VideoReDo from ts > mkv and trimming.
I've seen issues with VideoReDo re-encoding AVC > HEVC and deinterlacing and making the DVBSUB scaling be doubled and they can end up overlapping. This is also seen in Infuse. But *not* in ffplay, which is where I check!
I've seen ffmpeg crops (remove black bars) causing aspect ratio issues with DVBSUB projection.

So yeah, DVBSUB are a bit sensitive to having the video/container side of things played with... so another argument to just leave the recordings as they are.