Matroska with improperly encoded video

[zhenya_hacker]

Hello everyone!
I have a problem with improperly encoded video in matroska container.

I have a videofile, downloaded from youtube, whis was improperly encoded: visually frames are interlaced with TFF, but youtube encoded it as progressive.
I don`t want to re-encode it, so I decided to mux it into matroska with interlace flags. I`ve set interlaced flag to be true and field order flag to be 1.
It`s enought, you think? Nope. Not even one of videoplayers, including smart tv, applied deinterlace while playback, - all of them decided to override matroska flags with same flags from video bitstream.

What am I doing wrong? Is there a way to make without reencoding a file (there is no restrict to use matroska), that will be played back with forcefully enabled deintarlace?

[davejavu]

You said the video has been encoded as progressive. If this is the case you can't deinterlace it as it isn't interlaced to begin with. Setting flags in the container will make no difference. Telling a software or hardware player to deinterlace it will make no difference.

It sounds like the original source was not properly deinterlaced when it was encoded to progressive, either by the uploader or by YouTube, so you have complete video fields which include interlacing artifacts.

You may be able to correct this with some filtering/processing but that will require you to reencode the video and is beyond my experience so I can't help you with that.

[manono]

And if it was resized while still interlaced, then nothing can deinterlace it properly anyway. You might consider uploading 10 seconds or so of it so we can have a look. But, since it came from YouTube I wouldn't be very hopeful, given the number of incompetent people uploading there.

[zhenya_hacker]

I'm talking about video like this one.

It was recorded in theatre at 1080i, edited at tv studios as 1080i, and it was uploaded as 1080i. And in 1080 quality there are 1080i frames.

Is there any way to force playback like if it is a playback of an avisynth script like this?

dgsource(source)
Yadif(mode=1, order=1) #or other deinterlacer. Yadif is chosen because of not worst double-framerating deinterlacing

And, if it is not possible, please lead me to some articles, how to reencode this as truely interlaced video with least possible losses in video. I think it's more preferrable than deinterlacing it to 25p and reencoding in this state. Thank you.

[jagabo]

If the video wasn't resized vertically you can force a player to deinterlace (VLC has the option, for example) and get passable results. But there will still be problems with the chroma channels if the video is YUV 4:2:0 (most common for MPEG family codecs like MPEG 2, h.264, etc.) -- the chroma of the two fields will be blended together.

By the way, what program lets you mux to MKV with an interlaced flag? Nevermind, I see MkvToolnix's header editor lets you specify interlaced field order.

[Drowning]

The video is not resized. Have tried to force player to deinterlace, but sometimes never able to pass it.

[jagabo]

After trying out the MKV header interlace flag -- no player I tried would see the flag and automatically deinterlace. I tried MPCHC, VLC, SMPlayer, PotPlayer, and KMPlayer.

[sneaker]

It's a fairly new Matroska element.

[zhenya_hacker]

After trying out the MKV header interlace flag -- no player I tried would see the flag and automatically deinterlace. I tried MPCHC, VLC, SMPlayer, PotPlayer, and KMPlayer.

That`s exactly what I am talking about.

[jagabo]

After trying out the MKV header interlace flag -- no player I tried would see the flag and automatically deinterlace. I tried MPCHC, VLC, SMPlayer, PotPlayer, and KMPlayer.

That`s exactly what I am talking about.

So, you'll have to reencode. Or wait until all the players start paying attention to the flag.

[zhenya_hacker]

Which option if more preferrable,

- to apply deinterlace, obtain 25p and reencode,
or
- to keep frames as they are and reencode it forcefully as interlaced frames?

For me, keeping frames is more preferrable as it can be yadiffed to 50p and played smoothly.

[jagabo]

First I would check that the underlying video is really interlaced. It may be from a progressive source (film) but rather than two fields from the same film frame being stored as one video frame, each video frame consists of one field from on film frame and another field from the next film frame. That looks interlaced but can be restored to progressive frames simply by recombining the fields properly.

If the video is really interlaced, 50 different fields per second, I would use QTGMC in AviSynth and make a 50 fps progressive video. That will get you better quality than any play-time deinterlacing.

As as stated earlier, regardless of what you do the chroma channels are permanently blended together. Anytime anything colored is moving you will see color blending artifacts.

[davejavu]

I'm talking about video like this one.

It was recorded in theatre at 1080i, edited at tv studios as 1080i, and it was uploaded as 1080i. And in 1080 quality there are 1080i frames.

Doesn't matter how it was filmed/editted because YouTube don't do interlaced video. They reencode just about any video uploaded to them and a quick google search suggests that interlaced video is automatically deinterlaced as part of their reencoding process, but not always very well. All the playback options on their site are for progressive video. What you downloaded may have started as interlaced but YouTube have changed it as you have aleady stated in your first post.

Without a sample of the actual video, or a link to the page you downloaded it from, we can't determine the technical specifications of what you are working with or the exact nature of the video issues you are experiencing. The video you linked to was 720p when I downloaded it so if yours was 'like this one' it has been resized from 1080.

[jagabo]

I'm talking about video like this one.

I missed that link earlier. It's true interlaced frames, 25 fps, 50 fields per second, encoded as if the frames were 25 fps progressive. So you can use a smart bob to clean up the combing. But the low bitrate caused a lot of cross contamination of the fields so there will still be some comb-like artifacts.

YouTube don't do interlaced video. They reencode just about any video uploaded to them and a quick google search suggests that interlaced video is automatically deinterlaced as part of their reencoding process

From what I've seen Youtube doesn't deinterlace the video, they just encode the frames as if they are progressive. That's certainly the case here.

[davejavu]

YouTube's own guidelines recommend deinterlacing video BEFORE upload and there are plenty of people on Google complaining about the quality of uploaded interlaced material after YouTube get their hands on it.

Surely if you put in 50 fields per second and YouTube output 25 frames per second it is getting deinterlaced somewhere in the processing chain, even if it isn't being done properly.

Every bit of software I use on that YouTube download tells me it is progressive. Ran an ffmpeg idet test and all frames detected as progressive with MPC-HC and Mediainfo both reporting progressive. However, I did a step through frame by frame in Avidemux and did see a few 'jaggies' in fast panning shots which suggest either it was uploaded improperly deinterlaced or YouTube have butchered it.

[jagabo]

Surely if you put in 50 fields per second and YouTube output 25 frames per second it is getting deinterlaced somewhere in the processing chain, even if it isn't being done properly.

If you want to call encoding 25 interlaced frames per second video as if it was 25 progressive frames per second "deinterlacing" go ahead. The point is, all 50 fields are still there in the luma channel and can be bobbed to 50 frames per second video.

[poisondeathray]

From what I've seen Youtube doesn't deinterlace the video, they just encode the frames as if they are progressive. That's certainly the case here.

YT deinterlaces the re-encode - but only if the upload was properly flagged . For example just upload a typical consumer interlaced camcorder video. The interlaced original is still stored, but cannot be accessed except by the account holder. The displayed versions are deinterlaced

In that example, it was interlaced content encoded progressive before uploading to YT. YT doesn't "scan" the actual content, it only reads flags

But it today's age, you can just upload 1080p50 / 1080p60 to youtube, that way you can have it display properly. That example is pure user error

But none of this helps the OP. Maybe find a better version , or smack the uploader

[davejavu]

The point is, all 50 fields are still there in the luma channel and can be bobbed to 50 frames per second video.

No, just tried that and the 'jaggies' are still there. As I said in my first post, I suspect the file has been botched.

The YouTube video linked only had minimal interlacing artifacts that I could see and not to the degree that would bother me too much, but the file the OP is working with may be worse. Since he hasn't provided a link to the file he is actuall using we can't know.

[poisondeathray]

Jagabo's point is that 50 moments in time per second will still be represented when you separate the fields, or bob deinterlace - that is interlaced content by definition

The quality will suck ass for various reasons mentioned earlier, also because of YT's progressive re-encoding, but you will still get 50 , not 25, moments in time represented

You can test this yourself, as I have - Just re-encode a normal 1080i file as progressive and upload to YT. The fields will still be there , it won't be deinterlaced

However, if an upload is "flagged" interlaced or mbaff , it will get deinterlaced and degraded , regardless of content. So even if you have progressive content, YT will butcher the re-encode by deinterlacing if it "sees" the flag

I'd probably do what jagabo suggested with QTGMC, because that will help clean up some of the artifacts and problems

[jagabo]

From what I've seen Youtube doesn't deinterlace the video, they just encode the frames as if they are progressive. That's certainly the case here.

YT deinterlaces the re-encode - but only if the upload was properly flagged .

Thanks for that clarification.

So the OP's claim that "It was recorded in theatre at 1080i, edited at tv studios as 1080i, and it was uploaded as 1080i" is wrong -- if he meant uploaded to youtube flagged as 1080i.

[zhenya_hacker]

So the OP's claim that "It was recorded in theatre at 1080i, edited at tv studios as 1080i, and it was uploaded as 1080i" is wrong -- if he meant uploaded to youtube flagged as 1080i.

Surely, it is not true. I`m sorry, it`s my mistake. I think, speaking in proper terms,

In that example, it was interlaced content encoded progressive before uploading to YT. YT doesn't "scan" the actual content, it only reads flags

Am I allowed to ask another topic-linked question?
I`m a classic newbie in x264 CL options, and I`m trying to reencode this example in x264 as fake-interlaced.

"F:\pelmeni\Budte_bobry\x264-r2833-df79067.exe" --pass 1 --slow-firstpass --fake-interlaced --profile main --level 4 --tune film --keyint 250 --b-adapt 2 --b-pyramid normal --ref 2 --deblock=1:0:0 --qpmin 0 --qpmax 69 --qpstep 4 --vbv-maxrate 10000 --vbv-bufsize 20000 --aq-mode 2 --stats "F:\pelmeni\Budte_bobry\Budte_bobry_for_reencode_x264.stats" --me tesa --trellis 1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --threads 3 --thread-input --frames 5000 --output "F:\pelmeni\Budte_bobry\Budte_bobry_for_reencode.264" "F:\pelmeni\Budte_bobry\Budte_bobry_for_reencode.avs" 


"F:\pelmeni\Budte_bobry\x264-r2833-df79067.exe" --pass 2  --fake-interlaced --profile main --level 4 --tune film --keyint 250 --b-adapt 2 --b-pyramid normal --ref 2 --deblock=1:0:0 --qpmin 0 --qpmax 69 --qpstep 4 --vbv-maxrate 10000 --vbv-bufsize 20000 --aq-mode 2 --stats "F:\pelmeni\Budte_bobry\Budte_bobry_for_reencode_x264.stats" --me tesa --trellis 1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --threads 3 --thread-input --frames 5000 --output "F:\pelmeni\Budte_bobry\Budte_bobry_for_reencode.264" "F:\pelmeni\Budte_bobry\Budte_bobry_for_reencode.avs"

I can`t find any crf mentions in CL command, but when 2nd pass starts, I`m getting this. Empty 0-byte output file is being created. What am I doing wrong?

avs [info]: 1920x1080p 0:0 @ 25/1 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
x264 [error]: constant rate-factor is incompatible with 2pass.
x264 [error]: x264_encoder_open failed

Never mind, I`m stupid. Forgot to set bitrate.

[poisondeathray]

If you're using 2pass, you need to specify a target bitrate (--bitrate xxxx)

Don't use --fake-interlaced, use --tff for MBAFF, top field first . "Fake interlace" is for progressive encoding and compatibility for things like BD where some native progressive framerates are illegal . You want the player to bob deinterlace automatically, for that you need PAFF or MBAFF (you can think if it as interlaced)

Some of your settings have severe diminishing returns, for example --me tesa is going to be useless and dramatically slow down encoding speed

[zhenya_hacker]

If you're using 2pass, you need to specify a target bitrate (--bitrate xxxx)

Don't use --fake-interlaced, use --tff for true interlace . Fake interlace is for progressive encoding and compatibility for things like BD where some native progressive framerates are illegal . You want the player to bob deinterlace automatically.

Some of your settings have severe diminishing returns, for example --me tesa is going to be useless and dramatically slow down encoding speed

Yeah, obviously, I forgot bitrate. Thank you.

Source video has

true interlaced frames, 25 fps, 50 fields per second, encoded as if the frames were 25 fps progressive.

and I`m trying to preserve it rather than making new artifacts with tff encoding.

Which --me if preferrable due to source is video downloaded from youtube? Look above, there is quite a few words about this video.

[poisondeathray]

Source video has

true interlaced frames, 25 fps, 50 fields per second, encoded as if the frames were 25 fps progressive.

and I`m trying to preserve it rather than making new artifacts with tff encoding.

Which --me if preferrable due to source is video downloaded from youtube? Look above, there is quite a few words about this video.

If you encode --fake-interlace then you will be left with the almost the same problem as YT in the first place. Some TV's or players won't deinterlace, but some might. You want --tff because all players will treat it correctly.

If you want make it "better", then use QTGMC , to make it 50p . It's like yadif 2x, but better . The only time you might not want to do this is if your hardware doesn't support High@L4.2 playback (1080p50) . Older TV sets , BD players etc... are limited to L4.1, which isn't compatible with 1080p50

--me is motion estimation, but tesa will just make it much slower to encode with 0.0001% benefit at a given bitrate, especially since this is low motion, indoor studio set content. Even with high motion, very fast sports it makes only very little difference. hex or umh as the most. There are better settings to adjust , more "bang for buck" , such as references, b-frames, high profile which make a real difference.

[sneaker]

Yeah, use the --preset options to adapt speed instead of manually switching --me etc.

[zhenya_hacker]

Btw, setting --tff makes it to use scan type "mbaff" with scan order "tff". Is there a way to make it encode with interlaced scan type, not mbaff?

[sneaker]

mbaff is interlaced. You can't change that. And shouldn't.

[zhenya_hacker]

mbaff is interlaced. You can't change that. And shouldn't.

I know that mbaff is interlaced. And it cannot be concatenated with non-mbaff interlaced file without reencoding.

I have big non-mbaff interlaced file

Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile              : High@L4.0
Format settings, CABAC      : Yes
Format settings, ReFrames   : 3 frames
Codec ID                    : 27
Bit rate                    : 10.0 Mbps
Width                       : 1 920 pixels
Height                      : 1 080 pixels
Display aspect ratio        : 16:9
Frame rate                  : 25.000 fps
Standard                    : Component
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : Interlaced
Scan order                  : Top Field First
Bits/(Pixel*Frame)          : 0.193
Stream size                 : 7.47 GiB (90%)
Color primaries             : BT.709
Transfer characteristics    : BT.709
Matrix coefficients         : BT.709
Color range                 : Limited

And small mbaff interlaced file

Format                      : AVC
Format/Info                 : Advanced Video Codec
Format profile              : High@L4.0
Format settings, CABAC      : Yes
Format settings, ReFrames   : 4 frames
Codec ID                    : 27
Bit rate                    : 7 856 Kbps
Width                       : 1 920 pixels
Height                      : 1 080 pixels
Display aspect ratio        : 16:9
Frame rate                  : 25.000 fps
Color space                 : YUV
Chroma subsampling          : 4:2:0
Bit depth                   : 8 bits
Scan type                   : MBAFF
Scan order                  : Top Field First
Bits/(Pixel*Frame)          : 0.152

I want to concatenate them. They can`t be concatenated directly (or I haven`t found way to do it properly). Mbaff file is shorter, so is there a way to encode it with interlaced scan type, not mbaff?

[poisondeathray]

The first file is PAFF , you can't encode PAFF with x264, it's MBAFF only . Commonly used implementations which offer PAFF are Mainconcept AVC (e.g. in NLE's like Premiere Pro/AME, Sony Vegas) . Quality will be lower at a given filesize (x264 is generally rated the highest in terms of quality/ AVC compression efficiency; also MBAFF > PAFF in terms of efficiency) . All free software AVC encoders use x264, thus no way of encoding PAFF very easily, unless you use one of the commercial applications . Another option might be one of the GPU encoders like NVENC, QSVEnc , if you have compatible GPU - IIRC they offer PAFF , but again quality/compression is lower

[jagabo]

Yes, if you have an Intel CPU with Quick Sync -- QSVEnc uses PAFF with the --tff or --bff switches.