Header stripping

[jagabo]

And i don't believe there isn't

Yes, it's a secret. You're not part of the club so we're not telling you.

[Selur]

I'm wondering if ffmpeg uses an older version ...

iirc ffmpeg uses it's own code for mkv creation.

also fixMKV is a simple script to remove header removal compression (it only remuxed files which use header removal compression)

[_Al_]

I'm not sure where it stands now, because as a non English speaking person, word-order itself: Disable header removal compression is a mind blowing moment, to figure out what it actually does if I check the box.

There is always possibility to set compression to none in "extra option" TAB anyway regardless of settings in options, is it the same thing?

[Cornucopia]

Yeah, I guess that's like "anti-disestablishmentarianism". You've got to be able to parse the roots of the word in the correct order to even begin to understand what is the side one should take. Knowing the history will help you here, though.
Originally: Full Headers
Later came: Compressed Headers (because some were able to be removed)
Then came: Full Headers again (because compressed headers were found to not be compatible with some devices/players). Hence, "header removal/compression" being disabled.

Scott

[Selur]

small history from my memory to extend Cornucopias post:

As far as I remember, starting around 2005 Matroska first supported 4 different ways to 'compress' headers:

1. zlib
2. bzlib
3. lzo1x
4. Header Stripping

before that it had no support with for any header compression.
bzlib and lzo1x require extra libraries to be linked while building mkvtoolnix, which is why near to no people used it and why these methods were dumped a bit after the specification.
So basically regarding header handling there are three options:
a. no header compression
b. header compression through zlib
c. header removal compression (meaning duplicate headers simply will be removed) aka. header stripping

Since zlib is the most common compression method and subtitles are normally better to compress than other streams it became common to compress the subtitle stream with zlib and leave the other headers uncompressed.

Sadly hardware player developer companies notoriously try to take the cheap way, so a lot of players didn't support anything apart from a. no header compression and b. zlib header compression. Header removal 'compression' was left out.
This is partially due to the fact that using zlib on subtitles and leaving the other headers as they are became the default method in mkvtoolnix and most users never looked at anything that was not the default option. Problem is some hardware players only supported no header compression, a lot supported zlib compression, but only for subtitles, but only a rare few supported all three methods and some players even required zlib to be used on subtitles and didn't support the subtitles otherwise.
Pissed of by the fact that a lot of hardware player didn't supported the spec properly Mosu (the mkvtoolnix author) decided to change the default compression setting in mkvtoolnix (to header removal) to get people to be aware of the poor job the player (and splitter) developers did.
Like expected, this caused a lot of commotion and since a lot of people kept having problems with it Mosu switched the behaviour back to no compression + zlib for subtitles in mkvtoolnix 6.0.

Personally I think all in all it wasn't a bad move from Mosu to switch to header removal compression, it send a lot of publicity to Matroska and got people to be more aware of the features and specification of Matroska.

Side note: The whole thing caused people to write scripts like fixMkv which remove the header compression completely,... it would be more efficient (container size wise) to still stick to zlib for subtitles but since people are not that size fixed nowadays with CDs dying out for backups (in the western countries of the world), nobody really cares.

Cu Selur

[Cornucopia]

Thanks for that helpful extra info, Selur! I didn't know the full history of it.

Scott

[jman98]

Thank you Selur. But I disagree here.

Personally I think all in all it wasn't a bad move from Mosu to switch to header removal compression, it send a lot of publicity to Matroska and got people to be more aware of the features and specification of Matroska.

While this may be exactly what happened with technical users, most MKV users are people with little to no technical expertise in video. All this did was make them say "My video is broken!". The fact that you had to explain this also proves that even among technical people like us, this reasoning was not well known. And as far as I know there is not even one stand alone hardware player that supports MKV that understands how to play files with stripped headers.

I seriously question the wisdom and necessity of ANY header compression. For God's sake people, this is not the late 1990s when people only had CD burners and even those were expensive and it became important to save every bit possible to fit Divx encodes into a single CD-R disc. How much space does header compression save any way? Maybe I am really underestimating its size but it seems to me that compressing headers provides very little useful space savings in today's world of 128 GB (and bigger) flash drives and 4TB hard drives.

[jagabo]

as far as I know there is not even one stand alone hardware player that supports MKV that understands how to play files with stripped headers.

There are a few. My Western Digital WDTV Live and LG BD670 Blu-ray player, for example. Originally they didn't but firmware updates added the ability (a few months after the MMG change in the case of the WDTV, not sure when the BD670 added it).

I seriously question the wisdom and necessity of ANY header compression... How much space does header compression save any way?

Typically on the order of 1/10 of one percent.

[_Al_]

I was remuxing with this compression for a while not even realizing it because one WDTV Live Plus I was using could play it. So was this good for me or bad ? I might migrate to another player and might have problems. I think the mkvmerge author should leave default to none.

[El Heggunte]

I think the mkvmerge author should leave default to none.

Mosu has done that already.

2013-01-01 Moritz Bunkus <moritz@bunkus.org>

mkvmerge, mmg: removal: The ‘header removal compression’ method is not turned on by default anymore. This affects the following track types: AC3, AVC/h.264, Dirac, DTS, MP3. The setting in mmg that turned it off by default has been removed.

( http://www.bunkus.org/blog/ )

[_Al_]

yes, ..., anyway thanks to him for that free software and taking care of mkvtoolnix

[Selur]

I agree all-in-all Mosu is doing a splendid job maintaining mkvtoolnix.

[fits79]

So because i don't understand much(i don't know very well english) did you create a program that we can search and find out wich mkv has audio head stripping or a program that remove the audio head compression to many mkv files at once?

[Selur]

there's a batch for that called fixMV, I linked to it a few posts up

[DarrellS]

I'm wondering if ffmpeg uses an older version ...

iirc ffmpeg uses it's own code for mkv creation.

also fixMKV is a simple script to remove header removal compression (it only remuxed files which use header removal compression)

Thanks, Selur!

[jagabo]

Apparently not good enough for fits79.

[fits79]

NEVER SAY THAT.

The only think i said is i DON'T know how exactly to use it.