Variable resolution video

[TrevorGryffits]

Hello everyone.

I have to save stream (encoded with h264) to MKV (preferably) or MP4 file. However, encoded video has varying resolution. Part of this stream is sent as 720p and the other - in 720x576. All frames reference to PPS and SPS with indexes equal to 0. I know that I can scale both "substreams" to same resolution (e.g. 720p). However, I would like to avoid this if other solution is possible. And by "possible" I mean "work without problems at least on VLC on Windows and Linux"

Stream is sent with AVCC bitstream. During muxing whole extradata (SPS + PPS) are save to CodecPrivate field of Track structure in MKV file. However, in this situation only one part of stream is decoded correctly during playback, because the other half has no valid SPS/PPS. Bad decoding results in "grey" video with static, errors coming from encoder, etc.

I have tried using CodecState field, but with no luck. I was filling this field with new extradata each time stream has switched. It did not change anything.

So, is there any practical solution for this problem? Besides scaling streams.

Thanks in advance.

[El Heggunte]

If I remember correctly, transport streams support variable video resolution.

You should give a try to both ffmpeg and TSmuxer, and see which one gives you the best result.

[hello_hello]

Another option might be to keep the sections with different resolutions as individual MKVs and link them. I'm using an older version of MKVMergeGUI but for me it goes like this.
Open MKVMergeGUI.
Use the File/Header Editor menu.
Open the first MKV "segment".
Under "Segment Information/ Segment Uinique ID", make a note of the ID number. Repeat the process for each segment (opening a new instance of MKVMergeGUI for each segment might make it easier so you can keep them all open).
You'll see two other fields labelled "previous segments unique ID" and "next segments unique ID". You'll need to fill in those fields with the appropriate ID numbers. When that's done correctly and saved you'll have linked MKVs.

How various players will handle them......
Hardware players, probably not at all.
Software players should either automatically play each segment one after the other as a continuous file or loading one segment will load all of them as a single file (it depends on the player and splitter). A player may cope with the change of resolution, it may crap out, and for VLC I have no idea as I don't use it. I'm not even sure it supports segmented MKVs (you'd need to test it) but in a perfect world MKV segments should be able to contain completely different video or audio types and still work. If a player is using hardware decoding (DXVA etc) and it doesn't cope with the resolution change, try disabling that.

[TrevorGryffits]

Thank you for your replies.

For now I have decided to stick with segmented MKV. This feature is suprisingly well supported. VLC switches streams withour visible problems. It shows some error in console, but video plays fine. Even Windows Media Player plays such file (after installing proper codecs). Currently, at each resolution change I'm closing one segment and open another. These 2 segment are linked together with NextUID/PrevUID fields. New segments contain new tracks with correct CodecPrivate, PixelWidth and PixelHeight fields.

I have tested transport streams also, but without success. However, I have used only TSmuxer (without going deep into its capabilities), so it is possible that I have missed something. MKV files are widely used in other software I use, so previous solution is better (for now...).