Help: MKV file performed broken pixels after muxed by tsMuxeR

[anhlinhp4]

I often use tsMuxer to mux MKV file to see on tivi via my Sony BD.
Most of MKV files (after muxed by tsMuxeR) are played good, sometimes some files just give broken pixels on my entire tivi screen that I cannot wacht at all. Anyone here gets that problem like me?
I need help to resolve a wrong MKV file. Here it is:
File A:

Format : Matroska
Format version : Version 4 / Version 2
File size : 2.78 GiB
Duration : 59mn 9s
Overall bit rate : 6 735 Kbps
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L5
Format settings, CABAC : Yes
Format settings, ReFrames : 9 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 59mn 9s
Bit rate : 5 963 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.120
Stream size : 2.46 GiB (89%)
Writing library : x264 core 148 r2665 a01e339
Encoding settings : cabac=1 / ref=9 / deblock=1:-2:-2 / analyse=0x3:0x133 / me=umh / subme=11 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=48 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Default : Yes
Forced : No

The above file cannot be played, just performed broken pixels after muxed by tsMuxeR.
I have tried to play that orginal MKV file (not muxing by tsMuxeR), same as result !! It means something wrong in this MKV file.
The question is how to fix this file by using tsMuxer?

Below is the good MKV file that played good (after muxed by tsMuxeR) for your compare:
File B:

Format : Matroska
Format version : Version 4 / Version 2
File size : 2.26 GiB
Duration : 59mn 20s
Overall bit rate : 5 462 Kbps
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 59mn 20s
Bit rate : 5 000 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.080
Stream size : 2.08 GiB (92%)
Writing library : x264 core 128 r2216 198a7ea
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=5000 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=50000 / vbv_bufsize=50000 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Default : Yes
Forced : No

I don't know why file B is played good, but file A is not. And how to resolve that matter of file A?
Anyone here has idea? please help!

[poisondeathray]

You can't fix it with tsmuxer alone; You would need to re-encode it using compatible settings

Most BD players only support High Profile, Level 4.1, 4 reference frames max at 1920x1080 resolution

[anhlinhp4]

Hi, poisondeathray
Thanks for your idea.
it sounds bad for me to hear that I have to re-encode that MKV file, because it will take a long long time to re-encode it, and I do not like that at all!!
Anyway, thank you for your kind respond.
No other way for that matter?

[El Heggunte]

It means something wrong in this MKV file.

Nope — it means the file you downloaded was intended to be played on a computer, via software.
As poisondeathray already explained, hardware video players have limitations that the software players do not have.

[ron spencer]

If you are interested in hardware compatibility, when you see the following:

bluray_compat=0 vbv_maxrate=50000 vbv_bufsize=50000 nal_hrd=none

you are likely to have issues, as these are obviously not bluray compatible formats, especially with bluray_compat=0


If you need hardware compatibility, the following, with a 2 pass encode, will pretty much make sure all is ok as x264 will do the proper sanity checks:

bluray-compat=1:no-fast-pskip=1:no-dct-decimate=1:vbv-bufsize=30000:vbv-maxrate=40000: slices=4:ref=4:keyint=24:no-deblock=1:no-mbtree=1

haven't found a hardware player yet that won't play with above options set in an encode.

As El Heggunte said:

hardware video players have limitations that the software players do not have

[anhlinhp4]

Thank you, El Heggunte & ron spencer,
Thank you both for your particular knowledge that help me to understand the matter more.
As El Heggunte said, it's right that file A (bad MKV file) & file B (good MKV file) are both played well on my PC via software.
But now, let's see again the file B (good MKV file) that is played well by my Sony bluray player (after muxed by tsMuxer) as I mentioned above.
This file has endcode setting:

bluray_compat=0 vbv_maxrate=50000 vbv_bufsize=50000 nal_hrd=none

and as ron spencer said, it cannot be compatilbe with bluray player!
I don't understand this point. Please explain me more.
[Also remind you of that file A (mentioned above) is bad file (not file B), and cannot be played by bluray player]
Thank you all in advance.

[sneaker]

bluray_compat=0 only means it is not encoded strictly to BluRay specifications. That matters if you want to author a BluRay, let it be replicated in a big company and sell it to customers. If you want to put H.264 in MKV (MKV has nothing to do with video BluRay specifications) it does not matter most of the time or may even hurt some players.

[anhlinhp4]

Thanks for your opinion, sneaker.
Here I just wonder that both file A & file B have the same encoding setting: "bluray_compat=0, ..._hrd=none, ....", but the file A cannot be played while the file B is played well by my bluray player (after muxed by tsMuxer) (?!).
And I like to find a way to fix the file A, if possible, to be able to see it by bluray player (it's my joy )
After all the ideas above, it seems that only re-encoding the file A should be done if I want to see it by player. It sounds bad.

[El Heggunte]

............
but the file A cannot be played while the file B is played well by my bluray player (after muxed by tsMuxer) (?!).

File A is ref=9, whereas file B is ref=3

OFF-TOPIC: your avatar is horrible , please change it.

[anhlinhp4]

Hi, El Heggunte
Why is the ref of file A so high? It was set manually by the encoder?
To change that ref, the only way is re-encoding the file, right?

*P/S: Not many avatars for me to choose here.

[sneaker]

To change that ref, the only way is re-encoding the file, right?

Correct.