Splitting and appending messes with the length from UHD rip

[Perenista]

I posted all this about my issue... do you have any idea what is going on?

It's about MKVToolnix. When I say this doesn't affect content is because internally the movie and its tracks are still the same. But since the length increased I can't insert a new track after all the splitting and appending, because it will be out of sync.

This is what I explained:

**********

Am I wrong when I say that if I do this:

- Split a 89 GB Matroska file (4K resolution) into 15200M or 15300M parts;
- Then append all these parts...

The resulting file will somehow grow a few seconds longer, thus messing with audio sync in the middle?

Because that's exactly what happened here.

And no reports of any error with this job. I will try to create the original Matroska again, and repeat all previous steps.

But I am concerned now that splitting poses a serious risk of damaging the content forever, and if this is the only copy, then I need to identify what has caused this issue.

Because I've been doing exactly this: splitting and appending many different contents, yet I doubt I have seen this happening before.

The only change I did to the original MKV which was more or less 4 seconds shorter was to add a few audio/subtitle tracks. Mosu needs to clarify if splitting and appending sometimes breaks the file the way I described. Or perhaps I did something wrong during my edits...

*****
This is the file I just appended all 6 parts:

https://pastebin.com/dTMTJ70W

This is veeeeery odd. Because when I compare to a smaller one (4 GB, 1080p instead of 4K) the 1080p file has 3 hours, 51 minutes and 38 seconds. While the 4K has 3h51m42s.

For example, if we pause at these moments they will be different:

4K:
00:39:04.431

1080p:
00:39:04.282

******

4K:
02:08:46.470

1080p:
02:08:43.518

******

This can't happen. In the 2nd scene the moments should be the same. Not a 2-3 second difference between them.

I am now thinking this could be it:

4K:
Frame rate : 23.976 (23976/1000) FPS
Original frame rate : 23.976 (24000/1001) FPS

1080p:
Frame rate : 23.976 (24000/1001) FPS

1080p MEDIA INFO: https://pastebin.com/nDB5gZd7

Apparently MKVTOOLNIX inadvertently changed the FPS to something else, which could account for this abnormal result while appending the splitted 4K file.

Or it could be MAKEMKV's fault, while creating the Matroska.

I am going to have to redo this MKV, from the original 4K disc.

What is odd is that this disc shows the following info in the playlist report:

************
PLAYLIST REPORT:

Name: 00040.MPLS
Length: 3:51:36.549 (h:m.ms)
Size: 98,182,275,072 bytes
Total Bitrate: 56.52 Mbps

(*) Indicates included stream hidden by this playlist.

VIDEO:

Codec Bitrate Description
----- ------- -----------
MPEG-H HEVC Video 47387 kbps 2160p / 23.976 fps / 16:9 / Main 10 @ Level 5.1 @ High / 4:2:0 / 10 bits / 4000nits / HDR10 / BT.2020
* MPEG-H HEVC Video 2101 kbps (4.25%) 1080p / 23.976 fps / 16:9 / Main 10 @ Level 5.1 @ High / 4:2:0 / 10 bits / 4000nits / Dolby Vision FEL / BT.2020

************

So what is going on here?

*****
Update: confirmed: recreated MKV from the 4K disc (using MAKEMKV) has now 3h51m36s. I am going to do further tests to see if something changes.

MEDIAINFO:
https://pastebin.com/6pMt4ki9

Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS

Next tests:

- Check if using MKVToolnix adds a few more seconds to it, and/or change the FPS.

- I am going to add a few audio/subtitle tracks, and also rename the current ones internally.

MAKEMKV log says nothing wrong: https://pastebin.com/hJu80ZBx (ignore the A/V sync issue, it says this for every disc it tries to rip).

*****
Test result: changing the names of the tracks using MKVToolnix didn't mess with anything:

https://pastebin.com/jpGsrKNj

******
Next verification: add new audio/subtitle tracks.

Then once this is also OK, I'll try splitting and appending.

*******
New test also passed, which was adding new audio and subtitle tracks: https://pastebin.com/ucYccVVe

Again no change. Same fps, same 3h51m36s.

FINAL TEST GOING NEXT: splitting into 15300M parts and then appending. If this final test fails, then it's MKVToolnix's fault.

*********

I did everything again and didn't notice anything wrong with the audio sync. Looks like something wrong in the first attempt, or the rip failed somehow. So no bug from MKVToolnix. No problems splitting and then appending all 6 parts.

Yet I noticed something interesting:



This is the file BEFORE I split anything:

https://pastebin.com/NMHFEsE4

Then I asked MKVToolnix to split into 15300M parts.

Part 1: https://pastebin.com/zM91VSkE
Part 2: https://pastebin.com/nzjt9LjE
Part 3: https://pastebin.com/iipEjchr
Part 4: https://pastebin.com/xkjeKAc6
Part 5: https://pastebin.com/fFWUKD2e
Part 6: https://pastebin.com/dx3Gy7Ri

As you can see all of them have the same information:

Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS

****
Now look what happens when files 1-2-3-4-5-6 are APPENDED:

https://pastebin.com/n4FfXuVG

Frame rate mode : Constant
Frame rate : 23.976 (23976/1000) FPS
Original frame rate : 23.976 (24000/1001) FPS

What does that mean?

Also, as the first picture shows, the file when I didn't split was shorter. It had:



Now look what MPC says about the appended file:



And this is also reflected in the 6 splitted files. Because if we add 36+38+38+40+41+36 minutes, plus 161 seconds, that will result in 229 minutes, plus another 2 (120 seconds) = 231, then 41s = 231 minutes and 41 seconds. Or 3h51m41s.

I tried freezing the same frame/scene in all three. These two are both the same:

1080p small file:
01:35:01.167

4K file that had never been splitted.
01:35:01.167

The appended file is different. That same frame/scene is located at:

4K appended
01:35:01.641

Still, the 4K file has no signs of sync issues. So, we can all conclude it is the same content, and the length is different because the fps is different.

But... why is MKVToolnix doing this?

P.S. Another movie affected by this bug:

4K appended:

Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 23.904 FPS
Original frame rate : 23.976 (24000/1001) FPS



1080p smaller file:

Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS



As a result of this bug it's impossible to put any audio/subtitle track from 1080p smaller file into the 4K.

The resulting file is the same (I mean the content), but it grew 23 seconds.

The bug oddly changed the framerate to 23.904.

I know what was said here:

https://forum.doom9.org/showthread.php?p=1932607#post1932607

As one user pointed out. Still, that doesn't justify MKVtoolnix messing with the files only because we splitted and appended. If this doesn't work then I suggest the option is removed.

I will try to investigate if I can a) if this bug can happen again if I repeat all steps, and b) if an older MKVToolnix version creates the same scenario.

If this is something only the newer ones are causing then I'll have my answer: splitting and appended was broken. Or it could be due to the tracks involved.

4K has the following:

https://pastebin.com/Kxs74AEY

I have an old suspicion splitting and appending always worked, but when we introduced certain new codecs some of them created all these issues. Remember when I blamed MakeMKV for a few failures?

https://forum.makemkv.com/forum/viewtopic.php?f=1&t=23937

Of course I need to repeat ALL steps, including a new rip from the UHD disc, because this 4K for example had been splitted in August 2020. Ever since this date MAKEMKV and MKVTOOLNIX received many updates.

All these 4K discs are causing me a lot of troubles.

*******
Update: the bug has been confirmed. At least for my Matrix Reloaded UHD disc.

Splitting and appending causes the fps to change. This time into this:

Frame rate : 23.906 FPS
Original frame rate : 23.976 (24000/1001) FPS

I put the disc again through MAKEMKV. Then did all steps, and saved as MKV. As long as we don't split and append, then everything remains the same.

2h18m17s.

If we do split and then append, the length changes to 2h18m40s. Still that doesn't mess with the video/audio/subtitle sync, since these tracks were already all in sync with each other prior to the splitting and appending.

But that also means we won't be able to insert a new audio/subtitle track if these have been created for the old 2h18m17s content.

So, if you all wanted a proof that splitting and appending can cause an issue, this is it. And this time MAKEMKV isn't involved.

I'll just do a final check, I'll look into an old MKVToolnix version to see if splitting and appending are OK in it. Or if the bug is still there.

*********

I'll try this one:
https://mkvtoolnix.en.uptodown.com/windows/download/1690774

19.0.0
Dec 19th, 2017

Result: same issue, same lenght. Nothing changed:

Taxa de quadros : 23.911 FPS
Taxa de quadros original : 23.976 (24000/1001) FPS

**********

After doing some research I just found out why the length increased after splitting and appending.

The answer is precisely here:
https://gitlab.com/mbunkus/mkvtoolnix/-/issues/2747

>>>>>> Appending Constant Framerate Videos results in a Variable Framerate Video with wrong duration

https://www.reddit.com/r/mkvtoolnix/comments/hjbb5d/fps_variable_with_mkvtoolnix/

Appending in my case changed the original file from CONSTANT to VARIABLE in the fps department.

That's why the length from the movie increased from 2h18m17s to 2h18m39s.

Do you have any idea when will this be fixed?

As you saw before, while this didn't mess with the file internally, it makes impossible for me, for example, to do this:


>>>>>>>>>>>>>>>>>>
- Extract any audio/subtitle track from a smaller 1080p rip...
and
- Insert (using MKVToolnix) into the big 4K file.
>>>>>>>>>>>>>>>>>>>>>

For example, say if I pause into scene A at 16 minutes and 1 second, in both files...

They will be frozen at the same spot, if I wanted to take a picture and compare both.

But if I do this at scene B located at 1 hour and 13 seconds then 4K's scene will have moved to 1h 1 minute, for example. More or less.

If this bug wasn't there, then both would be at 1 hour and 13s, not just earlier at 16m01s. This means as the movie progresses the 4K becomes out of sync with the smaller 1080p rip.

Meaning without the bug I can split and append as much as I want and there will be no change from CONSTANT to variable fps.

P.S. Contrary to what was said in that link, the bug was NOT fixed in 44.0.0, 1 year ago.

It's still there.

P.S. 2:

In that same link an user said:

>>>>>>> Did some further testing. After manually setting the Default duration to 24000/1001, MKVtoolnix finally muxed a file with the correct constant framerate and correct duration. It seems that the issue only occurs when you leave the Default duration field blank. <<<<<<<<<<



But why should I always have manually input this when splitting and appending? Couldn't you simply modify this program to always enforce the default fps without us having to type that?

I'll try splitting and appending again with this tweak, to see if it will work.

Remember I am talking about a MKV created by MAKEMKV from my UHD/4K disc.

So the resulting file is this:

Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS

Trying now (spliting and appending) with this setting:



I'll see if this works....

>>>>>>>>>>>

LOL... it didn't work!

The file now has a constant fps, I manually inserted this while splitting and appending, and still it shows here

2h18m39s.

PASTEBIN from the appended 4K: https://pastebin.com/DV9pCuQt

I give up trying to understand what is going on.

hubblec told me "the splitting/appending issue comes directly from Matroska and has nothing to do with makemkv or MKVToolNix(MTX). It is a Matroska design issue".

The problem is there, but MediaINFO reads as this:

Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS

How is that possible? MEDIAINFO is being tricked.

I even inserted 24000/1001p for the 2nd track, which is 1080p (Dolby Vision).

[Perenista]

I was looking for a proof that splitting and appending doesn't work and it's buggy.

It was important to me to split and append because I put my backups in 15 GB free Google Drive accounts. And I also use an app called nPlayer Plus to stream and/or download them. And sometimes while using my PC I downloaded them back and reconstructed (appending) with MKVToolnix.

I haven't been able to prove this doesn't work with ACTUAL evidence, so I suspected this was somehow MAKEMKV's or MKVToolnix's fault, or due to some codec used in the first attempt to rip from the disc.

I thought splitting and appending worked just fine, so I imagined the disc or first rip were the ones flawed.

Now I have definitive proof that if I use splitting and later try to reconstruct the movie (appending) this will damage the content. Meaning splitting and appending should NEVER be used.

The fact other movies were able to work with this idea, under other circumstances or even using other codecs doesn't matter.

What it matters is that if you do this you are now aware in advance it will break things:

https://www.youtube.com/watch?v=90jcjpomH1s

This video shows a lossless rip from my UHD/4K disc.

Now here's the thing: the original 4K rip was edited by MKVToolnix, so I could add new audio/subtitle tracks to it. And saved again. No problems.

This file ended up having 78 GB. This is its MEDIAINFO: https://pastebin.com/G4LBiViV

If I edit this thing a million times it will still be 100% OK. From the 1st second to the last. ALL SCENES. Notice in the Youtube video this 78 GB file is displayed in the first 45 seconds. You can clearly see a specific scene plays just fine.

Yet...

I wanted to split to put into 6 Google Drive accounts. So 14.97 GB in each one.



This is part 1: https://pastebin.com/YZs9A4YT

If I do this then the first file will have 22m20s, the second 22m04s, etc.

Now go back to the Youtube video. Pause at 1 minute and 6 seconds.

You are now seeing that if we try to play part 2 of 6, then right at the first seconds this is going to be seen:



Some sort of macro blocking. The entire image (for a couple of seconds) stays that way (no freezing, just the macro blocking).

A reconstruction error.

Now, when I saw this I didn't worry about it. Why? Because I thought that splitting caused this, however when I appended all 6 parts everything would be fine.

That macro blocking at the first seconds was only because I had splitted, so this was a way of the file to indicate that suffered an abrupt split into a sensitive moment. Some sort of anomaly we had to live with, an inconvenience, yet nothing to worry about.

*****
2 minutes into that Youtube video I play for the first time the APPENDED file. This one is a result of all 6 parts that I had splitted... combined. PASTEBIN: https://pastebin.com/qSvB3CHk

Remember what I said earlier:

- ORIGINAL 4K RIP = 100% fine
- SPLITTED PARTS = appeared fine, except for this macro blocking only seen at the 1st seconds from a few of them.

Before I reach a conclusion, please note that for the first time ever we are seeing MEDIAINFO reporting a different fps.

Frame rate : 23.940 FPS
Original frame rate : 23.976 (24000/1001) FPS

And for the Dolby Vision 1080p track:

Frame rate mode : Variable
Frame rate : 23.939 FPS

Since MEDIAINFO is not accurate when dealing with Matroskas, I am not going to bother with that. So nevermind the fps and the fact this one also grew from 1h58m24s to 1h58m32s.

Because my last 4K video also increased in length but that was all that happened (and the fact if I extract anything from a shorter 1080p source, it wouldn't sync with the 4K one appended with the fuzzy fps - I mean, I can't insert tracks from the 1080p into 4K due to this bug).

What I really wanted to do was this:

- Check the APPENDED file if at the same scene, at the exact moment, we would see macroblocking. If not then that could mean everything else was OK.

If macroblocking was not there again, then that would have proved my previous point:

- SPLITTED PARTS = appeared fine, except for this macro blocking only seen at the 1st seconds from a few of them.

APPENDED = was supposed to be fine, despite the macro blocking.

Now skip in the Youtube video to 2 minutes and 58 seconds.

You'll see the APPENDED file has been DAMAGED. Not only the scene freezes, there is also macro blocking AGAIN. And this happens with any player I use with the resulting 78 GB video.

There, I just produced proof this feature should be avoided. If this is due to Matroska's nature or something else I don't know.

What all this means to me is a disappointment. Now the only splitting I'll be able to do is putting all my contents into RAR parts. I never liked this idea, yet it's the only way.

It's really unfortunate this can't work.

[ProWo]

Cutting and merging HD videos (h264,hevc,av1, etc) is a problem and there are few programs that can do it. One of them is VideoRedo. Whether it can also handle your UHD video, you have to test yourself.