TS to MPG/MP4/MKV and back to TS 1:1 -- possible ?

[abolibibelot]

I have quite a few TV recordings originally made in MPEG2-TS or MPEG4-TS format, which I then converted (losslessly) to either MPEG2-PS (for MPEG2-TS recordings), or MP4, or MKV (for MPEG4-TS recordings), to get files which are smaller by about 10-15% and allow a quicker random access. If the conversion was successful, the original TS recording should no longer be necessary, but so far I've kept both versions, just in case. The question may be stupid but here it is : would it be possible to recreate a TS file 100% identical to the original file, from the converted file ? Meaning : converting TS to (MPG/MP4/MKV) then back to TS in such a way that the newly generated TS is 1:1 identical to the original one. That would allow to delete those redundant TS files, knowing that I can regenerate them if push comes to shove.
But from what I could see it doesn't seem possible, as TS files generated by different programs appear to be significantly different, and for instance whether ffmpeg or TSMuxer is used, the newly generated TS file is markedly different from the original (which was either created by a standalone “set top box” recorder or downloaded) ; and it's not just the header that is different and then a large chunk is identical in both files (as it can happen with MP4 / MKV files generated by different utilities), the whole structure is different, apparently there is no standard structure for TS files.
Still, a large proportion of the data should be identical between the original TS and its converted counterpart ; so, if it's not possible to achieve through a direct video conversion, would there be a way to sort of “compress” the TS file, based on similar data chunks that are present in the converted file, keeping only what's different/unique, and allowing to regenerate it by combining the converted file and the “compressed” file ? (Which should be only a fraction of the size of the original TS file.)

Hope it's not too convoluted...

[zing269]

xdelta will do that.

[ProWo]

TS is just a container, so there is no reason why a remux from mpg, mp4, mkv to TS should not be possible, provided that video codec and audio codex are compatible with the TS container.
You can do this with clever Ffmpeg-GUI.

[abolibibelot]

TS is just a container, so there is no reason why a remux from mpg, mp4, mkv to TS should not be possible, provided that video codec and audio codex are compatible with the TS container. You can do this with clever Ffmpeg-GUI.

Of course, but what I meant was : how to re-create a 100% identical TS file (byte for byte) as the original recorded file, from the remuxed file ? (I've made a few tests with ffmpeg, TS to MP4, then MP4 to TS, in direct copy mode, the output was markedly different, even the size was significantly different, even though the video / audio content was presumably identical.)
Apparently xdelta mentioned above would seem to work for such purpose ; coincidentally, I've seen it mentioned first very recently : it was included in a patch meant to fix a mistake in a MKV file made available for download, and I've also seen, just a few days ago on a completely unrelated website, a similar patch meant to change the default audio track for a whole folder of MKV files in one click. Apparently this tool has become a staple in the “fansubbing” community.

[ProWo]

Of course, but what I meant was : how to re-create a 100% identical TS file (byte for byte) as the original recorded.

For what purpose?

[abolibibelot]

For what purpose?

Well, for the sake of OCD-ness I suppose...

[ProWo]

how to re-create a 100% identical TS file (byte for byte) as the original recorded.

copy file.ts newfile.ts

[netmask56]

Put file.ts into a mkv container using mkvtoolnix. If you want to go back to file.ts then extract the video and audio and remux in TSMuxer. But why on earth would one want to do this anyway?

[Cornucopia]

You are also mistaken - there IS a standard for TS. It is in fact, THE MPEG2 TS standard.
However, there exist in the spec a bunch of provisions for metadata & user- or application-specific custom ("private") data, and part of the multiplexing has to do with proper timing (and timestamping) of these various elements, and it is VERY likely that many of the apps that support muxing (and demuxing) of TS streams do not necessarily support ALL of its features. Therefore,things could easily and reasonably get dropped or rearranged along the way, even for legit, compliant apps (much less the reverse engineered and only quasi-compliant ones you are familiar with).

Scott

[abolibibelot]

@Cornucopia

You are also mistaken - there IS a standard for TS. It is in fact, THE MPEG2 TS standard.
However, there exist in the spec a bunch of provisions for metadata & user- or application-specific custom ("private") data,and part of the multiplexing has to do with proper timing (and timestamping) of these various elements, and it is VERY likely that many of the apps that support muxing (and demuxing) of TS streams do not necessarily support ALL of its features. Therefore,things could easily and reasonably get dropped or rearranged along the way, even for legit, compliant apps

That is probably what I meant, sorry for the incorrect use of terminology (although I wrote “no standard structure”, not “no standard” altogether) — obviously there is a standard, otherwise it would quickly fall apart, but there are apparently many ways to create files compliant with this standard, and you mentioned specific reasons I wasn't aware of.

(much less the reverse engineered and only quasi-compliant ones you are familiar with).

Interesting ; so that covers all utilities I mentioned, and pretty much all utilities available outside of high level professional applications ? Where would standalone recorders (like the set top box that was used to record most of those TS files) fall in that regard ?


@netmask56

Put file.ts into a mkv container using mkvtoolnix. If you want to go back to file.ts then extract the video and audio and remux in TSMuxer. But why on earth would one want to do this anyway?

I mentioned the very reasons why those files were converted to MPEG-PS, or MP4, or MKV : smoother playback (instant random access in particular, with TS it freezes for a short while and the picture looks garbled before it plays normally again) and file size significantly reduced because these containers have less “overhead”. But obviously it defeats that purpose if I keep both the original file and the converted file. As to the why-on-Earth question : at first (that was long ago) I converted those files with whatever method seemed to work (IIRC at first I used the clunky conversion module included with VLC Media Player), then deleted the original ; but later I found out that some of these converted files had severe synchronization issues, caused probably by missing or corrupted TS chunks (or possibly redundant, as I experienced more recently and exposed in this stillborn thread), and at that point it was too late to go back to the original file and try another conversion method. Then I tried several conversion methods to deal with such issues, found some that appeared to work well (either for MPEG2-TS or for MPEG4-TS) even for files with similar issues, but, lesson learned, from then on I always kept the original TS files. Besides, I haven't watched all those recordings from beginning to end (far from it actually — and may never get to watch them all unless someone designs a real life equivalent of the hyperbolic time chamber in Dragon Ball Z), so I can't be sure that even with these improved methods there aren't unforeseen glitches, which may not be correctable at all, in which case the original TS file would be the only one worth watching (and possibly editing — for instance, I've sometimes inserted short excerpts of such personal TV recordings made 5 years ealier in a video of an associative debate, to illustrate a point of a speaker, and sometimes, for some reason, the converted file can't be loaded into the editor, or can be loaded but appears all screwed up, it's even more unpredictable than playback, in which case going back to the original TS file can solve the problem).


@ProWo

copy file.ts newfile.ts

Well, thank you for your participation, but that's totally irrelevant... é_è
(If that was a serious reply and not stealth trolling : the idea in this case is to delete the TS file, yet preserve the possibility of re-creating it from its remuxed counterpart. I have yet to do specific tests with xdelta but it would seem like the most efficient solution, if it effectively detects redundant chunks of data between TS files and their remuxed counterparts despite their very different structure.)