Right way to join MPG files?

[yetanotherlogin]

Hello,

As explained here, I had an issue editing an MPG file that was actually several MPG files joined into one, due to the 2GB file limit imposed by FAT32 in my camcorder.

This is the first time I was struggling, but in case the issue comes up again, I wanted to ask: Is there a more reliable way to join MPG files than either of the following?

copy /b file1.mpg+file2.mpg merged.mpg
ffmpeg -i "concat:file1.mpg|file2.mpg" -c copy -target pal-dvd merged.mpg

Thank you.

FWIW, here's what MediaInfo reports about one of the source files:

General
Complete name                            : C:\file1.mpg
Format                                   : MPEG-PS
File size                                : 1.98 GiB
Duration                                 : 29 min 57 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 9 446 kb/s

Video
ID                                       : 224 (0xE0)
Format                                   : MPEG Video
Format version                           : Version 2
Format profile                           : Main@Main
Format settings                          : CustomMatrix / BVOP
Format settings, BVOP                    : Yes
Format settings, Matrix                  : Custom
Format settings, GOP                     : M=3, N=12
Format settings, picture structure       : Frame
Duration                                 : 29 min 57 s
Bit rate mode                            : Variable
Bit rate                                 : 8 810 kb/s
Maximum bit rate                         : 9 100 kb/s
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan order                               : Top Field First
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.850
Time code of first frame                 : 00:00:00:00
Time code source                         : Group of pictures header
GOP, Open/Closed                         : Closed
Stream size                              : 1.84 GiB (93%)

Audio
ID                                       : 189 (0xBD)-128 (0x80)
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Muxing mode                              : DVD-Video
Duration                                 : 29 min 56 s
Bit rate mode                            : Constant
Bit rate                                 : 448 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Compression mode                         : Lossy
Stream size                              : 96.0 MiB (5%)
Service kind                             : Complete Main

[jagabo]

You should almost never use COPY/B or "ffmpeg concat" like that to append A/V files. About the only time that works is when a large file was blindly split into smaller segments. In cases like that the individual segments usually won't play because parts of the A/V container structure are in different segments.

You should usually use something like:

ffmpeg -y -safe 0 -f concat -i list.txt -c copy output.mp4

where list.txt is a text file containing the list of files to concatenate:

file 'file1.mpg'  
file 'file2.mpg'

This way ffmpeg parses the structure of the individual source files and remuxes the data into a new container with a new structure.

[yetanotherlogin]

Thanks.

I did try the first method before trying the second one, but for some reason, the file was crap (1,3GB instead of ~7GB, and after about 30mn, the audio turned into garbage.)

I just gave method#1 another try, and, like before, 1) a lof ot scary error messages scrolled by and 2) the file was again about 1,3GB.

[pandy]

Check (ffprobe?) last GOP of first file and first GOP second file.You may try to extract (demux) elementary streams and combine them and later mux into new container...

[yetanotherlogin]

I have no idea how to do this

[netmask56]

If both files have the same specifications you could append one the other using MKVToolNix - the output would be a MKV file with no re-encoding or loss of quality.

[yetanotherlogin]

C:\>mkvmerge -o merged.mkv M2U00427.MPG M2U00428.MPG M2U00429.MPG

… although I learned in the other thread that an NLE doesn't necessarily need a single file, and the user could simply drag and drop the different parts in the video track — something that never crossed my mind —… mkvmerge is a useful option in case I need to send someone a full file.

In case I need to open an MKV file in Vegas Movie Studio, I'll just convert it to MPEG2 using eg. EaseFab Video Converter.

Thank you very much.

[TimA-C]

VideoRedo has a Joiner that I sometimes use, but it's not free. You can append an additional file to the current location in Virtualdub/2 but not sure how well it works (if at all) with MPeg2 files? I think you can do something similar with AviDemux as well - again, not sure how well it works (if at all).

[jagabo]

If ffmpeg is having problems with your files a NLE may too.

[yetanotherlogin]

I'll report back the next time I need to edit a video recorded with my camcorder.

[TimA-C]

Just a thought, does your camcorder actually require the memory card to be formatted as FAT32? Might it accept one formatted as NTFS or exFAT? (I'm assuming that it's recording to memory card rather than internal hdd?)

[yetanotherlogin]

I haven't checked since the Sony HDR-PJ30 camcorder can record several hours on its 32GB internal memory.

Apparently, that camcorder can use up to "SDXC UHS-I Memory Card Class 10 64 GB SF-64UX" card.
https://www.sony-mea.com/support/resources/en_AP/html/Compatibility/SDCard/handycam.htm

Could it then be formated in either NTFS or ex-FAT?

--
Edit: I had a 32GB SDCard lying around, so gave it a try. The camcorder also formats it in FAT32.

[smrpix]

What camera are you using? The "right" way to join the files may be by using the folder structure and intended software provided by the manufacturer. It appears you are talking about spanned clips?

[TimA-C]

FAT32 is probably the most compatible format but it might still be worth trying to format the card in your PC/laptop as NTFS or exFAT and then see if your camcorder can see and use it.

[Cornucopia]

Except best practices is to always let the camera do the formatting.

If the cam is the thing that broke these up, they should all be self-contained pieces that should be immediately appendable in most NLEs, incl. Avidemux.

However, if it uses a cliplist/playlist like AVCHD does, you shouldn't have to do anything like that. All you do is load the folder (structure) into the NLE (or in a few editors, the playlist) and it will automatically append properly. In fact, this is the recommended method for AVCHD, so maybe it is for your cam as well.

Scott

[yetanotherlogin]

It supports both AVCHD and MPEG-2, and am using the latter to record longer.

https://www.cnet.com/g00/products/sony-hdr-pj/

[smrpix]

What Scott said. Plus you should be able to transfer spanned clips from your camcorder as single clips by using the USB out on your camera.

[_Al_]

It supports both AVCHD and MPEG-2, and am using the latter to record longer.

But look where you are at right now,

use avchd , highest bitrate, frame rate (50fps if in Europe). Is there extra SD card slot? Look in manual how to import it to PC, there might be metadata with it, in other files. Then loading videos into Vegas should be not a problem.

[abolibibelot]

You should almost never use COPY/B or "ffmpeg concat" like that to append A/V files. About the only time that works is when a large file was blindly split into smaller segments. In cases like that the individual segments usually won't play because parts of the A/V container structure are in different segments.

Some types of video formats like MPG / VOB / MTS (and also audio formats like MP3 which is also based on the MPEG specifications) have a sort of “modular” data structure, with a new header for each segment of a certain size (which size depending on the specific format and perhaps the specific implementation – my knowledge on those aspects is mostly empirical), so for those formats even a partial file, missing the beginning which usually contains the general and unique header for most file types, can be played so long as the software or hardware player manages to find one valid header. (For instance I have a MPG file which is being downloaded in non-sequential chunks, and a large segment is missing at the beginning – which will probably never be completed as it has had no source in years –, the segment that has been downloaded can be read with VLC Media Player – but not with MPC-HC, or surprisingly SMPlayer which is usually about as good as VLC when it comes to parsing partial or corrupted files.)

You should usually use something like:
ffmpeg -y -safe 0 -f concat -i list.txt -c copy output.mp4
where list.txt is a text file containing the list of files to concatenate:
file 'file1.mpg'
file 'file2.mpg'
This way ffmpeg parses the structure of the individual source files and remuxes the data into a new container with a new structure.

Isn't this strictly equivalent as using the “pipe” symbol, without using an extra text file ?

What can be done with ffmpeg is first to convert the files to TS format (Transport Stream – as opposed to PS = Program Stream – which is even more “modular” than MPEG and MPEG-like formats – and more versatile as it can contain other types of streams beyond the standard MPEG compatible ones – systematically arranging, as far as I understand, the video and audio streams in small self-contained “packets” which can be read independently, so that even in case of missing packets caused by a failed transmission the playback can continue with as few glitches as possible), and then concatenate the resulting TS file while converting them back to the container of choice, here MPG, with no transcoding involved.

ffmpeg -i "file1.mpg" -c copy "temp1.ts"
ffmpeg -i "file2.mpg" -c copy "temp2.ts"
ffmpeg -i "concat:temp1.ts|temp2.ts" -c copy "merged.mpg"

That works with files having a non-modular structure, like MP4, and it prevents synchronization issues which are frequent with other methods.

Check (ffprobe?) last GOP of first file and first GOP second file.You may try to extract (demux) elementary streams and combine them and later mux into new container...

I have no idea how to do this

With ProjectX, or with MPlayer :
mplayer "input.mpg" -dumpvideo -dumpfile "input_vid.m2v"
mpleyer "input.mpg" -dumpaudio -dumpfile "input_vid.ac3"
Or with DGIndex (included in MeGUI), which is much more simple to use than ProjectX, and probably more reliable for MPG demuxing than MPlayer (MPlayer usually extracts raw streams which might be missing a valid header to be properly recognized by most players / editors).

It supports both AVCHD and MPEG-2, and am using the latter to record longer.

How much longer ? With MPEG-2 you get a measly (by nowaday's standards) resolution of 720x576, while AVCHD provides a vastly superior resolution at 1980x1080, and considering that AVC is much more efficient (in terms of bitrate to perceived quality ratio), the space saving must be largely offset by the loss in quality.

[jagabo]

You should usually use something like:
ffmpeg -y -safe 0 -f concat -i list.txt -c copy output.mp4
where list.txt is a text file containing the list of files to concatenate:
file 'file1.mpg'
file 'file2.mpg'
This way ffmpeg parses the structure of the individual source files and remuxes the data into a new container with a new structure.

Isn't this strictly equivalent as using the “pipe” symbol, without using an extra text file ?

Not it is not. Piping (concat protocol) is a blind concatenation of the input files before demuxing. Using the "concat demuxer" parses the contents of each input before concatenating the streams.

https://trac.ffmpeg.org/wiki/Concatenate