Exploring & understanding the mysteries of .M2T files

[Canon GL-2 Guy]

This format is more or less new to me so here goes:

1) What is a .M2T file comprised off? How does it differ from an MPEG-2 dvd compliant file for example?
2) What's the easy and most straight forward app to trim off & edit clips which HDV split created after importing them from tape?
3) Does it change the compression quality of the file to rename them with an .mpg or .mpeg extension?
4) If one of these clips is put on a timeline and edited with say Adobe Premiere Pro, what do I do to export it at the highest possible quality?

What would you do in my case?

Anything else I should know or ask to help get me started on editing & exporting .M2T files please let me know.

Thanks

[edDV]

1. m2t is an MPeg2 transport stream
http://en.wikipedia.org/wiki/MPEG_transport_stream
http://en.wikipedia.org/wiki/HDV

Think of it as a packetized (ruggedized + error correction) construction for transport in difficult environments. A TS stream or file can be unpacketized to normal MPeg2.

2. HDTVtoMPeg2 will cut on I frames and convert to normal MPeg. It can demux a multichannel ATSC or DVB TV stream or work with HDV.

3. Renaming the file extension does nothing to the data. Most programs will examine the header to determine video format.

4. You'll need to experiment with your particular version of Premiere Pro (or other editor) for acceptaance of unconverted transport streams. Later versions have specific project template support for HDV/XDCAM (MPeg2) and AVCHD (h.264) formats.

[Canon GL-2 Guy]

edDV: A real pleasure to hear from you!

A couple of more questions if may:

1) My main goal is to preserve the original HDV quality all the way to the end of the editing & delivery process which will be a dvd. If I use for example your reccomendation of HDTV to MPeg2 and convert to normal MPEG is there anything lost in the resulting file?

2) What Videohelp apps do you reccomend I use to edit the MPEG2 files?
a) To both trim and finalize clips.
b) Then join them together.
c) Export the merged clips in order as a single MPEG to preserve compression & quality.

Looking to hopefully get it right the first time.

Thanks so much for ocntinuing the dialogue.

[aedipuss]

just leave them as HDV. it's 1440x1080 and conversion to any other format will cost you some quality. all recent pro pc editors can use the HDV format and no conversion is necessary until you render to your final format. you can even print HDV back to tape if you need to.

capture with hdvsplit. import files to editor. edit. export final format.

[Canon GL-2 Guy]

Ed DV: Are you still out there?

Aedipuss: Unfortunately the version I have of Premiere (3.2) has limited export options.

I wish I had CS4 which would make it easier! That would cost me another $300 to upgrade.

[edDV]

Ed DV: Are you still out there?

Aedipuss: Unfortunately the version I have of Premiere (3.2) has limited export options.

I wish I had CS4 which would make it easier! That would cost me another $300 to upgrade.

Canon GL-2 Guy,

We are out on the deck waiting for the fireworks show to begin. I'll be back when it is finished.

What version of Premiere was that? Does it not support HDV?

[Canon GL-2 Guy]

Sorry EdDV,

I hope you had a nice 4th and a dazzling fireworks display!

I only have version 3.2 to work with which has considerably less options (than CS 4 for example) in regards to exporting and specific project presets.

The clip trimming process has already begun, the challenge now is how best to proceed after that process is completed.

Right now on the timeline in Premiere, I have a bunch of .M2T clips which I had imported in via HDV Split.

I want to preserve the original quality to the fullest extent possible and ultimately successfully deliver this to dvd.

My quandry is this, since .M2T files are so highly compressed, if I convert or export them to work with them on another program quality will take a hit!

My export options are these:

1) Tape
2) Audio track only
3) Adobe Media Encoder (which of course futher compresses the material and shrinks/downgrades it).
4) Single Frame
5) Encore (which I do not want to use to make a dvd with as there's better choices available.


What do you think I should do to get nices results and at least a semi professional looking result?

Thank you!

[aedipuss]

pp cs3 3.2 works with hdv fine.

The Adobe® Premiere® Pro CS3 3.2 update adds native support for the Sony XDCAM range of cameras, including disc-based XDCAM and XDCAM HD cameras, and the newer SxS-based XDCAM EX cameras. This release also includes improved handling of MPEG streams (for example, HDV video), along with error detection and other important bug fixes. Recommended for all Adobe Premiere Pro CS3 users.

do your editing and then export to dvd spec 720x480 mpeg-2 and 2ch ac-3 audio. author the dvd and away you go. if you need to use other external programs to edit with and you are going to only make dvds then you might as well transfer the HDV tapes from the cam as DVavi instead of HDV. the camera will do the 1440x1080 to 720x480 conversion with it's hardware for you.

[Canon GL-2 Guy]

Aedipuss:

Thanks for the affirmation/confirmation that at least it can be done.

There seems a wide divergence of opinion from what I've read online as to the best way to downconvert on the A1 from HD to SD which is confusing & I do not know who to believe. Some swear that by doing it internally within the A1 there's nothing better. Others say it yields a fair but not a good result & one is better to shoot in SD in the first place & not convert. Another camp says to use Jeff Bellunes/Issacs et.al famous video HD 2 SD tutorial (problem is that route needs CS4 which I don't have) to get an excellent conversion.

What do you think? Ever done it youself with an XH- A1 or another method?

Please let me know.

Thank you.

[aedipuss]

i mostly work with canon hv20/30/40's and gl1/2's. i've done it both in software, mostly vegas pro, but also cs2/3/4, and using the hardware to downcovert from the camera.

currently my method is to use the hv's to shoot 30p for use both as online video source and dvd creation. i video mostly concerts, musicals, recitals, plays, and other low movement performances. i transfer as HDV and use vegas's mainconcept mpeg2 encoder to convert to dvd spec. pp cs has the same encoder.

the only time i vary from that is if i do a multiple cam shoot with hv's and gl's. then i shoot HDV 30p on the hv's but for editing purposes use the camera to also export it as DVavi for the multicam editing.

i really don't notice much of a difference in hardware/software downconversion. they both have looked excellent from any HDV cam i've used. make sure the encoder settings are always at their highest quality settings and a bitrate to fill a dvd and it should look great either way.

[Canon GL-2 Guy]

Aedipuss:

Since you have used and /or own CS4, do you think it's worth $300 for me to upgrade so I can take advantage of it's new functions, expanded settings and options? Has the encoder engine/compression quality gotten any better in that version or is it about the same as in version 3? I also have CS2 and real old 6.5.

I too personally own 2 GL-2's and work with others at church who use GL-1's so we are both on familiar ground there.

Once tried a HD40 however the controls are so close together I kept hitting a wrong control in error while shooting!

I'm seriously considering buying a used A1 in the coming months.

Thanks for continuing the dialogue.

[aedipuss]

no i thought cs4 should have just been a cs3 update. not much changed and the encoder is the same in both. cs5 actually adds value with the gpu encoding, but needs heavy duty hardware to take advantage of. i prefer vegas pro for my day to day work. the only part of pp i like more than vegas is it's still picture to video features/controls.

i'd love an a1 to add to the arsenal. the hv has it's uses, it's extremely portable, inconspicuous. works well on a velbon f.h. portable tripod and i have 5 hour batteries for them.

[edDV]

I was bogged down most of the weekend with computer maintenance issues. I'll answer your questions in pieces during the day as time allows.

First, is you computer still a Pentium 4 2.66 GHz as listed in your profile?

Did you mention the camcorder model?

What are you trying to export to? Blu-Ray, DVD, Youtube? Other?

For archive, keep the HDV M2t capture files and export your project back to HDV format. You can even export to HDV camcorder tape over IEEE-1394 for backup (called "print to tape".

...
I only have version 3.2 to work with which has considerably less options (than CS 4 for example) in regards to exporting and specific project presets.

Does your Premiere Pro 3.2 have an HDV 1440x1080i project template? If not we will have to make one.

HDV editing on a P4 will be a struggle but doable.


PS: I read above you are using a Canon XH-A1.

[edDV]

You need to design a workflow for your general HDV editing. Premiere Pro more or less defaults to a DV workflow to DVD.

First set project to HDV under File New.

Import your m2t HDV files into the bin.

If set properly, HDV files will not require workspace render on the timeline. They will preview directly*.

For "best quality" edit archive, export back to HDV format (m2t file and/or tape). As mentioned above TS files are more robust for storage than simple MPg2.

From there you will proceed to downsize and/or format encode depending on your target audience plans. You now have HDV source archives and an HD edit master.


* timeline scan/scrub preview will be extemely laggy if your CPU is a P4. This is because MPeg must be decoded to frames for preview by the CPU as you scrub the timeline.

[Canon GL-2 Guy]

EdDV, here are my current Dell PC system & Premiere project specs:

Windows Vista Home Premium 32 bit
Q6600 2.4 Ghz
Intel core 2 Quad CPU
3 Gigs ram
1 Terabyte hard drive

HDV1080i
29.97 fps
1140 by 1080 (16:9)
HD Anamorphic (1.333)
Upper field
30 fps Drop-Frane Timecode
100% .M2T clips on the master project timeline which are now trimmed and edited, howver they are spread out between 2 sequences.
The vidoe card came with the computer and is nothing fancy but works.

The computer is somewhat over 2 years old.

[Canon GL-2 Guy]

I don't remember or know, so I'll ask, have you ever used Premiere Pro?..... Curious!

The end/target result may well be a MPEG-2 dvd compliant file because at this stage I don't know how to produce anything else. Certainly open to suggestions & alternatives, whatever's best. Yes, I certainly plain to save the finished project direct from the timeline to tape. Since the files on the timeline are still .M2T (even though they are edited), it will still be HDV video once back on tape right? This is all new to me. I do not know if Prem 3.2 had the capability to export the finished project to an .M2T file, wish I could. My choices are Tape, Adobe Encore, Audio, Adobe Media Encoder, Frame, EDL and "Movie".

The last category has some possibilities.

To:
a) Microsoft .avi (however Adobe is defining that)
b) Microsoft DV .avi
c) Uncompressed Microsoft .avi

Option a) has interesting choices for compression:
1) No compression
2) Cinepak codec by Radius (old)
3) UYUY 422- 8-bit
4)Indeo video 5.10
5) Intel ItuV Codec
6) Microsoft video one
7) XVID MPEG-4 Codec
8) Lagarith Lossless Codec

What would you use if anything in regards to the above?

Also, you can choice for keyframe & rendering: no fields ("progressive scan"),upper field first or lower field first.
There is also a choice/ check box to deinterlace the video.

[Canon GL-2 Guy]

I'd really like to learn how to:

export to a lossless codec, come up with very high quality.avi suitable for compression and

as you suggested to an .M2T/.TS file.

Looking forward to continuing this quest!

Thank you.

[Canon GL-2 Guy]

Gents!

Please feel free to join in to this discussion.

Some of you must have tread this ground already.

[edDV]

EdDV, here are my current Dell PC system & Premiere project specs:

Windows Vista Home Premium 32 bit
Q6600 2.4 Ghz
Intel core 2 Quad CPU
3 Gigs ram
1 Terabyte hard drive

HDV1080i
29.97 fps
1140 by 1080 (16:9)
HD Anamorphic (1.333)
Upper field
30 fps Drop-Frane Timecode
100% .M2T clips on the master project timeline which are now trimmed and edited, however they are spread out between 2 sequences.
The vidoe card came with the computer and is nothing fancy but works.

The computer is somewhat over 2 years old.

Please update your profile. It affects how your questions are answered.

HDV is typically 1440x1080i/29.97 or 25 fps Mpeg2. Camera variations support 29.97P and 23.976PF or 23.976P

The standard also supports 1280x720p at 29.97P with camera variations for 59.94P, 23.976 PF and 23.976P.

I don't remember or know, so I'll ask, have you ever used Premiere Pro?..... Curious!

The end/target result may well be a MPEG-2 dvd compliant file because at this stage I don't know how to produce anything else. Certainly open to suggestions & alternatives, whatever best. Yes, I certainly plain to save the finished project direct from the timeline to tape. Since the files on the timeline are still .M2T (even though they are edited), it will still be HDV video once back on tape right? This is all new to me. I do not know if Prem 3.2 had the capability to export the finished project to an .M2T file, wish I could. My choices are Tape, Adobe Encore, Audio, Adobe Media Encoder, Frame, EDL and "Movie".

I've been a user of Premiere and Premiere Pro since the early days. I currently own a PC with CS4 but it is out of town on a project for several months. I use Sony Vegas Pro 9 and Final Cut on the machines here. The following discussion applies equally to Vegas but not to Final Cut.

You seem to be having conceptual problems with DV vs. HDV formats. DV is a frame based format with intraframe compression only. HDV is MPeg2 with 15 frame GOPs (12 for PAL) and uses both intraframe and interframe compression. "Lossless" in the HDV context means avoidance of re-encoding except when necessary. If you set Premiere to an HDV project and cut only on I frames (first frame of the GOP) and then export in HDV format, your output GOPs will be identical to the camera original with no re-encode losses.

If you cut inside the 15 frame GOP, Premiere will decode the affected GOP to frames, make the cut and then re-encode that GOP only. If you process the image in any way (e.g. transitions, filters, resize, etc.) the affected GOPs will be decoded to YCbCr frames and then the YCbCr frames will be converted to RGB for processing. The RGB frames are stored on your scratch disk as temp files. When you export your finished project, the RGB frames are encoded into your export format. So if you want "lossless" for unprocessed GOPs, you would export back to HDV format.

Unlike Vegas Pro, Premiere Pro CS3 only allows HDV export in "print to tape" mode according to this Adobe document.
http://livedocs.adobe.com/en_US/PremierePro/3.0/help.html?content=WSCE52A514-41FD-4585...41ED9C380.html

Vegas allows export to an HDV m2t file or HDV tape. I'm not sure if Premiere CS4 or CS5 has added that feature.

Other options for high definition project archive export are

1. Uncompressed YCbCr (very large files)
2. Digital intermediate format such as Cineform (see discussion below)
3. HuffYUV or Lagarith codecs (must be installed as Video for Windows codecs)
4. Blu-Ray compatible 25Mb/s MPeg2 (minor recode losses)
5. Blu-Ray compatible h.264 or AVCHD (not good for future re-editing)
6. Other codecs for specific requirements.

If you want a standard definition (720x480i) DVD, you can resize in Premiere Pro and MPeg2 encode. Or for higher quality, export as uncompressed HD and work on it with AVISynth filters.

The codecs you listed, other than DV-AVI, Lagarith and XVID are for legacy or special purpose.

[edDV]

The discussion above relates to so called "native" HDV editing. The alternative is to use a digital intermediate format.

Digital intermediate formats decode MPeg into frames but retain intraframe compression to keep bit rate to that a single hard disk can handle. Full decompression would require a multi-disk RAID to play or search the video. Digital intermediates speed HD timeline responsiveness making it almost a fast as DV format.

A popular digital intermediate codec for Premiere Pro is Cineform Neoscene. Others are available from AVID, Grass Valley and Apple. Apple requires use of a digital intermediate (AIC or ProRes422) when editing HDV or AVCHD formats. Native MPeg editing is not supported by Apple.

Cineform has additional features besides speeding up the timeline response. It has a capture program that imports and converts HDV to Cineform. The capture program inverse telecines 24p source. The intraframe wavelet compression has low recode loss and has advantage for resize interpolation.

When a digital intermediate is used, it is common practice to archive the edit project in the digital intermediate format. The files are large (~60GB/hr.) but quality is high and re-edit is low loss. HDV source files can be archived as camera original HDV.

[Canon GL-2 Guy]

Ed,

I apologize for not putting you in a better position to respond earlier on in the conversation.

Not being a expert, techie or possessing a wide knowledge of or a thorough understanding of video editing in regards to codecs, thoery, relationships & structure the amount of detail was initially on the overwhelming side. I'll do my best to sort through it bit by bit.

Thanks for taking so much time to put that down & really appreciate this opportunity to digest this material, learn, grow & apply it not only to the tasks at hand but also going forward. Sorry, but I will have to ask by necessity (until this valuable instruction jells together & makes sense), what may seem to you silly or novice questions.

I guess perhaps the best way for now to relaunch this discussion & move forward since you are aware of my situation is this:

1) I don't mind having a HD format dvd. Actually I would like to try both, HD and SD. On CS3.2, Adobe Media Encoder will compress to MPEG-2. Will that dvd be SD or HD? Since the native timeline files are already .M2T, when compressed will that automatically create a HD dvd?

2) If you were me, would it be best to export the finished project from the master timeline to a Microsoft Avi file, Lagarith flavor? Would the quality be preserved? or:

3) What workflow do you think best fits my case?

More latter,

Please Take Care

[edDV]

1) I don't mind having a HD format dvd. Actually I would like to try both, HD and SD. On CS3.2, Adobe Media Encoder will compress to MPEG-2. Will that dvd be SD or HD? Since the native timeline files are already .M2T, when compressed will that automatically create a HD dvd?

2) If you were me, would it be best to export the finished project from the master timeline to a Microsoft Avi file, Lagarith flavor? Would the quality be preserved? or:

3) What workflow do you think best fits my case?

For a quick 720x480i/29.97 DVD you simply export to the DVD template in the Adobe MPeg encoder. The downsize won't be ideal quality but you can do it all inside Premiere CS3.

An alternative was suggested above. Export from the HDV camcorder in 720x480i/29.97 DV format. Capture with Premiere or WinDV. Set a DV format CS3 project and do an SD edit. Then export to the 720x480i/29.97 DVD template.

Next issue is how to save your HDV edit at highest HD quality.

Highest HDV quality will be HDV format. The scenes outside processed GOPs will still be first generation. CS3 only allows export in HDV format if you go back to the camcorder tape (print to tape). You can then recapture the tape to an m2t file.

Next best would be a Blu-Ray compatible Mpeg2 file. Tell me if CS3 includes Blu-Ray templates in the Abobe MPeg encoder. List the options.

[edDV]

I just remembered CS2 and CS3 Premiere include quick DVD modes with limited menus. They even burn to DVD disc.

[poisondeathray]

1) I don't mind having a HD format dvd. Actually I would like to try both, HD and SD. On CS3.2, Adobe Media Encoder will compress to MPEG-2. Will that dvd be SD or HD? Since the native timeline files are already .M2T, when compressed will that automatically create a HD dvd?

Encore doesn't support authoring HD content on DVD5/9 media. In order to play HD content on optical disc (regardless of its DVD 5/9 media or Blu-ray 25/50 media), you need a blu-ray player , PS3 or HTPC

You can export AVC or MPEG2 and use 3rd party tools to author instead of Encore , e.g., multiavchd (free), and this will work for DVD5/9 media, but you still need a device that's capable. Regular DVD players cannot play HD content; you are limited to regular DVD-video which is standard definition

[Canon GL-2 Guy]

1) I have a Panasonic DMP-BD35 blu ray player.

2) CS3.2 Adobe Media Encoder Blu-Ray options are:

a) MPEG-2 Blu-Ray
b) H.264 Blu-Ray

3) I tried out Cineform Neoscene which does a nice job. What with & how would one get a nice HD dvd from that file? What would you use to encode its' .avi file to yield the best results?

Thanks Poison Death Ray for your astute observations and comments.

4) I tried a quickie compress of a couple of .M2T files from the timeline to an MPEG-2 DVD compliant file just out of curiosity and my eyes were hoping to view a considerably better render. It was just ok. Definitely a substantial step down/hit from either the original footage and almost as much from a Cineform AVI which had exactly the same material.

5) I promise I'll send the finished product back to tape for storage.

6) A new wrinkle: before I can do step 5 as reccomended, I need to marry/join the contents of 2 sequences so that they are in sequential order on the main project timeline. One of those sequences (which consist of a series of clips) is located within a seperate tab. I want to lift the whole of the seq found in that tab and move it next to the other/main sequence which rests on the main timeline. Another obstacle to be figured out.

Thanks Gents

[edDV]

6) A new wrinkle: before I can do step 5 as reccomended, I need to marry/join the contents of 2 sequences so that they are in sequential order on the main project timeline. One of those sequences (which consist of a series of clips) is located within a seperate tab. I want to lift the whole of the seq found in that tab and move it next to the other/main sequence which rests on the main timeline. Another obstacle to be figured out.

You need to get all the clips to the timeline and preview until you are happy. When pleased, "print to tape" or export through Adobe MPeg Encoder will encode from the timeline.

4) I tried a quickie compress of a couple of .M2T files from the timeline to an MPEG-2 DVD compliant file just out of curiosity and my eyes were hoping to view a considerably better render. It was just ok. Definitely a substantial step down/hit from either the original footage and almost as much from a Cineform AVI which had exactly the same material.

DVD is 720x480i standard definition. If a standard def DVD is your goal, you have two alternate ways to get there. You can export HD to AVISynth and use advanced filters for downscale, or you can export DV format from the camcorder (downscale in hardware) and edit an SD project.

1) I have a Panasonic DMP-BD35 blu ray player.

2) CS3.2 Adobe Media Encoder Blu-Ray options are:

a) MPEG-2 Blu-Ray
b) H.264 Blu-Ray

You first need an archive format to save your project.

Option 1. If all you are doing is cutting video, HDV "print to tape" will keep everything except the GOPs containing the cuts in first generation (copy of camera original HDV).

Option 2. Almost as good is a transcode to Blu-Ray MPeg2 @ 1440x1080i/29.97 25Mb/s upper field first.
This may or may not be directly playable on your Blu-Ray player. If not, convert to AVCHD with multiAVCHD for playback.

You will need Encore or another Blu-Ray authoring program to create a complete Blue-Ray disc with menus. Most people don't bother opting instead for simplified AVCHD m2ts file playback.

Option 3. Cineform may be overkill in your case. Cineform becomes the timeline format. You convert HDV to Cineform, edit and filter in Cineform, then export encode in the Adobe MPeg Encoder. You would archive in Cineform format.

You need to experiment to see which video/audio formats play directly on your DMP-BD35. All Blu-Ray players play a fully authored Blu-Ray disc. Most will play AVCHD m2ts 1440x1080i or 1920x1080i. Some will directly play MPeg2. Some will play HDV m2t files directly. Audio may need conversion to 5.1 AC3.