MP4 AVC/H.264 encoding

[Jacobus Serpenti]

Hello everybody,

A long time has passed since my last posting. I recently bought an Xacti HD1010 camera, which give excellent results, both on picture and video footage side.

I still use ulead mediastudio pro 7.3. To be able to use it, I use the handy MP4CAM2AVI, to encapsulate the AVC/H.264 into an avi container which MSP can edit and handle.

After making the usual editing, I would like to create an avi file containing AVC/H.264 and MP3 streams.
I tried several different ways with various unsatisfactory results :
1. Divx HD :
2. virtualdub+ffdshow (h.264) :
3. TmpgEnc, Mpeg HD (HP@HL) : 8) (but motion blur, not as smooth and fluent as original footage)

What I like in MP4CAM2AVI is the option "uncompressed" I would like to do the same in virtualDub+ffdshow

Would somebody be so kind as to help me figure this out ?

Thanks in advance.

jacky serpenti

[jagabo]

Avi Demux can read and process MP4 files from that camera.

[45tripp]

i don;t think that's the question

of course,
i'm not sure the question is clear...

you define a goal.

I would like to create an avi file containing AVC/H.264 and MP3 streams.

yet you do something else?

TmpgEnc Mpeg HD (HP@HL) :

if you're looking for encoding tips,
probably
provide a sample to help out.


tripp

[jagabo]

I found a sample at this web site:

http://www.akihabaranews.com/en/news-16152-New+Xacti+HD1010+Hands+On!+Not+really+revolutionar y%E2%80%A6.html

[Jacobus Serpenti]

Hello, let me give some more details about what I would like to achieve :

I want to watch my creations on an TViX HD4100sh.
After editing in MSP 7.3, I would like to have a file to send to the TViX.
Since the files created by the HD1010 are a good compromise between size and video quality, I would like to create the same type of file out of MPS 7.3.
I frameserve to VirtualDub, use ffdshow as a codec (H.264 with 2 passes quality setting to 70%) but the resulting file is either to big (10 times bigger than the original project) or of degraded video quality (like poor DivX).

What would be the appropriate settings or is there another way around directly out of MSP7.3 (something uncompressed or lossless, to get the same thing out as in)?

Thanks in advance.

jacky serpenti

[poisondeathray]

If you are shooting in the highest quality mode, it is actually, 29.97 interlaced (unlike some Canon's who shoot PsF, ie. 30p in 60i) - therefore at some stage you should deinterlace to better quality (I don't think that TviX model has a good deinterlacer, so you should deinterlace and encode as progressive)

Avidemux will not handle interlaced h264 or PAFF, MBAFF streams very well, only progressive h264 material at this point in time (so if you were shooting in 720p, it will work)

I would use avisynth to do this, and deinterlace. If you needed MSP 7.3 to do some special editing (i.e. not simple linear editing), I'm not sure if it accepts .avs scripts. In that case, then you could use the .avs script to generate an uncompressed (or lossless compression like lagarith, huffuv) avi to use in MSP 7.3 by feeding the .avs into vdub, for example. Then do your edits, and export again using lossless or uncompressed and encode. I'm not familiar with MSP7.3 at all, and there might be alternatives to using that. (I'm assuming MSP gives you the option to export uncompressed)

Using ffdshow to encode to h264 will definitely give you inferior quality. Consider using a x264 CLI encoder (many options, eg. xvid4psp, ripbot264, MeGUI).

h264 in avi is not completely compatible and may cause "jerkiness" on playback, especially with more than 1 b-frame. Consider using .mp4 or .mkv as the container (since that TviX supports both containers)

Make sure you use a profile (encoding settings) that the TviX can play for device compatiblity, otherwise it won't play back at all. I think you can use an xbox360/ps3 profile - but you should check with the TviX forums to be sure; you might have to lower the VBV buffer. (AVC Level 4.1, max 3 ref. frames, max 3 b-frames, no b-pyramids, VBV 24000, VBV Max Bitrate 24000). Most GUI's will have preset profiles/settings for this. Also test the settings out on a short clip to see if it plays before you waste hours of time encoding the final goal.

[jagabo]

Avidemux will not handle interlaced h264 or PAFF, MBAFF streams very well...

Do you have any idea when it will be updated with the newer libavcodec which supports PAFF and MBAFF?

[poisondeathray]

Do you know when it will be updated with the newer libabcodec which supports PAFF and MBAFF?

I don't know exactly, but I hear in version 2.5 - I don't know when the offical release is. I saw a preview of the new GUI and there are some great changes coming.

There has been significant improvements in the libavcodec library recently with decoding h264 streams - I don't think they have made it into the avidemux svn builds yet. Linear access is much improved, but random seeks get corrupted still. It's still not "perfect"

There is no "perfect" h264 decoder yet. CoreAVC 1.8 still gets corruption on some streams with abnormal motion vectors.

DGAVCdecNV looks promising if you have a CUDA enabled card, but still some bugs. CoreAVC 2.0 is also supposed to support CUDA as well for decoding.

-sorry for the thread hijack

[Jacobus Serpenti]

Thank you very much guys , I'll test all these options, and keep you informed with a further post.

jacky serpenti

[Jacobus Serpenti]

Hello everybody, hello poisondeathray,

I've tried several things this night, but I'm getting more and more puzzled.

MSP stands for Mediastudio Pro, the ulead NLE tool. Between the () is meant video/audio stream.

Previoulsy when I mounted DV movies, I was familiar with the following workflow :

Camera -> AVI file(DV/PCM) -> MSP 7.3 (footage, titles, music, F/X) -> AVI (DV/PCM)

Once the final AVI file obtained, either compress it to Mpeg2 for DVD's or to DivX for TViX.


Now with the Sanyo HD1010, it use a similar workflow :

Camera -> MP4 file(Mpeg4 AVC/H.264,AAC audio) -> MP4CAM2AVI -> AVI(H.264/MP3) -> MSP7.3 -> ????

Although MSP7.3 is designed for standard definition formats, it accepts and properly handles the Sanyo HiDef files, but it cannot output HiDef files, except for WMV 1080i or WMV 720p. Formats I'm not interested in.

So I made a test with the lossless Lagarith Codec directly out of MSP: the input sample file sized 32MB (30sec of footage in MP4), output file sized 363MB. How comes that the file gets so big ? And the panning scenes are blurred, whereas in the original MP4 file these are sharp and fluid.

Another test with the "non compress" option form MSP : same result : an enormous file and unsatisfactory sharpness

What I need out of MSP is something similar to what is done by MP4CAM2AVI : put the mpeg4-AVC/H.264 stream unaltered into an AVI container

Any hints on how to obtain this ?

I would really appreciate since the sanyo give outstanding results in comparison to my older Panasonic 3ccd NV-GS400

Best regards

J. Serpenti

[poisondeathray]

You shouldn't put h264 streams in .avi containers. They are not completely compatible. A hack is used to "stuff" it in. You often get studdering issues on playback.

Lagarith uses lossless compression. It lossless - meaning identical to the original. The deterioration in quality is probably introduced from mp4cam2avi, or MSP. Playback might be "blurred" because your computer might be too slow to handle such a large bitrate. You can try encoding the lagarith output to something else. The big file size is expected. If you used uncompressed it would be even bigger.

I said earlier that highest quality mode is interlaced - did you deinterlace? Or are you using 720p? This could be the cause of your poor quality

h264 is difficult to edit. It is highly compressed. I doubt an older version of MSP can do it very effectively. The newest version of Vegas, and Premiere CS4 (not out yet) can handle AVCHD natively. There are a few other payware editors that can as well. You can also use free options for simple editing, do a quick search this has been discussed freqently here.

I believe the limitations in your workflow are with MSP and mp4cam2avi. You could try importing as uncompressed or lossless into MSP to rule out mp4cam2avi - again huge filesize - but identical quality - importing into the NLE. Another option - if MSP accepts avisynth scripts, you could frameserve into it. Personally, I would change editors.

[Jacobus Serpenti]

Hello poisondeathray,

I shoot in 1280x720@59.941 fps (I don't know whether this is Frames or fields, but according to Gspot it says 29.97 frames per second, although the progressive flag is not highlighted).

I did a comparison between the original MP4 and the one converted through MP4CAM2AVI, no visual difference even on my full HD plasma, Gspot reports the same caracteristics for both files, except for the audio : from AAC to MP3.

I also recently upgraded to 4.8 gig AMD dual core with 4GB of ram. Editing is smooth and both PC and MSP are responding rapidly, no locks, no crashes, all features in MSP work well.

MSP cannot import footage other than AVI, Divx, mpeg or WMV. I tested VideoStudio plus 11.5, but it cannot handle the Sanyo MP4 files, more exactly it crashes every time.

If there is another option to get that H.264, unaltered, into another container, I'm open to suggestions.
I'll still do some tests with the X264 link you provided me, there is a "no compression" option in that codec.

Best reagards
J. serpenti

[yonta]

If you only need just to cut some parts off your video, this also is one way to do the job.

1. extract .264 and .aac from your.mp4 using mp4muxer
2. mux .264 into avc.avi using avc2avi(video gurus don't recommend avc in avi container, though)
3. mux avc.avi and .aac into avc.aac.avi using AVIMux GUI
4. edit avc.aac.avi and save to new.avc.aac.avi using VirtualDub(direct stream copy mode. need to enable ffdshow vfw h.264 decoder)

No encoding involved and all programs are freeware.

[poisondeathray]

yonta's suggestion will work, but it's not frame accurate. Depending where your I-frames are, you might be 10-15 sec off each cut. If your footage is truly progressive, you could also do it all in avidemux (simple edits) - but again, not frame accurate.

The problem occurs when you add in transitions and effects in a NLE - the whole file is re-encoded (=quality loss) unless you have a smart rendering editor.

The highest quality would be achieved by using lossless through the entire workflow - this preserves quality but takes lots of HD space.

The other containers that are 100% compatible with h264 video (and aac audio) are .mkv and .mp4. You could use mkvmergegui or YAMB to place the streams into those containers - but I doubt MSP will accept them.

The other link, if you were referring to avisynth - is just a frameserver - it feeds frames to another application (so there is no codec, its like a dummy file or vehicle so the other application can "see" the original file). You can actually do many, many things in avisynth (even add transitions, fades) and just feed that .avs script into any encoder.

Assuming that there are no issues or container compatibility problems with the mp4cam2avi output, the problem is STILL with MSP (either it's settings or encoding output characteristics i.e. can't output HD). If you can't output something as simple as proper uncompressed, lossless or HD content, it's time to get a new NLE.

You mentioned you tried exporting a clip using lagarith, but it was "blurred" - It's tough to know what you mean unless you post a screenshot or small sample. That might give a clue; perhaps it was a setting. Again this suggests something amiss with MSP or something done in MSP, since lagarith is lossless.

What is your final format goal?

[jagabo]

The Xacti sample I downloaded from the site I mentioned earlier was 1280x720, 29.97 fps, progressive. It had only I and P frames and a GOP size of 30. AviDemux had no problems dealing with it. That leaves you with ~1 second granularity for simply cut editing without reencoding.

[Jacobus Serpenti]

Hello guys,

I think I've found it!!!!

I performed some basic tests : check what goes in and what comes out.
MP4 files converted with MP4CAM2AVI are ok, same visual quality as original.
Files in MPS7.3 are ok, detected as 1280x720 HiDef.
The problem lies with the frame server "Debug Mode FrameServer" : lots of combing effects in fast moving scenes.

Noticing this, I encoded a small MP4 file directly in virtualdub with the X.264 codec, just to check : excellent quality even in the fast moving scenes.

So I started digging in the various option of MSP when one needs to create or export a video file ... so I came accross an options dialog related to Smart Rendering, there was a checkbox called "auto de-interlace" (whatever that might mean, I don't know) but it helped, a lot, as a matter of fact, because the combing effects were gone.

I'll keep you informed with my further investigations

Best regards to all of you.

J. Serpenti

[arn1976]

Hi Jacobus, im using MSP 7.3 too, but i'm trying to figure out how to:

Capture HD footage (have a sony FX1)
Edit the captured footage in MSP 7.3
Export to burn in Blu-Ray

Do you think that would be possible????? How?

I'll appreciate any help you can give me, im lost !

Tnx

[jagabo]

lots of combing effects in fast moving scenes... "auto de-interlace"... the combing effects were gone.

Duh!

Interlaced frames contain two half pictures, called fields. One field is contained in all the even numbered scanlines, one in all the odd numbered scanlines. These fields are snapshots taken at two different times. So if there is any motion between those two snapshots you will see comb artifacts if you display both fields at the same time (as is typical on progressive computer monitors). Deinterlacing is the process of turning those two half pictures into one whole picture, hopefully with as little artifacting as possible.

If you take a 30i video (60 fields per second) and convert to 30p you are losing half the temporal resolution and as much as half the spacial resolution (depending on the deinterlacing method). It's also possible to convert 30i to 60p to retain more fluid motion.