AVI to MP4 H.264 buffering problem (not MOOV problem)

[Qexpres]

Hello guys,

I have this problem that has been bugging me for a few days now and I hope someone could help me solving it.

I've received a lot of AVI videos with, im guessing, an MPEG codec (Video: msmpeg4v2, according to ffmpeg). I've tried to transcode these videos to MP4 with an H.264 codec in numerous ways, two of which are Handbrake with "Web optimized" selected and ffmpeg, followed by qt-faststart. I think it's save to assume that the MOOV atom is at the beginning of the output videos, where it should be to buffer nicely.

However, whenever I try to view the video's with either HTML5 video or Flash, there seems to be a problem with the buffer. The videos won't autobuffer when the page loads and once you start playing the video there are a few problems, depending on the browser. Symptoms are not being able to play the video for more than 5 frames, or not being able to jump to other moments of the video (except for the buffered part AFTER the current time position), and a few other minor problems.

After a while I tried to transcode other movie files of my own (other than the AVI files I received for this job) to MP4 and they seem to stream flawlessly. So my guess is that the problem lies within the AVI files. I know that the videos are somehow generated from still images and that the videos have no audio tracks, but I'm really in the dark about how they were created.
It's not an option to record the videos in a different way; it has to work with the AVI files that I already have. Unfortunately I cannot show any of the videos because of security issues, making it not very easy for anyone who wants to help!

In short: I need AVI-files (presumably MPEG codeced) to stream on my site using MP4 H.264 with HTML5 or Flash. The MOOV atom is most likely not the problem. What could be wrong with the AVI files and how do I transcode the files to MP4 files suitable for streaming?

Before I started this project, I was just a simple web designer. I've learned some things about encoding and transcoding on the way, but I really don't know what to look for anymore. Hopefully I made myself clear and one of you guys could help me solving this problem.

With kind regards and many thanks in advance,

Jasper

[Selur]

ever tried to use mp4box instead of ffmpeg for muxing? (ffmpeg from time to time has broken mp4 output)

[Qexpres]

No, I haven't tried mp4box yet. I just downloaded it and I have mp4box working now. I already used -aviraw to extract the video and it creates an MP42 file. How do I get the desired MP4?

Thanks!

[Selur]

I ment only use mp4box for muxing. You can convert you content with ffmpeg, but send it through mp4box afterwards. (+ enable hinting)

[Wolfen]

download mediainfo it'll tell you and us what the files are encoded with but it looks like you demuxed the avi and you have a raw file so now you can take the file and convert it or put it into another container

[Qexpres]

Ok, I did what Selur suggested and I immediately tested the outcome, but no luck! It seemed like chrome behaved differently, but I guess that was me just getting my hopes up.

I downloaded mediaInfo and this is wat it has to say about my videos:

A source file:

General
Complete name : /Users/boot/test.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 4.76 MiB
Duration : 10s 640ms
Overall bit rate : 3 756 Kbps

Video
ID : 0
Format : MPEG-4 Visual
Codec ID : MP42
Codec ID/Info : Microsoft MPEG-4 v2 (pre-standard)
Codec ID/Hint : Microsoft
Duration : 10s 640ms
Bit rate : 3 749 Kbps
Width : 640 pixels
Height : 512 pixels
Display aspect ratio : 5:4
Frame rate : 25.000 fps
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.458
Stream size : 4.76 MiB (100%)

----------------

After transcoding with ffmpeg:

General
Complete name : /Users/boot/test.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 1.32 MiB
Duration : 10s 640ms
Overall bit rate : 1 040 Kbps
Writing application : Lavf54.29.104

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 10s 640ms
Bit rate : 1 000 Kbps
Width : 640 pixels
Height : 512 pixels
Display aspect ratio : 5:4
Frame rate mode : Constant
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.122
Stream size : 1.32 MiB (100%)
Writing library : x264 core 119
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=1000 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

----------------

After hinting with mp4box the size increases slightly, the stream size goes to 95% and it adds "Tagged date : UTC 2012-12-10 22:17:21" to the end. I guess that's not really relevant to me, but hey, I've been wrong before.

What I did:
1. transcoded with ffmpeg:
ffmpeg -i test.avi -vcodec libx264 -b 1M test.mp4

2. put it through mp4box:
mp4box -hint test.mp4

3. checked with qt-faststart if the MOOV atom was moved:
qt-faststart test_hinted.mp4 test_hinted_faststarted.mp4
- returned: "last atom in file was not a moov atom"

Then I uploaded and tested the video, with the same result as before.

I used those commands in Terminal on OS X 10.8.2 (Mountain Lion).

Any ideas? Thanks a bunch so far, though!

[Wolfen]

OK so you need a decompressor for that codec try here http://www.moviecodec.com/download-video-codecs/ and download and install one of those decompressors to view/open those (Microsoft MPEG-4 v2 (pre-standard)) proprietary files. hope it works.

[Qexpres]

I've installed the decoder on my laptop with Windows on it and it played & streamed the video as it should. That's great, but I can't ask all visitors of the site to do the same in order to watch the videos. Did you mean that somehow I have to use that decoder during the transcoding process? Could you be more specific about how I should use the decoder to get MP4 files that can be streamed without the visitors having to download this decoder?

Thanks!

[Selur]

1st using mp42 for streaming is a bad idea, since it's not even MPEG-4 standard conform
2nd encoded a small clip and used mp4box (with hinting) for muxing, uploaded it to webspace I own and wrote a small page to load it:..link removed ...
works fine here on all my systems,...
How do you incorporate the video in you page?
Are you really Streaming (or are you only using a progressive download)? If you do use real streaming, which streaming server software are you using?

[Qexpres]

I use a very simple piece of HTML5 to test the files to eliminate other variables created by players such as Flowplayer and MediaElementsJs. This is what I use to test them:

<video width="640" height="512" controls>
	<source src="/files/videos/test.mp4" />
</video>

You say I shouldn't use MP42 and I get that, but the outputted MP4-file (see the MediaInfo information from one of my previous posts) doesn't have anything to do with this codec anymore, right?

I also tried Adobe Media Encoder to get the desired MP4 files, but other than being very user friendly, it doesn't solve my problem.

However, I stumbled upon a post that said that streamed media shouldn't have the "Trasfer-Encoding: chunked" property present in their headers. In order to do so I had to disable gzip for the folders containing the media, which is okay, because it doesn't make the files smaller anyway. Et voilà, results! I'm not 100% convinced this is the only thing that caused the trouble of the past few days, but at least the HTML5 video element is playing the video's in the browsers that should be able to.

At least I have something to work with now, but if anyone finds there is still something wrong with my outputted MP4 files, please do say so! I don't want to leave anything to chance.

For those who are interested in how I disabled gzip: I created a .htaccess file in the top directory of my media files including:

RewriteRule ^(.*)$ $1 [NS,E=no-gzip:1,E=dont-vary:1]

[Selur]

gzip might be the problem (I normally have it disabled on on my media folders)

[Wolfen]

So now that you have the decompressor that'll open the files properly, install it on the computer you are going to use with (I like handbrake) the transcoder of your choice and you have no other choice than to Re-encode them (the videos)with a more common codec. That codec was Microsofts attempt to corner the MP4 market and it is PRE Standard before the standards came out for MP4. you have to Re-encode all those videos with that codec (MP42)or get everyone to D/L the decompressor which I'm sure you/they don't want to do.

[Qexpres]

I'm really happy with the results right now (actually crying a little)! Selur and Wolfen, thank you both for your help!

[Wolfen]

Anytime, You're Welcome.