remux avi dx50 to mp4

[HWgeek]

Been doing a fair bit of reading and experimenting and wondering why I can't get the following to work:

Have many avi files that were done with DX50 codec back in the day. Now looking to update my library and not re-rip but batch remux the files to mp4 for compatibility reasons. Also needs to play on mac quicktime. Yes the avi does play on quicktime correctly currently, however moving to different viewing platforms sometimes causes problems etc.

[EDIT 4/22/2013] The system has perian codec package installed. And the first sample file provided was mpeg4-asp which does not have native QT support. Will try running mpeg4-sp without perian and examining results.

So running the file through say avidemux, with audio (mp3 to aac) and video (copy) yields a file that works BUT seems to be jerky and not smooth. In quicktime specifically. copying it back to avi, and the jerkiness goes away. Trying to understand what is going on. As I understand it, mp4 container can contain h.263 video so I thought this would have been simple to remux.

ffmpeg on the cli ends up with the same effect, almost like frames are being skipped or missed.

Wondering what I am missing in the whole remux from avi container to mp4!

Yes I know I could do this with handbrake or some other transcoder and go to H.264, but that is a lossy conversion, and I already have compressed these as much as reasonably possible and don't want to loose more video information.

Thanks!!

MediaInfo of source file:

General
Complete name : Sample.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 698 MiB
Duration : 1h 37mn
Overall bit rate : 998 Kbps

Video
ID : 0
Format : MPEG-4 Visual
Format profile : Advanced Simple@L5
Format settings, BVOP : 2
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)
Codec ID : DX50
Codec ID/Hint : DivX 5
Duration : 1h 37mn
Bit rate : 861 Kbps
Width : 696 pixels
Height : 288 pixels
Display aspect ratio : 2.40:1
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.179
Stream size : 602 MiB (86%)
Writing library : XviD 1.2.1 (UTC 2008-12-04)

Audio
ID : 1
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 3
Mode : Joint stereo
Mode extension : MS Stereo
Codec ID : 55
Codec ID/Hint : MP3
Duration : 1h 37mn
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Sampling rate : 48.0 KHz
Compression mode : Lossy
Stream size : 89.6 MiB (13%)
Alignment : Aligned on interleaves
Interleave, duration : 42 ms (1.00 video frame)
Interleave, preload duration : 504 ms

[Selur]

have you tried mp4box?

[jman98]

Your goal is irrational. It's stupid to let Apple dictate that you remux every file you have.

A rational approach would buy to just buy a streaming media player of some type like a Western Digital model that understands AVi and just get on with your life.

As you have already discovered, many devices will not expect MP4 to contain Xvid video and MP3 audio and you are likely to have playback problems even if you do this. I was a bit shocked to remux an AVI file I had with Divx or Xvid video and MP3 audio to MKV so I could add another soundtrack to it and NONE of my various standalone players (Western Digital, Roku, DVD and BluRay players) could play the file correctly despite all supporting MKV and all supporting the audio and video codecs within the MKV container. Apparently all the firmware writers never anticipated that someone might put a Divx/Xvid video with MP3 and AC3 audio into an MKV container, although such a file is completely allowed by the way MKV works.

[Selur]

As you have already discovered, many devices will not expect MP4 to contain Xvid video and MP3 audio...

not really worth buying such a device which not even has some decent mp4 support,...

[HWgeek]

Your goal is irrational. It's stupid to let Apple dictate that you remux every file you have.

Its not entirely Apple in this case, I suspect, but rather adhering to standards rather than just trying to handle all/any cases whether they are correct or not (ie packed files in avi's which were not standard as I understand)

A rational approach would buy to just buy a streaming media player of some type like a Western Digital model that understands AVi and just get on with your life.

As you have already discovered, many devices will not expect MP4 to contain Xvid video and MP3 audio and you are likely to have playback problems even if you do this. I was a bit shocked to remux an AVI file I had with Divx or Xvid video and MP3 audio to MKV so I could add another soundtrack to it and NONE of my various standalone players (Western Digital, Roku, DVD and BluRay players) could play the file correctly despite all supporting MKV and all supporting the audio and video codecs within the MKV container. Apparently all the firmware writers never anticipated that someone might put a Divx/Xvid video with MP3 and AC3 audio into an MKV container, although such a file is completely allowed by the way MKV works.

[/QUOTE]
Mmmm, that would be throwing money at the problem, without understanding WHY there is a failure. Doing this is a learning experience for me in codecs, containers etc etc...
What is interesting about your statement is that you advocate buying a different player, yet you remux an AVI to MKV and all your various players wouldn't work. So you what, just go out and buy another player? It sounds like you did dig into the issue to determine what was going o, rather than just blindly buy another player and thinking your current players are junk.

What is interesting is the WHY primarily of what is going on here.
I could simply crank up handbrake and set the quality high and get 99% quality but never understand what was going on. Especially in this case where the video stream does play fine from an avi container but not from an mp4 container. Odd.

[HWgeek]

have you tried mp4box?

Will do so tonight and post back results.

[jman98]

If you think MP4 is really a standard, well, good luck with that. Many support it only partially. I suppose technically it is a standard, but since nobody enforces it or tests that devices meet the standard, it might as well not be one.

Look, I'm just pointing out that if MKV, which supposedly has fantastically good support, won't support what I did, you are stupid to think that remuxing to MP4, which is even poorer in support than MKV, is going to be a great idea. But I admit that there is some chance it could work.

Life is short dude. Some people get off on banging their heads against brick walls. I guess you are one of those people. My point, which you are missing, is that you could just get on with your life by buying a device that supports AVI. But honestly I do not care if you ever fix your problem or not. I give this kind of advice because in about 10% of the cases, a lightbulb goes on and the person says "You are right. It's stupid to spend all the time pounding this square peg into a round hole." But you're not one of those people so good luck.

[HWgeek]

have you tried mp4box?

Tried it and same results with the jerky video. Would putting up a sample of avi and mp4 be helpful? And if so, what is the best way to do so? Directly on VH or offsite host?

[Cornucopia]

Ok -
It's NOT about bitrate, as the file is pretty darn low with the bitrate already.
It's NOT about resolution, for the same reason.
It's NOT about framerate incompatibilities, as 23.976 (aka 24p pulled-down) is quite common and expected.

When you say it WORKS, do you mean it decodes the file correctly? (no solid GREEN, or WHITE, or BLACK or garbled frame)
You could try rewriting the fourcc code from DX50 to XVID, etc to see if a different decoder works better with it.
What decoder are you using to decode DivX/Xvid on your Mac? - Perian?, Xvid? Divx?, 3ivx? Which one you use (and which platform it is coded for - PPC vs. Intel) might make a big difference, performance-wise.

So you are saying that playing MPG4,part2-encoded video from within an AVI container will play back smoothly from within Quicktime Mac, but playing MPEG4,part2-encoded video from within an MP4 container will play back jerkily from within QT Mac? I want to make sure we're on the same page.

If that is the case, why would you not just leave them as AVI?

Scott

[Selur]

Would putting up a sample of avi and mp4 be helpful?

definitely

Directly on VH or offsite host?

small sample (a few seconds) directly here should be fine.

[HWgeek]

Okay, sample 1 is a trim of the original avi, no modificaitons. Opening in QUICKTIME, plays fine.
Sample 2, done with avidemux (same issue is seen with mp4box, gui windows) copy video with audio converted to aac (otherwise there was no audio if left as mp3)

[Cornucopia]

So, you do realize that Quicktime does not natively support MPEG4-ASP, only MPEG4-SP, don't you? To get MPEG4-ASP support on Mac, you have to add additional components - either DivX, Xvid, 3ivx or Perian (which probably provides Xvid). Without the additional components, you might get a white screen (or similar), or you might get a screen with video that is garbled when playing (because the additional features that ASP provides are not fully recognized).
I say this, because the AVI version has a fourcc of DX50, but the MP4 does NOT. QT will know to correctly pass along MPEG4-ASP video to the add-on component, since it clearly has a codec ID that is governed by the add-on, whereas QT will look at the MP4 and see "MPEG4" as the codec ID, will attempt to use its native capability in decoding.
I'm guessing you DID NOT use QT to do the transcoding/re-wrapping, but instead used ffmpeg or similar...

A maintaining of the codec ID (aka fourcc) as DX50 (or as Xvid), or of restoring it back to that after re-wrapping, using a hex editor or similar metadata chunk editor, could help alleviate your problem. I can't guarantee it will take care of everything, but you at least have to eliminate obfuscations like this.

Scott

[jagabo]

Have many avi files that were done with DX50... So running the file through say avidemux, with audio (mp3 to aac) and video (copy) yields a file that works BUT seems to be jerky and not smooth.

Try running them through Mpeg4Modifier with the unpack option before remuxing. Or, if they're already unpacked, pack them.

[Selur]

don't use qpel, gmc or mpeg-quant when encoding stuff for Quicktime

[HWgeek]

So, you do realize that Quicktime does not natively support MPEG4-ASP, only MPEG4-SP, don't you?

Actually I was not aware that Quicktime did not support MPEG4-ASP! And that in itself certainly is a big clue, and provides more clarity why there seems to be an issue.

I say this, because the AVI version has a fourcc of DX50, but the MP4 does NOT. QT will know to correctly pass along MPEG4-ASP video to the add-on component, since it clearly has a codec ID that is governed by the add-on, whereas QT will look at the MP4 and see "MPEG4" as the codec ID, will attempt to use its native capability in decoding.
I'm guessing you DID NOT use QT to do the transcoding/re-wrapping, but instead used ffmpeg or similar...

I used in this sample set, avidemux to copy the stream to the new container and only convert the audio to aac from mp3. I am trying to avoid several things in this process, and learn also.
Avoid - transcoding with a lossy transcoder process, having to load up 3rd party codecs/add-on like perian to run the video.

A maintaining of the codec ID (aka fourcc) as DX50 (or as Xvid), or of restoring it back to that after re-wrapping, using a hex editor or similar metadata chunk editor, could help alleviate your problem. I can't guarantee it will take care of everything, but you at least have to eliminate obfuscations like this.

Scott

I would assume the metadata editing would be to set the QTAG back to DX50, while leaving the wrapper as mp4, since then QT will use perian to decode properly ???

Thank you for the very clear explanation, this info was great and useful!

I will strip out perian so that I am ONLY running native Quicktime, to see where I get with conversions. Also will find a file sample that is set up with MPEG4-SP instead since it seems like that will have a higher likelihood of working in a copy conversion vs a transcode.

[HWgeek]

Ok -
It's NOT about bitrate, as the file is pretty darn low with the bitrate already.
It's NOT about resolution, for the same reason.
It's NOT about framerate incompatibilities, as 23.976 (aka 24p pulled-down) is quite common and expected.

So you are saying that playing MPG4,part2-encoded video from within an AVI container will play back smoothly from within Quicktime Mac, but playing MPEG4,part2-encoded video from within an MP4 container will play back jerkily from within QT Mac? I want to make sure we're on the same page.

If that is the case, why would you not just leave them as AVI?

Scott

Initially I was looking to standardize on a container so that any mac in the house could play files as needed with no additional codecs or software. Your other post has been helpful in understanding why there seems to be a problem, and has given me some idea's on how to recode with some different options, to see what might work.

As long as I didn't have to load up any add-on's then leaving them in AVI would be acceptable as QT X will open avi's and as per apple handle mpeg4 and h.263 (of course didn't know it wouldn't handle mpeg4-ASP, just SP. go figure!)

[HWgeek]

Have many avi files that were done with DX50... So running the file through say avidemux, with audio (mp3 to aac) and video (copy) yields a file that works BUT seems to be jerky and not smooth.

Try running them through Mpeg4Modifier with the unpack option before remuxing. Or, if they're already unpacked, pack them.

This particular file didn't have packed p&b-frames, and as I recall from a previous test the packed streams were more of a problem, BUT I will try packing them just for the sake of removing variables.

[HWgeek]

don't use qpel, gmc or mpeg-quant when encoding stuff for Quicktime

I note that in the mediainfo information that there is no qpel and no gmc warpoints. However I didn't see how to tell if mpeg-quant was set up or used. How is it possible to determine this? Is it a function of MPEG4-ASP vs MPEG4-SP not having that function?

[poisondeathray]

don't use qpel, gmc or mpeg-quant when encoding stuff for Quicktime

I note that in the mediainfo information that there is no qpel and no gmc warpoints. However I didn't see how to tell if mpeg-quant was set up or used. How is it possible to determine this? Is it a function of MPEG4-ASP vs MPEG4-SP not having that function?

Mediainfo will say "custom" for this line if a custom quant matrix was used

Format settings, Matrix : Default (H.263)

I think only ASP supports custom matrices . It's greyed out in xvid encoding options when you select a SP



If you encode to MPEG4 using QT, you have the option of MPEG4 basic vs MPEG improved . Obviously encoding using quicktime should yield a compatible file (just analyzing to see what is compatible in QT)

Basic

Format : MPEG-4 Visual
Format profile : Simple@L1
Format settings, BVOP : Yes
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)


Improved

Format : MPEG-4 Visual
Format profile : Advanced Simple@L3
Format settings, BVOP : Yes
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (H.263)



So it would appear AS@L3 is supported by QT , at least on windows . It might be that your AS@L5 file exceeds the maximum level supported by QT , and on a Mac you need to do what Scott said above

[vhelp]

mediainfo and mpegmodifer both report quant matrix, but they get it wrong. i believe they have to calculate some things to determine the quant values. anyway. i know.. i tried many times to figure it out so i could add the feature in my matrix app. sometimes, though rare, it gets it correct. most of the time, it is wrong. too many to make the assumption that i am getting correct quant matrix info. so far i have not been able to find an app or utility to properly calculate the quant matrix. it would have gave my app the edge it needed and i would have share the utility on this forum, but is worthless withtout this feature, imho

mediainfo displays the matrix in hex format as one string, separated by a colon.
mpegmodifer displays the matrix in grid format (under advanced)

[HWgeek]

Some summary -

Poisondeathray has the gist of it - what is supported by QT. For an MP4 container, yes, both MPEG-4 SP and ASP up to L5, however only ASP L3 by QT. After removing perian, having installed it ages ago and basically forgotten about it, sure enough the AVI wouldn't play, as expected.

A MPEG-4 SP will go into an mp4 with aac audio and work just fine. I can take QT PRO 7 and convert a MPEG-4 ASP@L5 to L3 and it too then will work. What seems to be the case in the jerky video is QT trying to take a ASP@L5 and play it with a decoder that handles L3.

Packing the bitstream made it worse, as I expected. by combining frames in the AVI as packed the info when played via QT gets lost and you get video that clearing is missing b-frames from i-frame to i-frame. gets pretty ugly.

Questions:
I expect that QT is transcoding from L5 to L3, can someone who has more MPEG knowledge confirm this as true or false???

Trying to change the fourCC info on a DX50 (aka MPEG4 ASP@L5) using avic.exe. I can change info, but am not sure what are legitimate entries for description and codec. Goal is to modify an avi with ASP@L5 to read as an ASP@L3 and verify that the jerky video shows up. Is there a better program to modify the tags???

Given I may have to transcode from ASP@L5 to ASP@L3 which tool would be recommended for doing it batchwise? I have tried ffmpeg, but, have had difficulty keeping the video quality, even with a q=0 setting. QTPro7 did a great job but not really easy to cli batch automate.