ffmpeg writing movie to memory.

[TugBoat]

I had this happen a couple of days ago but didn't realize what was happening. Basically, it involves a QT .mov with 3ivx video and AAC audio. Using the ffmpeg codec, the movie will be written to memory and when it runs out of real memory, it will write to virtual memory until you cancel the encode or it finishes. The file generated on the hard drive will remain at 0KB until you cancel or finish. Then if you do either, the movie will be transfered to the file on the HD and it will magically be the correct size and your memory will be freed. WTF?

I have this happening right now. The movie is described as 3ivx D4 4.5.2a8 video and MPEG4 audio. It also has a chapters track and it's inverted with the sound track first, video track second, and chapters track third. On this particular movie, to get around the inverted audio, I made a copy of the movie and then deleted the audio track on the copy. That way, ffmpeg sees the video track without using the inverted audio. I used the add audio option and pointed it to the original movie where ffmpeg only sees the audio file. I DID NOT do this on the other two movies that were also written to memory.

So why is this movie being written to memory verses the HD? Naturally the preview option won't work, and nothing will open the .ff.mpg file on the HD 'cause it's 0KB. My only option is to let it run until finished, no way to view the output 'til it's finished. And it's already consumed all my real memory so now it's writing to VM. I can watch all this in the Activity monitor.

Progress says;
Encoding started on Tue Nov 9 10:37:54 EST 2004
Input #0, mov,mp4,m4a,3gp, from '/Volumes/Video/Movie_copy.mov':
Duration: 02:32:20.7, bitrate: 1276 kb/s
Stream #0.0: Video: mpeg4, 720x424, 25.00 fps
Input #1, mov,mp4,m4a,3gp, from '/Volumes/Video/Movie.mov':
Duration: 02:32:20.8, bitrate: 1276 kb/s
Stream #1.0: Audio: mpeg4aac, 44100 Hz, stereo
Output #0, vob, to '/Volumes/Video/Movie_copy.mov.ff.mpg':
Stream #0.0: Video: mpeg2video (hq), 352x480, 29.97 fps, q=1-8, pass 1, 1610 kb/s
Stream #0.1: Audio: mp2, 48000 Hz, stereo, 192 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #1.0 -> #0.1

Ideas?

+++++++++++++++

This is exactly what it did the other day on another 3ivx AAC movie. The thing just now finished the first pass, an unusually long time, and the machine went totally unresponsive for 4 or 5 minutes. My guess is it was busy reading/dumping 900MBs of real and 1.7GBs of virtual memory. Now it's working on the second pass, the file on the HD is still 0KB and I can see the memory usage steadily increasing. This type of encode is usually done in real time on this machine. It should have finished the second pass hours ago

*******************

It finally finished after almost 10 hours. It shouldn't have taken over 3 hours. I have a video track but no audio track, this is pretty much the way things went the last time. The only errors listed in the log is about 50 of these [mpeg4 @ 0x331210]vop not coded. And then this;
video:1464040kB audio:0kB global headers:0kB muxing overhead 24.364391%
bench: utime=17191.860s
Encoding completed on Tue Nov 9 20:10:24 EST 2004

I have another one of these movies, it's one of the same series as the last one and also has inverted AAC & 3ivx. Wanna bet things will be the same?

[TugBoat]

OK, found a solution;
https://www.videohelp.com/forum/viewtopic.php?p=1096260&highlight=#1096260
Is this another bug?

Same video only movie as before, only without the AAC audio addition. First pass finished in 1:35 minutes. It's being written to the HD this time, and the preview option works.

Second pass finished, total encode time, 2:40 minutes verses the 10 HOURS it took the last time. Exact same movie, exact same settings. The only difference was the AAC audio.

[thoughton]

Sorry, your post is very long, but I don't see what the problem is - isn't it normal behaviour in 2 pass encoding that the first pass doesn't write anything to hard disk ?

No idea why your AAC is going so slow though...

[TugBoat]

No problem? Any other program that proceeded to consume ALL your physical memory and then proceed to write over a Gig and a half to VM would be said to have a serious memory leak :P

The program always writes BOTH passes to the HD in all other cases. If you check the links on the other forum you'll see that Major has confirmed that mPlayer doesn't recognize AAC audio. I don't know if things have changed since then, I'm running a fully updated system, but, not only doesn't it recognize AAC audio, but apparently causes a program hang taking 3-4 times as long to encode AND writing everything to memory.

I know about it now. But, I almost force quit my machine the first time it went unresponsive during a memory dump. I did force quit a few apps. The second time, I was already suspicious of ffmpegX and caught it with the Activity monitor. Anything that makes your machine use up all it's memory and go unresponsive is Not good

[thoughton]

No problem? Any other program that proceeded to consume ALL your physical memory and then proceed to write over a Gig and a half to VM would be said to have a serious memory leak :P

The program always writes BOTH passes to the HD in all other cases. If you check the links on the other forum you'll see...

Hehe okay, I guess I misunderstood the problem I always thought the first pass was strictly analytical and no part of your final file was written. My mistake I guess
Also, I'm a total layman when it comes to virtual memory, but I do wonder what the gigantic VM numbers in top and activity monitor actually mean. According to top i'm currently using 7.43GB of VM, but this Mac only has about 6GB free after startup...

[major]

Nothing should be written in VM for such big sizes. I never observed that. If you experiment it, please report reproduceable data to major4@mac.com. Also, please check if the actual free size in your disk decreased (by using apple-I).

[TugBoat]

Well, I've tried 5 different movies on two different machines. Both are fully up to date. Same same. So far, any movie with AAC audio will cause ffmpeg to write to memory instead of the HD. It's easy to check. Then, all you have to do is cancel the encode, disable the audio encode, and then it will write to the HD. I don't know if it will still take 3-4 times as long to encode or not. I'm not going to chance it, it's too easy to just delete the audio track from the movie and not worry about it.

And, I can't seem to get the 'Movie Audio to X' preset to work either. It seems to completely ignore the AAC audio and instead encode the video track even though the video encode is disabled. It appears that AAC audio is completely borked at the moment when using the ffmpeg codec

OK, back to the SAME OLD GOTCHA. YOU HAVE to use 'DECODE with Quicktime'
You still won't get any audio, but it will write to the HD. Try it without 'decode with QT' and it will write to memory. I don't like using that option, for reasons I've repeated numerous times

[davemchine]

I don't have the analytical information that tugboat offers but I will say that ffmpegx will bring my computer to it's knees after a few conversions. I encode vobs to mpeg4 mostly and with my g4-800 (640mb ram) I could encode 4-5 before the machine went to slug mode and required a reboot. I just purchased a g5 but am waiting on ram so I only have 256mb in it right now. I have a gig of ram on the way though so i could report back later to say if the problem persists. Personally I've just been considering it the price of admittance. Other than this small problem the program is pretty fantastic. At least I don't know any other way to get the job done!

Dave

[thoughton]

I'm not sure if it is ffmpegX itself. I've had it converting stuff for three days at a time before without noticing sluggishness afterward. However I have noticed that sometimes the 'progress' app will jump to 70% CPU and stay there for at least several minutes (I always force quit the progress app by that time) even if nothing is being encoded.

[TugBoat]

Yea, the problem described in this thread only happens with AAC audio tracks without the decode with QT enabled. And, it will only cause the machine to be unresponsive for about 3 or 4 minutes during which time the data it's stored in memory is either dumped at the end of the first pass, or written to the HD upon completion. At all other times the machine responds perfectly normal.

[DeusExMachina]

Just want to jump in as another voice in the ffmpeg (not ffmegx, different animal) will drag a machine whimpering and sniveling to its knees when attempting to decode to AAC controversy. It eventually becomes totally unusable for anything. I have gotten around this by demuxing the file and using A.pack or iTunes to encode the audio, but it is a pain. Main drive space is totally assimilated and available RAM (almost 1 Gb here) gets co-opted. Eventually things start locking up. You may not notice this if you are working on small files, but 300+MB files make short shrift of a machine.

In answer to major's question, yes, free memory soon goes to zero. I have encountered this often, the dreaded "system memory is running low" warning dialogue, forcing me to juggle files from drive to drive until I can get the encode to finish, reboot, freeing up the memory, then copying everything back. As an aside, I have found that the memory space is NOT released until reboot, even after quitting ffmpegX.