Ffmepg and Xgrid?

[Durendal]

I'm setting up a render farm of old Macs with Xgrid. I'm looking to use ffmpeg to format the video, but I can't find any reference to Xgrid. Has ffmpeg been written for parallel operation? Can it run on a cluster using Xgrid?

[major]

It is difficult to benefit from Xgrid with ffmpeg as video processing imply a very important amount of data which would need to be networked. I don't think useful optimizations may be expected in the immediate future.

[Durendal]

Yes, but will it actually work with XGrid? An application must be written properly for it to be clustered. If ffmpeg doesn't have the code for parellel operation, it simply wont work with XGrid at all.

[major]

Currently there is no support for Xgrid in ffmpeg.

[Durendal]

Sorry to resurrect the topic, but I've been busy.

I realize that video encoding over a cluster would mean a large amount of data would have to be pushed over the network. However, clustering ffmpeg on a 100mbit network or better could see enormous benefits, especially since Xgrid has been so well integrated into Tiger. Let's say that it takes, for example, a 1ghz G4 an hour to encode a certain video. If that were split among four 1ghz G4s on a network, theoretically, the time would be cut to 15 minutes. Now, we have to take into account the time to transfer files over the network. However, if we have a decent switch/router with a solid backplane, the time taken to transfer the video will be minimal in regards to the time taken to encode the video on a single machine, especially when the compressed video comes back to the controller. How long is it going to take to send 30 frames of footage to three other computers? Very little. Certainly less time than it would take for the single computer to render all 120 frames on it's own. Granted, faster computers would need a faster network (gigabit or fiber channel, maybe even firewire) to get the full benefit, but there are plenty of people running such networks that would love to be able to cluster their video encoding operations. Video encoding has long been a poster boy for parallel computing. Sorenson has been looking into writing Squeeze for parallel operation. Ffmpeg could benefit greatly from it and allow it to snag a larger portion of the encoding "market".

[major]

I have a to-do list entry about network encoding. It is not an easy task so it could take months to sort something out of it.

[Durendal]

Okay, cool. Should network encoding be added to ffmpeg, would it be added to the 10.2(.8) version, or perhaps put in a new 10.2 version, as 10.2.8 supports Xgrid?

[major]

No idea for now, sorry.

[TylerL]

It should be (relatively) trivial to split a .vob file up by chapters and send each chapter as a workfile to an Xgrid Agent.
Each Agent could compress as CQ or CB (no dependency on other chapters) and submit the compressed file as the "result" to the master, which will stitch (join) the final file together.

This is something I was thinking about toying with using ffmpeg. I'd have to learn more about Xgrid first, though

[major]

There are indeed many possible methods, all involving splitting the data at some level and rejoining them back. The correct method should be to work at GOP level though, and the method should work with any format, not only .vob.