Automated Conversion of MPEG2 to MPEG4/DVD-MPEG2

[PC Master]

First, my sincerest apologies to the moderators and administrators of this forum if this thread violates the 'No DivX to...threads' or similar rules for this board. I couldn't find a board that I thought this thread would fit in exactly, so feel free to move it to wherever you want.

Essentially, what I have is an HTPC that will be recording MPEG2 video to my HDD. What I want is to set up a piece of software that will automatically check for new video files in a specified directory at regular intervals (perhaps every hour or so), and then add any new files found to a queue. The program would work on transcoding the video files (using a pre-saved encoder configuration) to either some form of MPEG4 (probably XviD) or to DVD-compliant MPEG2 (I'd like solutions for both formats if anyone can suggest them) during idle time on the PC. By encoding only during idle, the work would still be done, but the encoding process would not interrupt any other stuff I want to do with the HTPC, like timed recordings and such.

Does anyone know of a way I might accomplish this goal?

[scottie78]

You would need to write a proprietary program in VB or something like that I would think.

[PC Master]

I was hoping to put off writing a custom program until later (as I don't have time to finish learning C++ right now, and would really prefer not to mess with VB at all).

If there's not a program already available to do this, is there an encoding software that uses command-line parameters. I could execute it as a scheduled task with parameters to encode C:\Media\Video\*.mpg and have windows always set the process to very low priority. Then the only two things I'd have to do is figure out a way to keep it from starting a second instance if the encoding takes more than 24 hours or so (perhaps a lot of recorded video in a day with minimal idle time or some such), and then come up with a way to automatically remove the files that were encoded upon completion of the encoding process. Either that, or find a way to make the program automatically skip any video files in the directory that have already been converted (ie - if the target file exists already).

Any further suggestions?

[iantri]

Yes, this can be done.

Under Windows? Well.. that is tricky. It's a shame you didn't build your HTPC with MythTV and Linux, where nearly everything can be done via the command line and this sort of thing is easy to do.

Encoding during idle time only should be easy to do; just spawn a process with a priority of low and higher processes should take precedence. Not sure about disk usage though.. the simultaneous activity could cause dropped frames. You could, as an alternative, have it happen at midnight or something.

Here is a rough idea for how I would do it; are you familiar with unix tools like bash and cron?

1. Install cygwin. It is a sort of unix on windows thing that provides unix-like facilities, including bash and cron.

2. Install the mplayer beta for Windows (from the Mplayer website, http://www.mplayerhq.hu). It includes mencoder, a command-line encoder. This is to do the encoding to Xvid/divx or Mpeg-2, however MPEG-2 may be somewhat tricky; the quality will probably be bad, and it is REEALY slow, and I think you need some tools that aren't yet available pre-compiled. I think Cinema Craft Encoder has a command-line mode.. if so this is an option, but you may need to write a bash, perl, python or your favourite scripting language script to craft Avisynth scripts. XVID output from mencoder should be very high quality -- it is from the same source as Nic and Koepi's windows binaries.

3. Set up a cron job to run a bash script that checks the directory and loops through each file. Have it run when you wish. Making a note of already processed files is no problem; use the archive attribute. It doesn't actually do anything (like read-only, hidden or system), it just is an indicator for backup programs or anything else that cares to use it. Or you can delete the files afterward with a simple rm. Or you can use a combination to make sure that if the task runs across the time you set it to start it doesn't try to encode a file twice or remove one being encoded. (i.e. set archive tag when process starts, delete file when ends. Have script check for flag before start.)

4. Consult the Mplayer documentation for examples of the commads to run to encode files.

So, doable, just a bit complicated. Ask if you have questions.

[PC Master]

Well, I experimented with using KnopMyth and such, but the problem I found was that some of the components I wanted (like DVB support) aren't available pre-compiled, and my knowledge of unix/linux compiling, or even the OS itself, is rather limited, so I haven't been able to use that. Additionally, I'm looking forward to seeing Meedio TV in October.

The method you suggested seems like it would be fine if I want to take the time to write up a script (currently I only really know PHP, but I could run that via PHP's CLI mode). If possible, however, I'd prefer to stay with Windows programs. Do you happen to know of any Windows MPEG2->AVI encoders that work via command line?

[iantri]

Bash is both an interactive shell and scripting language that is similar to a Windows batch file, except with real programming capabilities if necessary, like looping and evaluation.

I do not think you can do this with a Windows batch file (at least not the part about automatically encoding from a directory). Writing a bash script would be much easier than you think..

I'm not really sure what you mean by Windows programs? Everything I mentioned are command-line programs that run under several platforms, including Windows. Precompiled Windows binaries are available. No different from any other command-line program (Windows or otherwise).

As I said, Cinema Craft Encoder has a command-line mode, but regardless of the program this only handles the encoding step. There is the step of removing/flagging files that are done and also the need to craft an Avisynth script to do the resizing.

There won't be any sort of program that does all this; that is what the bash script (in my example) is for. Basically, something like (this won't work out of the box, also there is the bit with archive flags to be done):

#!C:/cygwin/bin/bash

for filename in C:/captures/*.avi; do
    encodercommand $filename
    rm $filename;
done

So complicated, but not too complicated..

[PC Master]

Eh, I don't actually need to resize any video, so Avisynth should not be needed. I just want to recompress in high quality (possibly 100% quality) MPEG4, and maybe MPEG2, though that's looking like it won't be needed at all, and MPEG4 is less than half the file size for the same quality in most cases.

Okay, now I need a tool that can convert MPEG2 to AVI (probably XviD, maybe DivX) via CLI. Actually, it needs to support raw DVB streams too if at all possible.

In addition, with bash...
1) Can I run windows commands through that? For example, CCE.
2) How would I implement it so that the script waits until the encoding process finishes and exits before deleting the file?
3) Assuming a program is capable of returning an error code, how would you detect this in such a bash script? For example, if the encoding failed for some reason, I don't want the source to be deleted.
4) Is there a way to spawn a windows process (such as the encoder) at lowest priority from the command line?
5) To prevent multiple occurances of this script from running, I figured on creating a dummy file somewhere. The script would check for the file at startup and exit immediately if the file was found, or create the file if it didn't exist, and then delete it when all files have been processed. How might I implement such a thing?
6) Can you point me to a good bash script reference?


Thank you for your assistance with this.

[iantri]

Okay, now I need a tool that can convert MPEG2 to AVI (probably XviD, maybe DivX) via CLI. Actually, it needs to support raw DVB streams too if at all possible.

The aforementioned mencoder can read in MPEG-2. It also handles DVB streams.

In addition, with bash...
1) Can I run windows commands through that? For example, CCE.

Yes. Simply by specifying the command (just like in a Windows batch file). Ex: (note that you can leave the .exe off of commands.., paths are specified with unix-style forward slashes)

#c:/cygwin/bin/bash

for file in *.avi do
   c:/encoder/encoder -q 10 -t 5 $file
done

2) How would I implement it so that the script waits until the encoding process finishes and exits before deleting the file?

This happens naturally with any command-line program.

3) Assuming a program is capable of returning an error code, how would you detect this in such a bash script? For example, if the encoding failed for some reason, I don't want the source to be deleted.

The magic variable $? reveals the exit status (aka errorlevel in Windows). A simple if/else if statement can be then used to decide what to do. Check the cygwin docs to see how this works under Windows, though.

4) Is there a way to spawn a windows process (such as the encoder) at lowest priority from the command line?

nice -n 15 command

Under Windows, though, there are not multiple numbered levels, like in various Unices, just a few levels called "High", "Normal", "Low", etc. In cygwin, 0 = Normal, >0 = Low, <0 = High.

5) To prevent multiple occurances of this script from running, I figured on creating a dummy file somewhere. The script would check for the file at startup and exit immediately if the file was found, or create the file if it didn't exist, and then delete it when all files have been processed. How might I implement such a thing?

Sometyhing like:

if -e filename then
dostuff
fi

Or use the windows command "attrib" to play with the archive attribute (It is on by default when a file is created; disable to indicate that file is being processed, then have script delete when done unless error code), and have script only operate on files that have archive attribute set (might need to use 'awk' on the output of attrib command to figure out attributes). That way nothing bad happens if the script runs past the time it is supposed to start again.

[quote]
6) Can you point me to a good bash script reference?
[/code]

This is the granddaddy of them all. Also, consult the cygwin documentation for any cygwin-specific info.

[PC Master]

Thanks for your help. I've ended up writing a C++ program to do what I desire and am using VDub for the command line encoding.

[iantri]

Great. Had no idea that Virtualdub could operate on the command-line.. that's interesting.

[PC Master]

Yeah, actually, I'm using VirtualDubMod because it can read MPEG2 files. The help file for it lists all the command line options. You can even write powerful scripts to do things like resizing, etc.

What I ended up doing was opening VDubMod, selecting my encoding settings and such, and then have the program save a configuration file. Then you just specify that config on the command line along with the source file to encode (and an output file name), add the 'run jobs' and 'exit when done' switches, and watch it run.

I also ended up using the 'start' command to run VDubMod so that I could make it work in low priority and stay minimized.