This is a minefield! (converting mpeg to mpeg for DVD)

[uth]

I am trying to edit and convert some MPEGs that I got from my Tivo to DVD, but I feel like I'm crossing a minefield! I have successfully created DVDs from videos captured from a video4linux compatible card, so I am somewhat familiar with the process, but I can't believe how much trouble I'm having with these mpegs!

Everything I try seems to cause one of the following problems
1) Audio/Video ends up horribly out of sync
2) tools that are supposed to work end up crashing inexplicably, or error out.
3) dvdauthor rejects the mpegs I give it.
4) If I successfully cross the minefield and burn a DVD, it doesn't play right in my player.

The tools I have installed on my system:
Editors: lve, gopedit, avidemux2
conversion tools: ffmpeg, mplayer/mencoder, transcode, mjpegtools, dvdauthor

The Tivo mpegs are actually from Tivo2go, been run through a process to convert them to plain mpeg on Windows. When I start with them on Linux, they are 480x480, 4:3 aspect, ntsc with MP2 format audio. I need to edit them down, and convert them to 720x480 or 704x480 with ac3 audio (my player won't do mp2)

I've used all sorts of combinations of the tools above, but have yet to find one that gives all the desired results.

A few specific questions:

At one point, I had mpegs that I produced with ffmpeg and looked right, but dvdauthor rejected them. Apparently, dvdauthor likes to have things multiplexed with mplex, and not ffmpeg. So I figured I could demultiplex them, and remultiplex them with mplex. Easier said than done! For one, it's tough to find the right command line options that will demultiplex into m2v and ac3 files. I ended up using tcextract, but not thanks to the documentation (which is rather poor for most of these tools), I searched and searched until I found an example. Worse, when I finally got it done, mplex errored out trying to process them.

1) So what is the best way to demultiplex an mpeg using any of the tools I list above.

2) A lot of examples say to use mencoder or transcode. Both of these seem to make heavy use the ffmpeg library, libavc. Is there any reason I can't just use the ffmpeg command? It seems alot faster than transcode.

3) After trying and giving up on lve and avidemux2, I ended up using gopchop to edit down my mpeg. But I noticed that if I try to make an mpeg from more than one edit sequence, it seems to cause trouble for the other tools For example, if I start with a 20-minute mpeg, and use gopedit to make an mpeg that contains the first 5 minutes, and minutes 15-18, it causes sync problems and other trouble in the other tools. But If I make an mpeg from the first 5 minutes, and a second mpeg from minutes 15-18, then those seem to work Ok.

4) 352x240 is legal for NTSC DVDs, right? How come they never seem to play right on my (Toshiba SD1600) DVD player? Is there a certain way they need to be processed?

So if anyone can answer any of the above questions, provide insights on why I might be having the problems I am, or give suggestions on a reliable series of tools to edit and convert MPEGs to legal DVD format, preserving the sync, I'd appreciate it!

I can provide more detailed examples of what I tried on request. I've been doing trial and error for about two weeks now, so there's alot that I tried.

[oldcpu]

or give suggestions on a reliable series of tools to edit and convert MPEGs to legal DVD format, preserving the sync, I'd appreciate it!

Audio/video being desynched can be a real bear when it happens.

Did you try Audacity or ReZound to fix your audio? I also note on an AVIDEMUX forum that ProjectX was recommended to help fix audio desync.

From what I read you are using the correct apps. Its merely a matter of finding the best (possibly complex) way to put them all together.

You could try "tovid" to create your compatible DVD format, and if the audio is desync'd with the video, strip it out with a different tool, fix it with audacity (or similar package) and remux.

[uth]

I did read that note about ProjectX, and I even downloaded projectX, I just never installed it. Thing is, it didn't say how to use projectX to fix it (is it an automatic thing or a manual thing?)

I do have audicity installed too.. But wouldn't that involve trial and error to get it right? I'm hoping to build a streamlined process that can be mostly scripted. I will take a look at tovid. Maybe they already solved my problems.

I wish there was better documentation. When I'm done, I think I'll convert my notes into a guide and post it here.

I did find that ffmpeg has a (poorly documented) -async command that seems to do a pretty good job of fixing the sync, once you know what parameters to give it. Again, dvdauthor will reject ffmpeg mpegs that haven't been processed with mplex. Fortunately, you can make an m2v with ffmpeg, if you figure out the (again, badly documented) parameters.

I gather mpeg was never designed to be edited in the first place, and that's what leads to the sync issues. Maybe I should try dumping the tivo file first to mjpeg or something if I can, that may save lots of pain later on

[courtrrb]

Have you tried tovid to convert your videos? it a command line program nut it easy to use!

[oldcpu]

Have you tried tovid to convert your videos? it a command line program nut it easy to use!

There is also a tovid-gui under development, and available for use. The command line is more powerful than the gui, but the gui is even easier to use than the command-line, especially if one has many video's for inclusion in a single dvd, and also if one has a slightly more complex menu for the one new dvd.

[wmcbrine]

Tivo video is pretty crufty IME (but that was with DirecTivos). You might try some of the Tivo-specific tools, though the ones I'm familiar with are made to work with hacked Tivos rather than Tivo-to-Go.

Meanwhile, the simplest thing may be to buy a new DVD player. 352x240 is supposed to be valid for DVDs, but a lot of players don't support it anyway. Look for a player with VCD support; also look for SVCD support, which will cover 480x480. Finally, get a player that supports playback of burned MPEG files, so that you don't have to go to the trouble of authoring (sometimes not possible), and so that a much broader range of files will be playable. You can get all this in a $20 player -- sometimes more easily than getting it in a $100 player, at that.

BTW, tovid is nothing but a frontend. If you can't do something with mencoder or the mjpegtools, then you can't do it with tovid, either. tovid is only to simplify tasks.

[EwanG]

Or you can do what I just did, and go back to Windows

Was getting to be too much of a pain to have to move .tivo files to a machine where they could be converted to a regular .mpeg, and leaving them as .tivos meant I was wasting space to store commercials as well.

Add in that I never was able to find a good batch mpeg editor (similar to TMPGEnc 3.0 Xpress), let alone avoid the sync issues being mentioned here...

It's a conspiracy I tell ya

[oldcpu]

I did read that note about ProjectX, and I even downloaded projectX, I just never installed it. Thing is, it didn't say how to use projectX to fix it (is it an automatic thing or a manual thing?)

I downloaded and installed projectX, and I haven't a clue how to operate it. It appeared most the documentation is not in English. The English guide I did find, was for an older version, and not applicable to the current version.

Does anyone have a link to an English guide for ProjectX ?

[oldcpu]

I did read that note about ProjectX, and I even downloaded projectX, I just never installed it. Thing is, it didn't say how to use projectX to fix it (is it an automatic thing or a manual thing?)

I downloaded and installed projectX, and I haven't a clue how to operate it. It appeared most the documentation is not in English.

Well, I'm still stumbling around blindly on this. I did find that ProjectX can also be run from the command line. Bizarre as it may sound, I am finding for doing something simple, the command line is far less intimidating.

My goal is to take a .mpg file, and edit it between the key frames. Thus far I have succeeded in using ProjectX to demux a .mpg file. ... I'm hoping I can then take that output, together with a line number (for the cut) obtained from lve, and use m2vmp2cut (or avidemux) to cut the mpeg file where I want cut. .. AND keep audio synch. (As a note - I tried avidemux first on the original mpeg, but the file I am using has also given AVIDEMUX problems (with audio synch)).

At this point in time, that appears to be a rather "lofty" goal, and I'm not all that confident that I will succeed. Has anyone else tried and succeeded at this?

[oldcpu]

At this point in time, that appears to be a rather "lofty" goal, and I'm not all that confident that I will succeed. Has anyone else tried and succeeded at this?

Well, it turns out I was too pessimistic. With ProjectX, from an input .mpg file, I generated an output .m2v and .mp2 files. I used these two ProjectX files as an input to Avidemux. I was then able to edit/remux the audio & video files with avidemux, and this time retain audio/video synchronisation. My edit/cut was successfully made in between two keyframes (where as before, using only avidemux on the original .mpg file gave me an audio/video desync problem). Impressive!

[uth]

It's been awhile, but I did get this problem sorted out. As promised, here's a general guide that works for me. I would recommend using the tools listed first before experimenting, since I had many problems using other tools.

1. Download the video from the Tivo, this can be done with Tivo desktop, straight through a web browser, or with Galleon. I recommend using galleon since it has some nice features to automate this process. Downloading can be done under Windows or Linux, because of the amount of time it takes, I usually do it under Linux while working on other things.

Pitfall: TiVo files can be several Gb in size, some filesystems have a 2Gb file limit. My XP partiton was upgraded from Win98, so it's using FAT32. I was having problems with transfers just inexplicably hanging. I found out they were hitting the 2Gb limit. I created an NTFS partiton on an empty partition, and use that. I also found some Windows tools that can copy these files from Linux partitions, because Linux NTFS write support isn't 100% safe.

2. In Windows, convert the Tivo file to mpeg. I won't describe this convuluted process here, you can find the details on the net.

3. Back in Linux, I use gopchop to edit out the commercials. GOP-level editors are nice in that you don't need to reencode your edits. I save each segment to a separate mpeg, because I found if I combined them, I'd get sync issues later on. These separate files will simply become "chapters" on the dvd.

4. using mencoder, scale the video to DVD size (704:480, for NTSC), I also use the harddup option for better a/v sync. I also need to convert to audio to AC3, since my DVD player doesn't play the MP2 format Tivo uses (other players may)

Note:, I hope to combine 3 and 4, using mencoders .edl files to specify edits. So far I haven't been able to find a tool that lets you mark edits in an MPEG in a manner that can be accurately converted into an EDL.

5. Now I have mpg files that can be used with dvdauthor to create DVDs
[/url]

[oldcpu]

It's been awhile, but I did get this problem sorted out. As promised, here's a general guide that works for me.

Thanks for your guide and sharing your experience. Most appreciated!!

Some suggested things you may wish to consider:

dvbcut is very good at chopping pieces out of .mpg files AND retaining audio-video sync. It can also splice between key frames. It is a very simple interface/gui. While what it does is limited - what it does do, it does extremely well.

Reference your experince with NTFS, there is a new "beta" version driver for linux called "ntfs-3g" which is VERY reliable and fast for reading and writing from an ntfs partition as a regular user. Root permissions NOT required. I believe it is superior to all other linux ntfs drivers, including the commerical drivers. I highly recommend it. Basically, it means one no longer needs a vfat32 for transfer files from Linux to Windows and back. It means the old 2GBtye file limitation is now a thing of the past.

[uth]

dvbcut is very good at chopping pieces out of .mpg files AND retaining audio-video sync. It can also splice between key frames. It is a very simple interface/gui. While what it does is limited - what it does do, it does extremely well.

Can it save the segments without reencoding the video? That's the main reason I'm using gopchop. The video wil get resized and reencoded in mplayer, so I want to avoid transcoding more often than necessary

Reference your experince with NTFS, there is a new "beta" version driver for linux called "ntfs-3g" which is VERY reliable and fast for reading and writing from an ntfs partition as a regular user. Root permissions NOT required. I believe it is superior to all other linux ntfs drivers, including the commerical drivers. I highly recommend it. Basically, it means one no longer needs a vfat32 for transfer files from Linux to Windows and back. It means the old 2GBtye file limitation is now a thing of the past.

I'll have to check it out. My very first experience with NTFS write on Linux in like 1999 trashed my NT installation on the first write attempt. Seven years later most NTFS drivers still warn of corruption. It would be great to finally have one that works

[oldcpu]

Can it save the segments without reencoding the video?

yes.

My very first experience with NTFS write on Linux in like 1999 trashed my NT installation on the first write attempt. Seven years later most NTFS drivers still warn of corruption. It would be great to finally have one that works

Take a look at this URL:
http://sourceforge.net/mailarchive/forum.php?thread_id=23836054&forum_id=2697

The current version has been well tested for 32-bit. Equivalent testing has not been done on 64-bit, prompting many to state the 64-bit version doesn't exist. But this is not accurate, as I have read of 64-bit users using the ntfs-3g driver.

The 32-bit driver is purportedly very reliable, and very fast in the testing that has taken place to date.

There is a plan to integrate it into the linux-ntfs project: http://www.linux-ntfs.org/

[redtux]

I use ffmpeg. With mpegs I have never had any real problems (just video tapes but fixed now)
Do ffmpeg -i filename -target <pal/ntsc>-dvd newfilename.vob
Then dvdauthor will lap the files up

[uth]

I finally gave dvbcut a try this past week. It does what it's supposed to, it is nice in that it can reencode the missing frames in-between key frames, so you can cut on any frame, and not have to reencode your entire mpeg.

However, I found an editor I like better: gopdit. Every other mpeg-gop editor I tried have to index the mpeg the first time you try to edit it, a slow and disk consuming process. gopdit does not do this indexing, so it opens your mpeg right up, ready to edit! That alone makes it much faster. You still have to make your edits along keyframes, not usually a problem for me.

Also for my Tivo woes-- There is now a tivo decoder for linux! I think it's called tivodecode. No more booting into windoze to decrypt it, no more having to manually graph-out the filters in the graphedit gui, tivodecode can run as a batch process.

so now the process of archiving tivo recordings to DVD is almost automatic, I only have to intervene to edit out the commercials, and grab screen caps for the dvd menu... I have everything else scripted.
[/b]

[shirsch]

I've been beating a flat spot in my forehead trying to burn a DVD from converted TiVo programs. One of the later posts talks about using mencoder to set the scaling properly.

Can someone post the exact syntax for this command? No matter what I've tried it ends up "squeezed" horizontally. Many thanks!

[uth]

Here is the script I use. It contains two files that should go in the same directory. The mencoder.conf file is what you can edit to change settings like bitrate, scaling, interlacing, # of passes, etc. It even has a cooldown setting b/c running mencoder on a large file tends to make my cpu heat alarm go off on a warm day.

After mencoder.conf is editted to your liking, run ./tivo2dvd-share.sh <your mpg file>, it will show you your settings and ask you to confirm (the output file will be the same name as your input file with dvd_ prepended.)


mencoder_scripts.zip

[uth]

It looks like the latest Tivo update (9.1) broke things again, I am no longer able to process
files with gopdit or mencoder or the other tools I tried (that used to work), I am seeing 'invalid GOP' type errors.

This is the second time Tivo broke the TTG process in a year, but I think it might be deliberate this time.

I'll post a solution/workaround if/when I find one.

[shirsch]

That's odd! I just downloaded a show recorded on Oct. 15 (after the update) using a web browser pointing at https://<my_tivo_ip>. Ran it through the Linux tivodecode utility, then mastered a DVD using 'tovid' (runs ffmpeg underneath to do an audio+video conversion on the fly). No errors at any stage and it plays just fine! In general, I've been seeing much better results using ffmpeg. This is on a Ubuntu 7.04 (Feisty) system using the latest version of 'tovid' (0.31) downloaded as a tarball from its home site. All other tools are the current Ubuntu levels.

Were you seeing this after editing with gopchop, or on a new download?

[uth]

Hmmm, Did you edit your files? Or save them straight to DVD?

Seems that they play OK under Linux, but you can't edit them (at least not in any editor I tried). Also if you skip around the video with mplayer, it now gets confused. Something wrong with the indexing now, I guess.

I usually use gopdit to edit them, gopdit freezes when you save the file now. Other editors (kmenc, avidemux2) won't even load the files now.

I might have found a workaround, it seems I can run them through mencoder to change the format, I can then edit the resulting file. I can't do two-pass encoding on them because that crashes too!

If you have gopchop or gopdit installed, can you let me know if you can edit it? Or if you see GOP errors now?

[shirsch]

Hmmm, Did you edit your files? Or save them straight to DVD?

I have not tried editing anything since the new software came up.

I might have found a workaround, it seems I can run them through mencoder to change the format, I can then edit the resulting file. I can't do two-pass encoding on them because that crashes too!

Can you elaborate? What's the sequence you follow?

If you have gopchop or gopdit installed, can you let me know if you can edit it? Or if you see GOP errors now?

I'll give it a try this evening.

[uth]

I might have found a workaround, it seems I can run them through mencoder to change the format, I can then edit the resulting file. I can't do two-pass encoding on them because that crashes too!

Can you elaborate? What's the sequence you follow?

My original sequence is to edit with gopdit, then run it through mencoder to encoded it to DVD compliant format.

But apparently, even though you can't edit parts, you can run the whole file through mencoder. Now I am experimenting with running it through mencoder first, then editing the resulting file, It doesn't seem like dvdauthor likes this though.

The other possible sequence is to first run it through mencoder to convert to mpeg4, edit that, then run it through mencoder again to convert to DVD format. This is a real pain, it is very lossy, but it looks like the only way I've found to have a usable, edited show suitable for DVD backup.

[shirsch]

Have you tried running it through gop_fixup at any step along the way? It's intended to correct issues with gaps in the time code and may help.

I don't recall now where I found it, but Google should be able to tell you.

[uth]

Yes, I did try gop_fixup both before and after. It didn't fix the problems.

I got a valid DVD mpeg finally. My steps were:
1) convert to msmpeg AVI with high bitrate using mencoder (more editors can deal with AVIs better than mpeg)
2) open with avidemux2, click Audio->Build VBR Time Map (to prevent sync issues)
3) Make edits and save editted file to AVI using 'copy' as the audio and video codecs *
4) Run the editted AVI through my tivo2dvd script to get the usable mpeg.

* You can also save straight to mpeg format with avidemux2, if you set the filters, codecs and output settings correctly. I tried this, and it froze at 99%, so I'll stick with what works

[shirsch]

Would you mind terribly sharing the mencoder command line args used? I can almost follow your explanation, but there are so many variations and switches that I get dizzy thinking about it.

I've grabbed the Ubuntu avidemux package, so I'm set there.

Does the script you posted a while back convert AVI to dvd? If not, could you put up the latest? Many thanks!

[uth]

The script I posted awhile back will convert AVI (and just about anything else) to dvd.

on the mencoder options- I know what you mean, I get dizzy too! These are what I used to convert the tivo file to avi: (2 pass shown)

mencoder "$INFILE" -ovc lavc -lavcopts vcodec=msmpeg4:vbitrate=5000:vpass=1 -oac mp3lame -vf harddup -mc 1 -o $OUTFILE

mencoder "$INFILE" -ovc lavc -lavcopts vcodec=msmpeg4:vbitrate=5000:vpass=2 -oac mp3lame -vf harddup -mc 1 -o $OUTFILE

One other thing.. On a couple of my tivo files, they only partially converted to avi (maybe the first 45 minutes of a 60 minute program). To convert the rest, I had to resort to this even more painful procedure:

Open the tivo mpeg in avidemux2 trim out what you don't want, and save it as mpeg video (video only, because the audio will be out of sync- avidemux can't sync mpegs too well) Then open the file again in avidemux, and clip out the audio portion you need (doesn't have to be exact, yet). Export it.

Open the audio file in audacity, clip off any extra bits you don't need, save it, convert it to ac3 with ffmpeg, and mplex it with the video you saved. Watch the video to check the sync, if it's off, edit the audio some more and repeat until you have the sync acceptable.

I made the videos I edited this way into chapters. You could convert the entire video this way, but you will probably end up wanting to shoot yourself in the head just to end the agony. (the audio may be in sync in the beginning but not in the end -- it's best to use small chunks, like between commercial breaks. Of course YMMV)

The good news is that the majority of my videos didn't have this problem. The bad news is by majority, I mean 3 out of 5

[ragtime]

Sounds like a very complicated procedure...
Are you sure DVBcut does not accept the TiVo Files directly? I'm not familar with TiVo at all,... but does this box not also just save DVB Transport Streams like any other PVR out there?

Usually, I only need DVBcut + dvdauthor to bring my DVB recordings to a DVD!
OK,... for some recordings which have a not-DVD-compliant resolution (for instance MTV) I have to do a
"ffmpeg -i input.mpg -target DVD -acodec copy output.mpg" after cutting / before authoring... but that's all. An additional (lossy!) conversion to MPEG4/AVI inbetween is neither necessary nor acceptable!

[Marvingj]

Good Post....

[uth]

Sounds like a very complicated procedure...
Are you sure DVBcut does not accept the TiVo Files directly? I'm not familar with TiVo at all,... but does this box not also just save DVB Transport Streams like any other PVR out there?

Usually, I only need DVBcut + dvdauthor to bring my DVB recordings to a DVD!

I did a quick test with dvbcut. It too reports alot of errors while indexing, but it did not have the problems of losing its place like gopdit and others did. However, when I tried exporting the video, it didn't seem to keep the sound (is this normal?)

The tivo provides mpeg program streams, they are encrypted, and need to be decrypted before doing anything with them. Plus I've discovered that not all DVD players will accept the raw Tivo, so that's why I reencode it at least once, to get it in a acceptable format.

OK,... for some recordings which have a not-DVD-compliant resolution (for instance MTV) I have to do a
"ffmpeg -i input.mpg -target DVD -acodec copy output.mpg" after cutting / before authoring... but that's all. An additional (lossy!) conversion to MPEG4/AVI inbetween is neither necessary nor acceptable!

I agree with it not being acceptable. So far, since the tivo software upgrade, it's been necessary. I'll try playing with dvbcut some more to see if it can provide a workaround. Thanks for the hint on that.