Archiving DVDs: What approach would you suggest (apart from ISOs)?

[fellowweb]

Hi there,

This being my first post in this forum, I have naturally used the search function and Google. However apart form two posts I have not been able to find something close to my question:

Using DVD Shrink, DVD Decrypter or DVDfab, I have archived a bunch of DVDs as ISOs not using any of these tools' offered compression capabilities. Usually, I have the main film, all extras, and most of the languages and subtitles.

As a second step, I am now looking for a decent way to reduce the memory footprint of this solution.

This is where I would highly appreciate your help and opinion.

Once, I saw RatDVD. It pretty much does what I am looking for: Keeping a DVD-like backup (incl. menu, extras, langauges and subtitle). However - as far as I know - there is no ungoing development of RatDVD anymore and its codec (and as such quality/file size trade-off) is inferiour to more recent solutions.

The DivX codec seems to do well, however it doesn't seem to be that appropriate for what I would like to achieve - as I could read here.

Matruska (MKV) seems to have the same problem.

In essence, I am looking for the functionality of a simple ISO of the entire DVD (incl. menus, extras, subtitles and languages - as I get by using DVD Shrink) supplemented by the compression quality of a modern high-quality codec.

What would you suggest?[/url]

PS: The ability to restore the PC file to a DVD does not matter to me.

[neomaine]

fellowweb,

I think you're asking for too much. You've got three real options:

- Extract and re-encode the VOBs from your DVD ISOs and find some way to get your menu/extra functionality in an MKV or DIVX (Ultra) container.

- Extract and re-encode the VOBs retaining the MPEG2 encoding but use a smaller bitrate and rebuild the ISOs.

- Get a bigger or another harddrive.

With all due respect, and I do mean that, get another hard drive. It let's you keep 100% functionality of the menus/extras with 100% of the original quality....its not nearly as fun, but, its waaaaaaaaaaay easier.

[victoriabears]

The hard drive solution is disliked by me for one simple reason, you archive, say 500 dvd's on it, the hard drive goes bad, you've lost all the data.

If you back up the discs to blank dvd's using imgburn then if the disc goes bad, you've only lost that one.

Store all media in a cool room away from direct sunlight, and with luck and good Media, Verbatim or taiyo yuden, all should be good.

This is a realtively automated solution and is far faster than converting of any kind.

Why convert data? If there is a reason then fine but not for backup

[neomaine]

The ONLY way to change the size is to convert/reencode. The formula is simple: time * bitrate = size

If keeping them on DVDR is your solution, then you already have a good tool. DVD Shrink. (Well, techinically it transcodes...) It will be the easist way to keep everything you're looking for.

Besides, why would you like to reduce the size if they're already on DVDR? Or, are they currently on your harddrive as ISO's and you're looking to shrink them to fit on single layer (DVD5) disks?

However, you say 'archiving' which implies that these are copies. If you should loose a copy, you always have the original, right?

[tomwil]

Quick question -- If a Video DVD was extracted onto the hard drive (which will probably be over 5GB), made into an ISO, and then the ISO compressed using Winzip, could it fit onto a 5GB DVD-R for archival purposes?

The thinking is that whenever dual-layer and/or Blu-ray gets less expensive, one could unarchive that compressed ISO and burn onto newer media without losing quality.

[victoriabears]

I have never had any success using zipping for any format, has anyone?

With blank media so inexpensive, even dual layer really, why fart around?

[stiltman]

Quick question -- If a Video DVD was extracted onto the hard drive (which will probably be over 5GB), made into an ISO, and then the ISO compressed using Winzip, could it fit onto a 5GB DVD-R for archival purposes?

The thinking is that whenever dual-layer and/or Blu-ray gets less expensive, one could unarchive that compressed ISO and burn onto newer media without losing quality.

Think of a DVD that contains VOBs which in turn are MPEG2s as a zipped file for video.

You'll be lucky to reduce the file size by 10%

[fellowweb]

Thank you all for answering so quickly and in-depth!

@neomaine:
Option 1: MKV or DivX container incl. menu funktionality
This sounds pretty good. However as mentioned in the two articles I listed, this still seems to be pretty difficult.

Option 2: Reducing VOBs' bitrate
This catalogue of DVD ISOs should retain a pretty high quality. Could this really be achieved by this solutions?

Option 3: Bigger HDD
Currently I have an external HDD with 1.5 TB. So, I think I am fine.

@victoriabears & neomaine
You are asking why I would like to convert for backup reasons. And this is something I might not have been clear enough about: The HDD currently storing my ISOs (each with 5-9 GB of uncompressed/unconverted content depending on the original DVD's size) should sooner or later work as the sole storage for films. Using some kind of multimedia hub some months in the future (think of an HTPC connected to a HD-TV maybe together with MythTV or whatever) the films should still have a reasonably high quality (video as well as audio) for an HD-TV and a decent home cinema setup.

As soon as this HTPC setting has been realized, I would prefer to rip (and convert) each newly bought DVD, to put the video file on the HTPC (or the attached external 1.5 TB drive, respectively), and to pack all DVDs in a box kept in the basement. Then, essentially, the DVDs serve as the real backups for the video content stored somewhere where I do not need to see them.

However, having seen the high quality of Matruska encoded videos, I have started doubting that keeping the ISOs is such an efficient solution for this. The loss of quality with DivXs or MKVs seems to be pretty low compared to their remarkable loss in size (750 MB to 1.5 GB).

It is not only about the question how many films I can put on my 1.5 TB HDD (sure much more than enough for me). But such file sizes simply are much easier to handle: Think of taking three to eight films on a single 8 GB USB stick to a friend for an evening. But with FAT file system on the stick, I could run into difficulties to even put one larger ISO on it.

As a consequence - having heard of RatDVD - I thought, someone might have come up with a nice solution to effectively retain everything just as I do with a full DVD-ISO while at the same time replacing the (pretty old) MPEG2 codec by a more current (more efficient) one. And my understanding has been that this is exactly what RatDVD achieved. However the codec used by RatDVD seemed to be a bit low profile to me.

@stiltman:
In essence, this is what my (far from knowledgable) thought comes down to: Wouldn't it - for my purpose - be more efficient to find a solution that (a) retains all information but (b) replaces the VOBs' MPEG2 codec by some more advanced (newer) codec whose compression algorithm provides for a better quality/file size ratio.

Whereas tomwil wants to reduce filesize by zipping MPEG2 (where he is probably pretty much out of luck), I would like to reduce the MPEGs' filesize by reencode it with a superior codec (if there is any - maybe DivX, MKV or whatever). Does this make sense?


Apart from that: Please apologize that I am always writing these long posts. It's something I have to work on.

[stiltman]

You also forgot to take into account cost of storage vs cost of your time to reduce the file size.

[fellowweb]

@stiltman
You are entirely right. On an abstract level, my thought is pretty easy: It could be some kind of "one-click-solution" comparable to DVD Shrink, which maybe - using batch processing - could do pretty much everything and with every ISO already available in one folder (say overnight or over several nights ) alone.
I don't want to reauthor a lot - maybe deleting some audio and subtitle languages - that's it. In case, there is a solution for my needs, it could be only little time consuming.

[ntscuser]

DVD Shrink for speed.
DVD Rebuilder for quality.

[neomaine]

fellowweb,

Getting your movies to another container to lower the size with a codec that can minimize the quality loss at a lower bitrate in order to reduce size isn't a problem. Plenty of one-click solutions for that.

The issue is your requirement of maintaining menus and extras. That can only be achieved by keeping it in its ISO container.

If you're willing to forego that requirement, you can pound away at the movie and most extras one at a time:

- SomeMovie.avi (or .mkv)
- SomeMovieExtras1.avi
- SomeMovieExtras2.avi

...and so forth keeping them in whatever folder structure is supported by your future HTPC needs.

I think it comes down to this for you: Is it more important to keep menus/subtitles/languages/extras with 0 quality loss or more important to shrink with minimal quality loss to another codec?

Personally, I reencode the movie to divx home-theater specs at constat quality of 2 (very very minimal loss...) and go back to the original DVDs if I want the extras...which is almost never.

[fellowweb]

Thank you both for your helpful remarks!

@ntscuser:

I am familiar with DVD Shrink as I have currently used it in order to come up with the ISOs. I have looked up DVD Rebuilder. But I am afraid I do not understand what it does.

As far as I understood, it remains the DVD format, however, it uses a better compression algorithm than DVD Shrink!? (And since I have not used DVD Shrink's "shrinking" feature, I have not had any loss in quality yet.) Is this correct or am I wrong?

@neomaine:

The issue is your requirement of maintaining menus and extras. That can only be achieved by keeping it in its ISO container.

Okay, sad to hear this so clearly. But still it's good to hear this as a definitive statement. Then I don't need to search further for something that doesn't exist.

If you're willing to forego that requirement, you can pound away at the movie and most extras one at a time:

- SomeMovie.avi (or .mkv)
- SomeMovieExtras1.avi
- SomeMovieExtras2.avi

...and so forth keeping them in whatever folder structure is supported by your future HTPC needs.

I think it comes down to this for you: Is it more important to keep menus/subtitles/languages/extras with 0 quality loss or more important to shrink with minimal quality loss to another codec?

Good, then this might be the best alternative for me. If one is honest, usually you would have to admit that you watch the movie but do not care equally about the extras.

However, as far as I know, some codecs seem to support the integration of more than one language (say English and German audio in my case) as well as subtitles (English and German would be sufficient for my case as well). I think I saw this feature when using HandBrake. Is this correct?

Are you aware of a fairly simple (but high quality) program (with a low complexity such as DVD Shrink) which could automatically extract the main movie and extras as separate files according to the structure suggested by you? Telling me the name would be sufficient. Then I could try to find some further information here in the forum.

Personally, I reencode the movie to divx home-theater specs at constat quality of 2 (very very minimal loss...) and go back to the original DVDs if I want the extras...which is almost never.

This sounds pretty good. Which program have you been using most of the time for this purpose?

Why have you decided for DivX and against say XVid or Matruska?

[fellowweb]

Why have you decided for DivX and against say XVid or Matruska?

Okay, probably you will have to correct me here. MKV ist a container wheres DivX and XVid are codecs, aren't they? So let's say:

Why have you decided for DivX and against say XVid or x264?

What type of container have you decided for?

Thanks again!

[lordsmurf]

I vote hard drive.

Still worried? I vote RAID 10 external array.

Still worried? I vote several RAID 10 arrays, each stored in a different location.

Still worried? See a therapist. You need meds. You worry too much.

[guns1inger]

.Divx is a container based on the AVI container but extended to support Divx encoded video with menus, subtitles and chapter support. These extended features are viewable with t either the Divx Player, or Divx Ultra Certified hardware. Other media and hardware players will ignore the extended features and only play the video (note : some non-Divx certified players that happily play Divx encoded videos in the AVI container will not see the .Divx extension, and so wont play videos in the .Divx container at all)

[ntscuser]

@ntscuser:

I am familiar with DVD Shrink as I have currently used it in order to come up with the ISOs. I have looked up DVD Rebuilder. But I am afraid I do not understand what it does.

As far as I understood, it remains the DVD format, however, it uses a better compression algorithm than DVD Shrink!? (And since I have not used DVD Shrink's "shrinking" feature, I have not had any loss in quality yet.) Is this correct or am I wrong?

You are correct. You could also use VobBlanker to remove the forced previews and other junk which is totally unrelated to the main title, saving space without sacrificing quality or functionality.

[fellowweb]

@lordsmurf:
Maybe I misunderstand but how is this post related to our thread?

@guns1inger:
Great, thank you for the elaboration. So I have some (limited) backward compatibility with a DivX container, which sounds good for my purposes. Are you aware of a simple software which would convert my DVD-ISO to a DivX container retaining decent video and audio quality and (to the extent it is possible) subtitles and chapters?

@ntscuser:
This sounds like a good alternative to the conversion to codec XYZ approach. However, having used the compression offered by DVD Shrink some time ago, I was pretty much unimpressed by the significant loss in quality.

When you use DVD Rebuilder for a DVD with approx. 8 GB original file size what file size do you end up with in the end? Is the loss in quality easily recognizable for the eye?

Thank you also for the reference to VobBlanker. Sounds good.[/b]

[ntscuser]

@ntscuser:When you use DVD Rebuilder for a DVD with approx. 8 GB original file size what file size do you end up with in the end?

Normally 4.7 or 4.43GB, depending on how you measure it. Either way it's the same size as a single layer DVD.

Is the loss in quality easily recognizable for the eye?

It depends on the orignal content and number of passes used to encode. A two-hour plus movie with fast moving detail and multiple soundtracks will require more passes than shorter movie with single audio track. The average viewer couldn't tell the difference. The person doing the encoding just about could if he knew where the tricky bits to encode were. The trade-off is that using many passes in DVD Rebuilder will take much longer than any of the quality settings in DVD Shrink.

[MOVIEGEEK]

fellowweb,
The best solution is to use ISO or DVD folders and store them on a HDD,the cost/benefit ratio is better than re-encoding to DivX,etc.Yes HDD will fail but it could be 5,10 or 15 years from now.It's best to keep your media on at least two storage mediums just in case.
I have experimented with compression utilities and found 7-Zip to be the best,using 7z will reduce the size by ~40%.

[guns1inger]

There are several apps that will compress DVDs using Divx compression, but I don't know of any that will retain menus and chapters. A few will hard encode subtitles. If you want to have menus and soft subs etc in the Divx container you will have to re-create it all yourself.

There was, a while back, and alternative to RatDVD which claimed to do a similar job without the issues that RatDVD had. You had to use their player in order to make use of these features. I cannot recall the name now, but perhaps someone else remembers it.

DVD Rebuilder is basically a super shrink. Instead of transcoding to reduce size, it re-encodes using one of several quality encoders. The fully free version uses HCEnc, which is very good, but you can also use CCE or ProCoder if you own them. Like Shrink, it aims to reduce the size down to that of a single layer DVD, however the quality is far superior to Shrink if a large reduction is required.

Personally, I don't understand the love for the menu. If you are planning to store these on some form of media hub down the track, then chances are you will be playing them through some form of interface that will server a similar purpose. Buy using something like Microsoft Home Server, or a home built box that supports RAID, you can minimise risks of HDD failure as well. All of this means that menus will be largely superfluous, and opens the way for using a better compression codec such as H/264 instead of Divx.

[Nelson37]

It's a sliding scale. Smaller file size with hi-quality means long encoding times and high hardware requirements for playback. Less size reduction and/or less quality means shorter encoding times and less hardware requirement.

Shrink or rebuilder keep all the extras, H264 will give better quality in smaller size but few bells and whistles. Xvid a good compromise, but probably going to become obsolete.

Decide between Wants and Needs. Hi quality #1? Extra audio channel sounds like a need. Subtitles? Menus would be at the absolute bottom of my list, in fact I would do Extra Work to get rid of them. To each his own.

SFAIK subtitles can be done with a 264 file in some containers, not sure about alternate audio but I think so. If you could get those two things in one-quarter the size with the same quality, would that be sufficient?

Most likely a significant percentage of other people's PC's would NOT be able to play back these files without problems. That will change over the next year or two.

I am in the process of converting my captured library to H264. On my second fastest PC, a 2-hour HD Documentary is gonna take a full day to convert, and will not play at all on my two older PC's. But, storage space for future needs dictates a major size reduction.

[fellowweb]

Again, everyone, I am impressed and want to thank you again by all your helpful and detailed remarks. This is by far the best forum experience I have had so far! I really appreciate your time spent on helping me!

@ntscuser:
DVD Rebuilder seems to be very impressive. Since you mentioned the multiple passes, how long does it usually take to rip a DVD with DVD Rebuilder (just as a proxy)?

@MOVIEGEEK:
I see your point. In addition, this is what I wanted to do initially and why I got this 1.5 TB HDD. Still this drives would be quite sufficient for me. However, using DVD Shrink without any compression (maybe cutting out Turkish, Spanish, French etc. audio and subtitles but keeping the English an German audio/subtitle) I simply come up with files that are too big to handle. It does make a profound difference to end up with files 1-1.5 GB or 4 GB and more.

@guns1inger:
Thank you for the detailed and technical explanation about DVD Rebuilder. I think I understand now.

If you remember the name of the RatDVD alternative, please let me now. However, RatDVD on the one hand seems to be a kind of outdated technology and on the other hand not a widely supported standard. Although, it sounds pretty convenient for my aim, I see a certain risk of ending up with a movie library which I won't be able to easily play back after a couple of years (due to a general lack of support of this format).

Regarding your point against the menu:

I have thought this over today. After this discussion with you all, it sound much more important to end up with a movie file whose video and audio is of high quality and of a decent size (probably up to 2 GB - in case of DVD Rebuilder even 4.5 GB) and whose codec(s) will be supported for a longer time.

Then - with reference to neomaine' suggestion - I would end up with several files for each movie named XYZ:
- XYZ - Main Movie.???
- XYZ - Extra #1.???
- XYZ - Extra #2.???
- ...

For each file I would like to have:
- English audio
- German audio
- English subtitles (preferably not hard encoded; being a German native speaker, I shouldn't really need the subs here )

Just a short question in between: Hard encoded subtitles probably means that they are shown all the time (technically speaking, they are probably "written" on top of the video sequence). Whereas soft subtitles can be switched on and off (kind of a layer on top of the video). Is this understanding right?

With splitting main movie and extras as well as with only keeping two audio streams and one (soft) subtitle, I should end up with a pretty standard approach to converting a DVD movie collection to a harddrive, shouldn't I?

@Nelson37:
As I have written above, as a consequence of the comments of you all, I will either go with DVD Rebuilder or use a scheme as the one described just above (several video files for main movie/extras, 2x audio, 1x soft sub).

For the second option, would x264 be the generally accepted way to go if
(a) you want to retain a quality sufficient for playback in a decent home cinema environment (HD device and multiple loudspeakers) and
(b) the underlying codec should still be supported some time in the future?

You say that Xvid might become obsolete. Is x264 its successor?


Since I maybe shift to an entirely different approach, I will have a look around VideoHelp in order to get more acquainted with recent containers, codecs etc. If you are aware of a good summary/ overview on this topic, please let me know!

Thanks again!

[ntscuser]

@ntscuser:DVD Rebuilder seems to be very impressive. Since you mentioned the multiple passes, how long does it usually take to rip a DVD with DVD Rebuilder (just as a proxy)?

It takes me eleven hours to re-encode a movie with all extras from start to finish using a 3.2GHz dual-core and ten passes but then I'm a perfectionist and it's often a case of overkill. I leave the PC running when I go to bed so it doesn't really matter to me how long it takes. Television episode compilations take a bit longer as the video is often interlaced.

[guns1inger]

3 passes on CCE is more than enough, 5 if you are a perfectionist. The difference between 5 passes and 10 passes is not visible to the naked eye and can only be seen by doing bit by bit comparisons. HCEnc only does two passes anyway

[PuzZLeR]

I vote hard drive.

Not what many like to see/read, including me, but it seems to me this solution would solve about 50% of the problems in these types of forums... :P

[Nelson37]

H264 is what BluRay uses, mostly, and is definitely the best available right now.

Staying with MPG2 will not allow for any really significant size reduction.

When you also look at doing HD files, MPG is just not practical anymore.

Xvid/Divx came close, even got some limited hardware support. Close, but no cigar.

You can encode a visually identical h264 from a DVD and get well under half the size, with files standardized to play on close to a million hardware playback devices. Software tools are still a bit light but getting there rapidly.

Adding more drive space will happen anyway, you will fill it faster than you can keep adding drives over the long run. More efficient compression is Always necessary.

And yes, in 3 to 5 years we'll be doing it all over again with the new Super Whizzo 3000. We'll need something to handle the 3D holographic display.

[PuzZLeR]

You can encode a visually identical h264 from a DVD and get well under half the size

So the theory states. But we are nowhere near there today unless you're willing to pay mega$$$.

Even when encoding from the acquisition source itself this theory is challenged with implementations currently available. Encoding directly from the MPEG-2 source? More unlikely.

You were more accurate here:

Software tools are still a bit light but getting there rapidly.

Without better support for interlaced content, and optimization tweaks, among others, formats like x264, and others, will be very far away from producing professional quality encodes.

Adding more drive space will happen anyway, you will fill it faster than you can keep adding drives over the long run. More efficient compression is Always necessary.

I agree totally and that's what the cheaper/free H.264 implementations are optimized for - producing watchable content at crushingly low bitrates. I believe x264 is by far and away the best at this today.

However, this does not mean great quality, only something that outdoes DivX/Xvid/MPEG-2 at low, low bitrates. The curve is not linear. Beef up the bitrate/quality and x264 quickly loses its compression advantage.

And yes, in 3 to 5 years we'll be doing it all over again with the new Super Whizzo 3000. We'll need something to handle the 3D holographic display.

What are you talking about? I thought the next thing would be H.269 (for Hyper) and Violet Blaster Super Ray!