Help converting AVI to MPEG2 with NO QUALITY LOSS (source file provided)

[Bobbo01]

Hi all,

Although I know quite a bit about digital video procesing, I still consider myself a newbie since it is such a vast field of information

I have provided a link to a 51 second AVI clip. I would like to challenge someone to convert it to DVD compatible MPEG-2 and retain as much fo the source quality as possible. File size is not an issue, quality is what is important to me.

http://www.filefactory.com/file/7gt7dto4ki9z/n/InternetAviUpload.avi

I know, there are annoying popups, and you must wait before the link becomes available - I assure you this is NOT a virus!! If it helps, I will pay to have my account upgraded so the direct link becomes available.

This clip is from a group of files that I created over 10 years ago, from Studio DC 10+ equipment. The video codecs used for capturing, and converting were the MJPG codec. I use Cinema Craft Encoder 2.70 and with alot of research, I and have only been able to achieve 'okay' results. I would grade it as 5/10.


Analogue source material: combination of VHS, HI8, & Digital8
Capture equipment: Studio DC10+ (back in 2001)
Downmix equipment: Also Studio DC10+

I am looking for an expert who truly knows video processing/conversion inside/out who can:

A) analyze this file and tell me EXACTLY what the details are (what are the codec details and should I convert it to something else?) I have other (source material) files on my PC that look to be this exact codec, but they won't play back, yet this one will - why?

B) MOST IMPORTANTLY - convert it to MPEG-2 and retain 100% quality if at all possible (preferably with CCE or TMPGENC), and explain to me EXACTLY what settings were used to get it there. (Once in .mpv format I can finish the job).

Thanks all! Look forward to chatting about this. I have several hours of of material where this clip came from that I would like to output to DVD.

Bob

[Cornucopia]

File size is not an issue...

and

... I would like to output to DVD

are mutually exclusive, as DVD ALWAYS has a size & bitrate limitation.

Which one has priority?

Scott

BTW, if FS is truly not an issue, use CQ/CRF encoding at best quality setting using HCEnc/FreeEnc/QuEnc. Or just go to h.264 MP4 with Handbrake or similar and play on media player.
If FS needs to be within DVD limits, you WILL have quality losses, but doing 2pass VBR with HCEnc/FreeEnc/QuEnc will minimize the loss.

[Bobbo01]

Hi Scott,

Thanks for your reply.

I am approaching this in two ways:

I want to determine the highest MPEG-2 quality I can convert this to. Once achieved, I will play with settings I am familiar with to deplete file size for when I output to DVD.

So for now, priority focus is highest quality possible - I don't care about filesize.

I am not familiar with "HCEnc/FreeEnc/QuEnc or h.264 MP4 with Handbrake" - what is that?

Thanks
Bob

[Bobbo01]

Again, my goal here: I want to challenge someone skilled in video conversion to convert that file with minimal quality loss and share it with me, along with the settings used to get it there -)

[davexnet]

Seems like a somewhat difficult encode because it's quite grainy.
There's only so much you can do with the settings of the mpeg-2 decoder.
For example, in Hcenc, you can play with AQ to see if the allocation of bits helps at all.

Seems like there are much more options in the CCE you mentioned.
I tried this years ago and always had best result in the default PQ settings (natural picture).

What is wrong with the mpeg-2 you created (that you're not happy with).
Could you post a sample of that too so we can see the issue you're referring to,
and something to compare.

[manono]

What a crappy and slow download site. That's the best you could do? Is there something wrong with Sendspace or MediaFire?

[Bobbo01]

What a crappy and slow download site. That's the best you could do? Is there something wrong with Sendspace or MediaFire?

I did not know about those...I don't post stuff alot, , I'll upload it there! Thanks for taking the time to check it out.

[Bobbo01]

Seems like a somewhat difficult encode because it's quite grainy.
There's only so much you can do with the settings of the mpeg-2 decoder.
For example, in Hcenc, you can play with AQ to see if the allocation of bits helps at all.

Seems like there are much more options in the CCE you mentioned.
I tried this years ago and always had best result in the default PQ settings (natural picture).

What is wrong with the mpeg-2 you created (that you're not happy with).
Could you post a sample of that too so we can see the issue you're referring to,
and something to compare.

I know - garbage in = garbage out - there's no way around that...I get it.

But one thing I am using as a constant is the titles and sub-titles... they look terrible in the MPEG-2 output - pixalized and have lines through them. These *SHOULD* look great (I would think). Let's start with that...

[jagabo]

Your request doesn't make sense. The highest quality MPEG 2 settings would be far from DVD compatible. You won't learn much from that. Your source is MJPEG, 29.97 fps interlaced, top field first. But the chroma channels are screwed up from prior mishandling. The colors of the two fields are co-mingled. There's nothing you can do about that now.

In TMPGEnc start with the DVD preset. Open the AVI file. Make sure it detects it as interlaced, top field first (if not, force it to that). Select the High Profiles @ High Level. Set Constant Quality mode with a quality of 100 and the max bitrate to 60000. Set the GOP structure to I frames only. Set DC Component Precision to 10 bit. Motion Search Precision to Highest Quality. You'll hardly see any difference between the resulting MPEG 2 video and the MJPEG source. The average bitrate will be over 20000 kbps and the peak bitrate will be nearly 30000 kbps.

[Bobbo01]

Your request doesn't make sense. The highest quality MPEG 2 settings would be far from DVD compatible. You won't learn much from that. Your source is MJPEG, 29.97 fps interlaced, top field first. But the chroma channels are screwed up from prior mishandling. The colors of the two fields are co-mingled. There's nothing you can do about that now.

In TMPGEnc start with the DVD preset. Open the AVI file. Make sure it detects it as interlaced, top field first (if not, force it to that). Select the High Profiles @ High Level. Set Constant Quality mode with a quality of 100 and the max bitrate to 60000. Set the GOP structure to I frames only. Set DC Component Precision to 10 bit. Motion Search Precision to Highest Quality. You'll hardly see any difference between the resulting MPEG 2 video and the MJPEG source. The average bitrate will be over 20000 kbps and the peak bitrate will be nearly 30000 kbps.

THanks for your help. I want to work backwards from perfect quality and figure out myself how to shrink the file....but I need to start with perfect quality first.

Couple of questins:
1. How are you setting a max bitrate to 60,000? TMPGENC won't allow that, and, I thought DVD standard was 9800?
2. I don't have a "High Profiles @ High Level" profile in TMPGENC. Where can I find this?

[_Al_]

So you want to encode only for DVD. You confuse everybody, because it was not clear that you want exclusively to encode for DVD.

If you want to test max quality for DVD, you just encode CBR (constant BitRate) 8500kbps. Crank some quality levels within encoder, whatever they are, to maximum. That will give you about 1hour 10min + 224kbps for audio.
There is nothing much you can do otherwise.

Then you lower your quality just to encoding your DVD with 2pass VBR with average set lower then 8500kbps, lower you go with your average bitrate, longer the video could be ...Use bitrate calculator. Max bitrate you keep about 8500

[vhelp]

what software did you use to reconvert back to mjpeg with ?

i ask because the studio dc10plus card is 640x480 NTSC resoution, and 720x576 PAL resolution, since the clip you uploaded is 720x480. i also saw some dropped frames.

[jagabo]

1. How are you setting a max bitrate to 60,000? TMPGENC won't allow that

I'm using the old TMPGEnc Plus 2.5.

and, I thought DVD standard was 9800?

You said you wanted the maximum MPEG 2 quality, not the maximum DVD quality. DVD is a very small subset of MPEG 2. You won't get perfect results with the max MPEG 2 quality that's DVD compatible. It will be full of macroblocks.

2. I don't have a "High Profiles @ High Level" profile in TMPGENC. Where can I find this?

If you're using a newer version of TMPGEnc, I don't know. In TMPGEnc Plus 2.5:



You might be able to trade off small details and sharpness to get fewer macroblocks by using custom matrices. Also consider reducing the frame size to half D1 (352x480). Some heavy noise filtering will help reduce the bitrate requirement too.

[Bobbo01]

what software did you use to reconvert back to mjpeg with ?

i ask because the studio dc10plus card is 640x480 NTSC resoution, and 720x576 PAL resolution, since the clip you uploaded is 720x480. i also saw some dropped frames.

I originally captured the raw footage back in the year 2000 with Studio DC10+ with some custom resolution of 608x464 (i learned later that this was an error). I then went and down-mixed it the project into a single file that was the same resolution.

Years later, in 2005, I went back and re-edited it (and re-captured some footage at the correct resolution of 640x480, but still with the Studio DC10+ card), and then I downmixed the entire project again to 720x480 with Studio 9, as DV had become a big thing.

What you see now is from that 2nd master down-mix. It's a combination of several types of source material (hi8, dig8 and VHS) captured in several different ways, but ultimately downmixed into one 720x480 project). Kind of a mess, I know!

Truth be told, had I known what I know now, I would have never started out with Pinnacle Studio. The dropped frames are due to a bug in the original studio dc10+ software - the card didn't work well with bad source material.

[Bobbo01]

1. How are you setting a max bitrate to 60,000? TMPGENC won't allow that

I'm using the old TMPGEnc Plus 2.5.

and, I thought DVD standard was 9800?

You said you wanted the maximum MPEG 2 quality, not the maximum DVD quality. DVD is a very small subset of MPEG 2. You won't get perfect results with the max MPEG 2 quality that's DVD compatible. It will be full of macroblocks.

2. I don't have a "High Profiles @ High Level" profile in TMPGENC. Where can I find this?

If you're using a newer version of TMPGEnc, I don't know. In TMPGEnc Plus 2.5:


[Attachment 14546 - Click to enlarge]

You might be able to trade off small details and sharpness to get fewer macroblocks by using custom matrices. Also consider reducing the frame size to half D1 (352x480). Some heavy noise filtering will help reduce the bitrate requirement too.

Thanks for your help. I was not using the "plus" version. I have located and installed that and now all the issues I previously had have gone away. I am able to set a 60K bit rate and have High Profile available.

However, I am still seeing the same results I have always seen. It doesn't seem to matter if I use 60000 or 6000, it looks the same. I can tell right away at the opening title sequence that it doesn't match the source. I changed only the settings you have told me to change, but I get this as my output:

Notice the difference in the text from the original, vs the mpeg2. How can I make that match the original? I'm sure cleaning that up will have positive affects all over the video.

Also, why is the size different? I specified automatic (720x480)

Thanks
Bob

[jagabo]

Your problem has nothing to do with MPEG 2 encoding. You need to change the settings in your MJPEG decoder. Turn off the "swap fields" option. Or, if it's already off, turn it on.

The size is different because the program you're using to display the MPG file is resizing to adjust for the display aspect ratio (4:3). It's not adjusting the MJPEG AVI (3:2 frame size) because there's no AR information there.

[Bobbo01]

Going through VirtualDub, I was able to find the settings for my MainConcept MJPEG codec:

Do I just change "invert fields" at the bottom? Do I need to change any other settings here? Will these settings impact the conversion in TMPGENC? (I'm sort of confused on exactly how the codec interracts with conversion tool...I always have been)

Also, I have other codecs installed - do I need to worry about any of these? Which one will my playback default to?

Thanks
Bob

[jagabo]

In order for a program to process a video file it has to first decompress the video. There are two video subsystems in Windows: the older VFW (Video For Windows) and the newer DirectShow. In addition to those, any program can use its own internal codecs. The MJPEG codec you are accessing with VirtualDub is a VFW codec. TMPGEnc Plus can use VFW or DirectShow decoders. I'm not sure about newer versions of TMPGEnc.

I'm not familiar with MC's MJPEG decoder. The Inverted Fields Order setting (in the Decoder section) might be the one you want.

Basically, some old video capture devices stored the scan lines in the wrong order. Instead of lines 0, 1, 2, 3, 4, 5... they swapped each pair of scan lines: 1, 0, 3, 2, 5, 4... So MJPEG decoders usually have the option to restore the correct scan line order. Your MPEG 2 encoded image showed exactly that.

[poisondeathray]

Also just because you have a certain codec installed , doesn't mean a program is using it .e.g you might have mainconcept mjpeg installed, but it might not be doing the DEcoding

In vdub, with the AVI file loaded directly use file->file information to see what VFW decoder is being used

There are utilities and ways to change the preferred directshow or vfw decoder being used. eg. vcswap for vfw, graphstudio, preferred filter tweaker for directshow. I don 't use tmpgenc, but jagabo says it can use VFW or DirectShow - there should be a way to specify otherwise you might get different results

[Bobbo01]

Thanks guys.

If I'm understanding this correctly, my main problem appears to be interlaced content that is being displayed wrong. Is that right?

Would I be better off just running the source material through intelligent interlace filters (NOT deinterlace), to correct the field order issues, and then convert to MPEG2?

My hopes is that this would eliminate the problems I experience when playing back on VLC player and DVD player. I would have to do some research on interlace filtering.

[jagabo]

VirtualDub also has a built in MJPEG decoder. It also has a Field Swap filter that does the same thing as the option in MJPEG decoders. Your source file opened with VirtualDub's internal MJPEG decoder, then field swapped:



You can force TMPGEnc to use VFW by moving the AVI source filters to the top of the list: Option -> Environment Setting -> VFAPI Plug-in.

AviSynth has the SwapFields() function.

[Bobbo01]

The field swap in VDUB totally fixed it! Thank you!!!

[Bob Belusko]

Hello!
Resurrecting a 15 year old post here as I'm now again dealing with a variation of this problem that I shelved since then.
The DVDs I created with the help in the posts above has been playing back fine since that last post back in 2006. The playback of this material on progressive monitors is smooth and resembles that of the original VHS source tape as played on an interlaced TV.

What I'm trying to do now is re-encode the source material from the source files, with modern encoders (H.265) to create better quality videos with a lower bitrate. Originally the source material was mastered with Pinnacle Studio Dc10+ (a now-EOL product) in the MJPG codec.

I cannot get the source material to play properly on ANY modern screen. It's interlaced and the material does not play as smooth as it should. I think the root problem (described above) is still there, but I'm unsure as to how to address it with today's technology.

Any suggestions? And I'm happy to provide further information.

Thanks
Bob

[Bob Belusko]

Hello!
Resurrecting a 15 year old post here as I'm now again dealing with a variation of this problem that I shelved for that time. I am 'DJBobbo01' - I could not create that same username.

The DVDs I created with the help in the posts above has been playing back fine since that last post back in 2006. The playback of this material on progressive monitors is smooth and resembles that of the original VHS source tape as played back on a VHS TV back in the day.

What I'm trying to do now is re-encode the source material from the source files, with modern encoders (H.265) to create better quality videos with a lower bitrate.

Originally the source material was stored created with Pinnacle Studio Dc10+ (a now-EOL product) in the MJPG codec.

I cannot get the source material to play properly on ANY modern screen. It's interlaced and the material does not play as smooth as it should. I think the root problem (described above) is still there, but I'm unsure as to how to address it with today's technology.

Any suggestions? And I'm happy to provide further information.

Thanks
Bob

[jagabo]

AVI doesn't really support interlaced encoding. You can put interlaced video in the container but most codecs don't indicate the video is interlaced. So players don't see it as such and play it as if it's progressive. That doesn't matter to an editor/encoder where you can tell the software the video is interlaced and have it treated properly.

These days I recommend you start with VirtualDub2 rather than the old VirtualDub. It has more filters and support for more input and output codecs/containers built in. If you want the absolute best deinterlacing you'll want to use QTGMC in AviSynth. But that has a very steep learning curve. Staxrip has a GUI and the ability to use AviSynth filters, including QTGMC.

[Bob Belusko]

Thank you again! (15 years later!)
Do you suggest I move the codec into a different containor and then process it in VirtualDub2? Or does that not matter?

My first thought is to try the same thing I did back in 2006: the field swap - yes?

[jagabo]

I suggested VirtualDub2 since you mentioned VirtualDub. There's no need to remux your MJPEG to another container. If your source is the same you need the field swap. But that can be done in the MJPEG decoder (many of them offer the option, ffdshow and PicVideo, for example) so it's not necessary to use VirtualDub(2) or AviSynth.

[Bob Belusko]

Thanks, one last question.

What exactly is happening with the 'field swap' that corrects this and makes the motion/video smooth?

My understanding is that the interlaced material displays at 60fps (30 odd and 30 even). I don't understand why changing the order would make the motion appear fluid, and how it got misaligned in the first place back when I digitized this video from it's VHS source?

Thanks
Bob