VituralDub + Deinterlace MAP file + 57+GB file????

[Bstansbury]

I hope someone out there can help me understand this problem. I used VideoStudio to transfer several hours of DV video via firewire to my harddiskl and it saved in DV format (type-1 avi). After much editing & authoring with titles ,etc. I have produced several MPG files. All had a lot of artifacts from interlace input data which creates more problems in the MPEG compression process.

So I used VirtualDub and the deinterlace MAP filter on the edited AVI file. I was shocked to see the filter process did not finish as I ran out of space. I was putting the output on one of my 4 disks, which had 57 GB free when I started. The file stopped processing at about 90% complete as there was no more disk space. I can't understand how a 7 GB avi file was converted to 57 GB just by using the MAP filter. This deinterlace filter does double the number of frames, how I don't know, since it should be taking to fields of 320 x240 and converting them into one progressive frame of 640 x 480.

When I played back the file that was captured, the de-interlace filter was very successful, and the jaggaged edges were pretty much removed. I guess if I knew more about VirtualDub I could have frameserved that into TMPGEN and converted the whole thing into SVCD MPEG on the fly rather than saving the output AVI.

If anyone has suggestions on how to proceed,... please post them here.

Thanks

[au7usa]

DV is lossles but it is still compression. If you used VirtualDub with even one filter without selecting any output compression, the resulting video will be uncompressed!

TMPGEnc can handle interlaced input. If you use the Wizard in TMPGEnc it should determine the correct field order. Then use the "File / Preview" to see what the output will look like.

If you can't make that work, then try to deinterlace through the TMPGEnc, "Setting / Advanced-Tab / Deinterlace". (Even-Odd field is easy choice)

[Bstansbury]

Thanks for you reply. The de-interlace function in TMPGen is very basic and just eliminates either the odd or even field. This is tossing out 1/2 the inmage. Sure it gets rid of the jagged edges but at a large loss in quality.

I know that DV .avi is compressed,... but I don't believe VirtualDub has any method to decompress it. This is like saying the is a tool to decompress MPEG files back into the orginal film. I don't think so. No matter what compression format you have, and all formats have some compression, like jmpeg, WMF, YUYV, etc., once they are processed into the file, they can't be restored to the orginal,... it is gone forever.

Last night I tried a different deinterlace filter called Smart deinterlace 27B. It also did a great job on the video but again the file size grew to
67 GB. This time I got all the file, with no lost frames. I viewed the file and everything was there,... nothing missing, nothing added, except that since VirtualDub can not handle Type-1 DV audio stream, this is missing.

Now I'm looking for a way to get the audio stream frameserved back together via TMPGen with the Video stream and then compressed down to SVCD. I was able to strip out the audio stream from my orginal .avi file and save that in .wav format. Now to get them both into TMPGEn.

I tried to load the entire .avi file back into VideoStudio but it would only load the first few clips. It can't view the whole file and the properties icon says the file is only 68 seconds long (should be 38 minutes).

Hope some one out there has some good ideas.

Bud

[BJ_M]

DV is NOT Lossless !

It is a Lossy format ..

Besides that ..... if you are going to play your video back on a normal tv , you should NEVER de-interlace your video files froma DV source (unless a progressive type DV camera -- but you will need a program like vegas to handle that)
...


yes you will see interlacing lines on your PC monitor - which is progressive scan only -- but you will not see them on a TV , which is interlaced , or even on many progressive scan rear view projection tvs ...

you are just losing detail in your image

virtualdub will open DV files if you install the main concept codec or the free panasonic one .. and you can re-compress back to the same format (DV) that way (which virtualdub will do with no problem - IF you remember to select compression type first)..

[au7usa]

"DV is not Lossless!"

You weren't supposed to catch that part. I was simply trying to explain that he is going from a compressed format to uncompressed. I just cut a few corners.

Your right about the interlacing too. California, although a strange and unusual place, is still part of NTSC-land. Just let TMPGEnc Wizard determine the correct field order and encode as-is.

[Bstansbury]

The reason I started this effort was because of all the artifacts in the SVCD disk I made from the orginal interlaced .avi files. I don't understand you comments that--"you won't see the interlaced lines because TV is interlaced. You can't tell me you've never seen them. They are terrible. Also, It has to affect the MPEG compression.

DVD disks are encoded to MPEG from clean "Film" which is progressive. When it gets to a DVD player the interlace is added so it will play on TV.

It's my belief that a clean deinterlaced file input to a good MPEG encoder will yield the best quality video. Of course it gets interlaced again because it is going to TV, but not before the MPEG compression is done.

I'm not sure but I believe that if the input source is interlaced, MPEG encoder does one field at a time and then all fields are output as interlaced TV. The problem is that the two fields have a time difference and are "NOT" the same at any one instant in time. The results are two different MPEG "Macro" blocks being interlaced on TV. The results are crappy jagged lines on any action scenes where there are lines or edges like swinging a golf club. I think some of this could be the difference of how Digital Video is recorded in the camera and how it is interlaced before it is transfered either by firewire, S-video, or composite. I understand that it is very different than what is done with an analog camcorder.

I've made the interlace MPEG disks in TMPEGen, VideoStuido, and ATI MMC, and they all have the jagged edges. My input is good quality DV from a Sony DV Camcorder and transferred to harddisk via Firewire with without loss of any data.

Bottom line,... I'm trying to get the best quality I can from camcorder to SVCD.

[BJ_M]

Bstansbury - i'll make it short and to the point.... i do this for a living and you are incorrect ...

but - do what you would like ... nothing will catch on fire ...

[Bstansbury]

In support of my post above, here is the URL of the Doom web site where they have 50-60 pages of information from "The Experts" on just about everything to do with Analog Video. While it mostly deals with Analog Video for TV, the basic point is that they are interlaced,... and so is Digital Video from Camcorders. Here is a quick summary of theirs on why to capture in max resolution and as lossless as possible.

The advantages/disadvantages of capturing at Full NTSC:
+ all possible information is captured
+ best quality possible
- very high requirements for hardware and software
- huge amounts of hard disk space needed
- Huffyuv can't be used with smaller hard disks
- deinterlacing is necessary
- generally requires a lot of filtering
- requires large processing power and time

The URL is; http://www.doom9.org/index.html?/divx-encoding.htm

Given I'm Transferring data from my DV camcorder, that has an input resolution of ;
Digital Video Type-1, 24 bits, 720x490, Field A, Interlaced, DV audio-NTSC, 32,000 Hz, 16 bit Stereo.

If you read the preface of the article, it's focus is on "Quality" and as such they highly recommend using VirtualDub and as "Many Filters" as necessary to achieve the desired results. They also point out that, in their opinion, the best de-interlacing software is the Deinterlace - MAP, that I used and posted above.

I started this thread looking for answers to why my file sizes were so large. I guess I understand it is because the files are saved in uncompressed RGB format, not DV .avi format, or anything else. Since I was finished with all editing of the file, I guess I could have saved it in some other format,... but that is my next step, to frameserve it to TMPGen and create the SVCD MPEG file.

The guide at the Doom web site mentioned above is very complete and I recommend it to anyone who wants more information about VirtualDub and what you can do with it.

[Bstansbury]

It's been awhile since I added anything to this post and hopefully this will close it out.

I want to thank all the reply's i got, especially the ones that told me I was wrong. I still have the stuborn streak in me that came from distant relatives. No way to get rid of it,... it has served me well in the past. I never trust information that comes to me without convincing explanation. When someone says, "trust me,... just do it this way,.. " I can't leave that alone until I understand it.

What really got me going was the information I found searching other sites for information about Interlacing and DE-interlacing,... especially the tools that do it. The people that right those articles are so far beyond what I'm doing, they never mention what is obvious to them or why it is obvious,... they just do deeper into how, what, where,... etc.

Anyway, I went through the entire process,...capture, de-interlace, re-encode back to interlace, burn to SVCD, and to my eyes things were getting better. IF you could have seen some of the artifacts in my first few sVCD, you could see why I had convinced myself that de-interlace was working.

Because "BJ_M" really challenged me,... I went back and re-did everything and tested all my formats, bit rates, CQ-VBR, CBR, VBR, and left my orginal in interlace mode. The final SVCD look good,.. just as good as the others I'd done the hard way.

In tracing down what the real story is on Field A, B order, which has taken another several days, but coming out just as I had started, which is the answer for AVI is "Bottom Field First", I finally got that out of my way.

It also turns out that most of the arifacts that I was seeing in my video are there from the orginal DV when viewed on TV. They are a function of lighting, action, and Cr, Cb, of the Sony Composite Capture. It can leave some rough edges on lines that move with reflected lite. Let's face it,... this is not a $35,000 Digital Betacam, or not even a $2000 DV unit. But it ain't bad.

Bottom line,.. I want to thank everyone,... I'm happy I have this pretty much down pat, from raw video, to transfer, to editing, adding transitions and effects, music, voiceover, titles, chapters, and finishing the video into something that resembles an acceptable production. By now I've made done 2 projects with 3 hours each of raw video,.. 2 VHS convertions to digital and down to great quality DVD's, and now working on the 3rd 2 hour video I shot at Pebble Beach.

Now I'm wishing I had that $2000 camera.

I look back on the last two weeks and "Wow,... the things I have learned.

[BJ_M]

there are over 50 types of "AVI" ... DV is always bottom field first - but most others are top field first .. many can be either way , operations like resizing and cropping can change the field order in some editors also ..

some frame servers also - will only serve one way