Unanswered questions on processing and compressing DV-AVI

[sd_smoker]

In the process of learning how to process my DV-AVI files and compress the finished product I've done tons of searching and reading in these forums (and elsewhere). I really want to avoid posting the same repetitive newbie questions and I've been able to find most of the information I need, but I find myself with a few questions remaining unanswered. Mostly I just need some clarification on a few things.

Here's my situation. I'm capturing DV from a camcorder via WinDV (interlaced, 16:9, Panasonic DV Codec). I'm doing some minor editing in Ulead VideoStudio 8 and saving back to DV-AVI with SmartRender enabled, which allows only the modified portions of the file to be re-rendered. My next step is to apply some filters and compress in VirtualDubMod and there is where I need some assistance.

Question 1: If speed is not a concern, what exactly are the drawbacks of doing filtering in Vdub instead of AVISynth? Right now I'm using the 2d Filter, DNR (MMX), and Temporal Smoothing in VDub, just to give you an idea.

I'd love to use AVISynth but it's just way over my head at this point. I would learn much better if I had a script to start with that did more or less what I want that I could play with. So...

Question 2: Are there any AVISynth script repositories where you can find existing scripts geared toward certain tasks. For example: "Cleaning up and deinterlacing a DV-AVI", "Cleaning up DV and converting to MPEG-2", etc.

I think I mostly understand the benefits and drawbacks of deinterlacing my video, but I think I'm missing something. For one, I literally cannot tell the different between interlaced and deinterlaced on my CRT monitor (using Smart Deinterlacing in VDub). I was leaning toward not deinterlacing for various reason, not least of which is that display deinterlacing will only continue to improve, but I seem to remember reading that DivX is a "progressive" format, whatever that means. I have been leaning towards DivX but I just read that MPEG-2 would be the preferred format for archiving (between the two, anyway), especially for interlaced video. Also, let's assume that I will not be keeping the original DV source. Sorry for the rambling, but I'm on information overload here and could use some help sorting it out. In a nutshell...

Question 3: Assuming I won't have access to the source, is there anything to justify deinterlacing the end product given that display deinterlacing will only get better over time?

Question 4: Based on the answer to #3, would DivX or MPEG-2 be preferable? I'm not concerned about being able to play the file anywhere and everywhere and I don't plan on turning them into DVDs. I'm just interested in quality for a given size and how each format handles interlaced/deinterlaced video.

Sorry for the long post, but this is where I find myself after many hours of research and experimentation. Thanks for your help!


Edit: I found a post on Doom9 suggesting that GSpot may sometimes report progressive video as interlaced. I'm assuming my camera (Canon ZR500) shoots interlaced since GSpot reports the video as interlaced and since I can't find any info about it being progressive.

[Soopafresh]

Question 1: If speed is not a concern, what exactly are the drawbacks of doing filtering in Vdub instead of AVISynth? Right now I'm using the 2d Filter, DNR (MMX), and Temporal Smoothing in VDub, just to give you an idea.

If the Vdub filters look good to your eyes, then continue doing so. Avisynth's filters are pretty amazing, but you're right, there's more work involved. There's a learning curve to figure out Avisynth. A few automated apps such as AutoMKV have built-in Avisynth deinterlacers and noise reduction plugins which you can try.

There are also some really good vdub filters here: http://acobw.narod.ru

Get Camcorder Color Denoise and Smart Skating Rink


Question 2: Yes, Avisynth has a wiki with configured scripts. Mcbob is a great deinterlacer, but tremendously slow (1fps). Deinterlacing is an argument with no solution. Both sides make sense. Some people like it, some don't. Mpeg2 is somewhat better suited to interlaced footage than Divx/Xvid.

Questions 3&4: Up to you. Ultimately, storing your original DV files might make the most amount of sense. Disk space is cheap and the quality will arguably never be better than your source. Standard Definition DV doesn't take kindly to filtering or deinterlacing, in my opinion. There's too little information in the picture to begin with. It always ends up looking worse than if you left the source alone.

[sd_smoker]

There are also some really good vdub filters here: http://acobw.narod.ru

Get Camcorder Color Denoise and Smart Skating Rink

Great site, thanks. It would nice if there were explanations for the filters. Since you recommended it, what does Smart Skating Rink do exactly?

Standard Definition DV doesn't take kindly to filtering or deinterlacing, in my opinion. There's too little information in the picture to begin with. It always ends up looking worse than if you left the source alone.

I'm beginning to agree with you. Of the filters I listed I find I really like the temporal smoother, especially for low light, which many of my videos are. The rest don't seem to help much at all. Other than that I really wish I could find a good color adjustment filter. One of my biggest complaints is that the color just seems to fall kinda flat. The only hue/saturation filter I found was terrible. If I could just smooth it out and add just a little punch to the colors I'd be happy.

Thanks again for the reply.

[2Bdecided]

mc_spuds (an AVIsynth script) or Neatvideo are two good denoisers.

People doing scene-by-scene colour grading usually use Vegas, not VirtualDub.

You need a good reason to deinterlace. If you can't see the difference, then either your footage is progressive (unlikely) or you player is already deinterlacing (quite likely with DV), or your player is only showing every other field.

Cheers,
David.

[sd_smoker]

You need a good reason to deinterlace. If you can't see the difference, then either your footage is progressive (unlikely) or you player is already deinterlacing (quite likely with DV), or your player is only showing every other field.

I only viewed it using VLC which (I believe) has deinterlacing turned off by default. I'll have to check when I get home...

[edDV]

...

Here's my situation. I'm capturing DV from a camcorder via WinDV (interlaced, 16:9, Panasonic DV Codec). I'm doing some minor editing in Ulead VideoStudio 8 and saving back to DV-AVI with SmartRender enabled, which allows only the modified portions of the file to be re-rendered. My next step is to apply some filters and compress in VirtualDubMod and there is where I need some assistance.

There is no DV codec used for WinDV capture (other than for low res monitoring). The DV-AVI file is a direct transfer. Likewise Ulead VS8 uses an internal codec for conversion or monitoring. If "minor editing" means simple cuts (no levels or other filters), a DV project will copy input frames to output without recode. "SmartRender" is intended for MPeg2 projects.

Virtualdub does use a DV codec on import. This is where things get difficult because YCbCr gets converted to RGB on import. Use caution with the Panasonic codec. It converts DV level 16 to RGB zero and DV 235 to RGB 255 thus hard clipping the overshoot whites and black. I do my conversion to RGB inside Vegas where overshoots can be managed and then export RGB to Virtualdub. I haven't attempted this from ULead Video Studio.

Also, I'd avoid deinterlace if MPeg2 DVD is a goal. Encode to MPeg2 (or divx) as the last step.

[sd_smoker]

There is no DV codec used for WinDV capture (other than for low res monitoring). The DV-AVI file is a direct transfer. Likewise Ulead VS8 uses an internal codec for conversion or monitoring. If "minor editing" means simple cuts (no levels or other filters), a DV project will copy input frames to output without recode. "SmartRender" is intended for MPeg2 projects.

Sorry, I meant that the camera uses the Panasonic DV Codec. Interesting about SmartRender. Their documentation apparently leaves much to be desired.

Virtualdub does use a DV codec on import. This is where things get difficult because YCbCr gets converted to RGB on import. Use caution with the Panasonic codec. It converts DV level 16 to RGB zero and DV 235 to RGB 255 thus hard clipping the overshoot whites and black. I do my conversion to RGB inside Vegas where overshoots can be managed and then export RGB to Virtualdub. I haven't attempted this from ULead Video Studio.

Are there any other (free) DV codecs that would not do this that I could use from within VirtualDub? I'm not sure Ulead VS is equipped to handle overshoots. Even if it were, I'm not. If I could avoid that ordeal altogether that would be ideal.

Edit: On second thought, if my camera is encoding with the Panasonic DV Codec in the first place does that not make this a moot point?

Also, I avoid deinterlace if MPeg2 DVD is a goal.

Understood. Is it as important to avoid interlaced video for DivX?

[Soopafresh]

Smart Skating Rink is a pretty powerful denoiser.

[edDV]

There is no DV codec used for WinDV capture (other than for low res monitoring). The DV-AVI file is a direct transfer. Likewise Ulead VS8 uses an internal codec for conversion or monitoring. If "minor editing" means simple cuts (no levels or other filters), a DV project will copy input frames to output without recode. "SmartRender" is intended for MPeg2 projects.

Sorry, I meant that the camera uses the Panasonic DV Codec. Interesting about SmartRender. Their documentation apparently leaves much to be desired.

Camcorder hardware DV codecs are fairly generic (to standard) so produce equivalent levels at camcorder output. "Panasonic DV Codec" usually refers to the free Video for Windows DV codec often used with Virtuadub. It is more of an issue for camcorder generated DV source rather than broadcast captures. See this thread for overview.
https://forum.videohelp.com/topic352511.html

Alternative to Panasonic is the Cedocida DV Codec. I haven't investigated it fully since I use Vegas (Sony DV codec) for RGB conversion.

[edDV]

Edit: On second thought, if my camera is encoding with the Panasonic DV Codec in the first place does that not make this a moot point?

The hardware DV codec in the camcorder is a different animal than the software Video for Windows Panasonic DV codec. The camcorder codec works the full overshoot ranges (0-16 and 236-255).

Expect any camcorder shot video to range 16-255.

[sd_smoker]

Alternative to Panasonic is the Cedocida DV Codec. I haven't investigated it fully since I use Vegas (Sony DV codec) for RGB conversion.

Thanks. Since the RGB conversion happens when importing into VDub, is this even relevant? My guess is that selecting the Cedocida codec within VDub only applies when the video is output encoded. If so, is there another free app that I can use to do the RGB conversion before importing into VDub?

[edDV]

Alternative to Panasonic is the Cedocida DV Codec. I haven't investigated it fully since I use Vegas (Sony DV codec) for RGB conversion.

Thanks. Since the RGB conversion happens when importing into VDub, is this even relevant? My guess is that selecting the Cedocida codec within VDub only applies when the video is output encoded. If so, is there another free app that I can use to do the RGB conversion before importing into VDub?

DV import to virtualdub is where white clipping occurs so it is a potential problem. Cedocida offers more settings control including YUV and YUV to RGB conversion models but I haven't tested it. Others can comment.

The DV codec provides both import and export conversion.

[sd_smoker]

Another question about interlaced video. Everything I've read about applying filters says that you should always deinterlace before applying filters. If I decide to leave it interlaced, what does that mean for me? Will the filters still work or am I better off not applying them at all (or deinterlacing)?

[sd_smoker]

The DV codec provides both import and export conversion.

I saw that but I'm not sure how to take advantage of that with regards to importing into VDub. I'm guessing I'd need to utilize the codec with another piece of software to perform the RGB conversion, but what?

[edDV]

The DV codec provides both import and export conversion.

I saw that but I'm not sure how to take advantage of that with regards to importing into VDub. I'm guessing I'd need to utilize the codec with another piece of software to perform the RGB conversion, but what?

The DV codec does the DV YCbCr (YUV) to RGB conversion plus any conversion of RGB back to DV.

In the picture above, the decoder setting [16..235] --> [0..255] is what the Panasonic codec does.

RGB [0..255] --> [0..255] may preserve overshoots but needs testing with the desired filters.

[sd_smoker]

The DV codec does the DV YCbCr (YUV) to RGB conversion plus any conversion of RGB back to DV.

In the picture above, the decoder setting [16..235] --> [0..255] is what the Panasonic codec does.

RGB [0..255] --> [0..255] may preserve overshoots but needs testing with the desired filters.

Sorry, I think I'm missing something here. How do I tell VDub to use the Cidocida codec as opposed to the Panasonic codec for import conversion when I have both installed?

[edDV]

The DV codec does the DV YCbCr (YUV) to RGB conversion plus any conversion of RGB back to DV.

In the picture above, the decoder setting [16..235] --> [0..255] is what the Panasonic codec does.

RGB [0..255] --> [0..255] may preserve overshoots but needs testing with the desired filters.

Sorry, I think I'm missing something here. How do I tell VDub to use the Cidocida codec as opposed to the Panasonic codec for import conversion when I have both installed?

You would uninstall the Panasonic and install the Cidocida. Only one DV codec is recognized under the Video for Windows scheme. It will show up in all VFW apps including ULead Video Studio and Virtualdub.

[gadgetguy]

Another question about interlaced video. Everything I've read about applying filters says that you should always deinterlace before applying filters. If I decide to leave it interlaced, what does that mean for me? Will the filters still work or am I better off not applying them at all (or deinterlacing)?

Deinterlace isn't really the right term here. Some filters are interlace aware, but what you need to do to use most filters on interlaced material is separate the fields before applying the filters and then weave them back together after filtering, maintaining the proper field order.

[sd_smoker]

You would uninstall the Panasonic and install the Cidocida. Only one DV codec is recognized under the Video for Windows scheme. It will show up in all VFW apps including ULead Video Studio and Virtualdub.

Aha! THAT was what I was missing...

[sd_smoker]

Deinterlace isn't really the right term here. Some filters are interlace aware, but what you need to do to use most filters on interlaced material is separate the fields before applying the filters and then weave them back together after filtering, maintaining the proper field order.

Makes sense. Is this possible in VDub?

[edDV]

You would uninstall the Panasonic and install the Cedocida. Only one DV codec is recognized under the Video for Windows scheme. It will show up in all VFW apps including ULead Video Studio and Virtualdub.

Aha! THAT was what I was missing...

Here is where you find it in ULead VS. You should also see Ulead's DirectShow DV codec in the list.

[gadgetguy]

Sorry, I don't use VDub enough to know that answer. I do all of that stuff in AviSynth and mostly only use Vdub to view the output.

[sd_smoker]

I'd acutally be very suprised if this was possible in VDub, so it looks like AVISynth is in my future despite my many efforts to avoid it. I even found a function for applying filters to interlaced material in AVISynth (here).

I also just read that performing all filtering in AVISynth retains the YUV colorspaces (YUY2->YV12 as opposed to YUY2->RGB32->YV12), which addresses the other issue I've been discussing with edDV.

[sd_smoker]

Found this interesting tidbit from this thread...

If you want to do your filtering in VirtualDub and avoide losing blacker-than-black and whiter-than-white portions use [in an AVISynth script]:

AVISource("c:\test.avi")
ConvertToRGB32(matrix="PC.601")

Not sure what that does technically speaking but it sounds like a good workaround if true.

[edDV]

With AVISynth you can stay in YUV space. Virtualdub, ULead VS and Vegas are all internally RGB* so DV->RGB->MPeg2 conversions can be tricky.

* Ulead and Vegas have virtualized YUV user control for many of the filters but most Virtualdub user interface is in RGB. In the case of Vegas, RGB conversion is done only when a filter or transition is applied and affects those frames only. Virtualdub first converts all import frames to RGB.

[redwudz]

I would recommend the Cedocida DV Codec over the Panasonic DV codec. Also for a good color, etc., adjustment filter for VD, try ColorMill: colormill2.1.1.zip

[sd_smoker]

Also for a good color, etc., adjustment filter for VD, try ColorMill: colormill2.1.1.zip

Thanks, I'll check it out.

Here's something really strange. I just ran Info() on my source file with AVISynth and it's telling me that it's already in the RGB24 colorspace. When would that have happened? Does this eliminate the concern over the RGB conversion in VDub?

[jagabo]

If you want to edit DV in VirtualDub without losing blacker-than-black and whiter-than-white the Cedocida decoder can be set to avoid the luma expansion on converting YUV to RGB. Force it to convert output to RGB and select the [0-255] -> [0-255] option:

I just ran Info() on my source file with AVISynth and it's telling me that it's already in the RGB24 colorspace.

If you are using Panasonic DV Codec and AvisSource(): Panasonic's DV decoder always converts to RGB with luma expansion. Use Cedocida instead and allow YUY2 output.

[sd_smoker]

I just ran Info() on my source file with AVISynth and it's telling me that it's already in the RGB24 colorspace.

If you are using Panasonic DV Codec and AvisSource(): Panasonic's DV decoder always converts to RGB with luma expansion. Use Cedocida instead and allow YUY2 output.

At what point does this conversion occur? I haven't done any processing on the source. My understanding was that VDub converted to RGB when VDub is run in Full Processing Mode. Does it convert to RGB when all you do in VDub is open an AVISynth script?

[guns1inger]

If you are opening a script just to view it, or to compress it without using any of vdub's filters, use Fast recompress mode. This retains the original colourspace, which will be whatever you set it to in avisynth.