NLE DV with WMM? Any losses??

[towtruck]

Hello everyone

What I've been doing:
1) Transferring DV from my mini-DV cam to computer using WMM (firewire, no USB. I understand the transfer is a direct copy of what's on the cam).
2) Using WMM to cut & join, then save back to computer (DV-AVI, NTSC)
3) Frameserving (avisynth) the saved DV-AVI movie to HCenc for encoding, etc.

My concerns:
Do I incur any losses in saving back to DV-AVI in WMM? Is there any re-encoding going on? I've been reading some stuff that makes is sound that way, but it's really not that clear.

What I've been thinking of doing:
1) Transferring DV from cam with whatever
2) Using some undiscovered software to virtually cut & join in visual-timeline fashion.
3) Frameserving the original DV to HCenc for encoding, this time using the script to tell avisynth what to leave out and what to keep, etc.

The questions:
1) Are my concerns with WMM justified, or am I just fine? (I find WMM easy to use)
2) What is the undiscovered software I am referring to?
3) Any other suggestions for avoiding losses, or generally making life easier?

Thanks to all!!

[Cornucopia]

Here's a reasonable way to tell:

1. Cap/Xfer the DV with WinDV, DV-IO, VirtualDub, etc. You know for sure that the files are plain, compatible, Type1 or 2 DV-AVI files. Nothing lost yet (as compared to what's on tape).
2. Edit/append in Virtualdub, using DirectStreamCopy. Measure the time it takes to consolidate to 1 new output file (save to same directory so as not to have differences between drives, etc).
--this is basically the equivalent of a file copy!--
3. Edit/append same clips with same lengths inWMM. Export with DV-AVI. Measure how long THIS takes.

If they're nearly identical in time taken, it probably neither decodes/reencodes nor changes/reverts colorspaces (the 2 main places where there'd be loss).

Note that the filesizes might be different if 1 is Type1 (more compact) and the other is Type2 (duplicated audio), so there may not be exactly the amount of time taken. Since we know WMM only exports Type1, if you're using Type1 for those other apps, the comparison should be closer.

WMM seems to work fairly well with DV-AVI, but I wouldn't put it past M$ to screw something up...

Scott

[edDV]

Somebody was saying WMM converts to 640x480 square pixel if you use any of the processing filters.

I wouldn't trust it. WinDV is solid for transfer. Programs like Premiere Elements, ULead Video Studio, Sony Vegas Movie Studio maintain DV integrity when set for DV in and DV out.

When the Panasonic DV codec is used in VirtualDub problems result for consumer camcorder material coming in DV or DVD-MPeg2 format. It will convert 16 black, 235 nominal white digital material to 0 black, 255 white RGB.

The problem is consumer camcorders (all formats) cheat signal to noise by pushing whites up into the 235-255 region (above 100 IRE analog) like this



When the Panasonic DV codec converts this to 0-255 RGB for VirtualDub, all video above digital level 235 gets clipped to RGB level 255. If you do nothing else and export the clip to back to DV or DVD-MPeg2 format, you get all white clipped to 235 like this.



This is mostly a problem for consumer 8mm, Hi8, Digital8, MinDV and MPeg2 camcoder material. Prosumer and Pro camcorders tend to control whites to level 235 (100 IRE in analog) with only highlights and overshoots extending above level 235. Same goes for TV captures. Here is a higher end Hi8 camcorder for comparison. Ignor the lack of color balance, we are looking at white.



Since very little video exists above 235 white (100 IRE analog) it passes through VirtualDub + Panasonic DV codec with little damage.



Bottom line:
Avoid the Panasonic DV codec when the source is consumer 8mm, Hi8, Digital8 MiniDV or MPeg2 camcorder material.

[edDV]

For what it's worth, I decided to run the consumer clip above through Windows Movie Maker 2 to see what happens. It did OK for this test passing the full 235-255 range from import to export (no processing). BTW the audio did require reconstruction when the WMM DV output file was imported into Vegas DV project format.



Bottom Line: If all you are doing is cut edits, WMM2 will maintain levels for DV format in to DV format out. The rest needs testing.

[towtruck]

Thanks Scott! I'll try that.

EdDV,
Thanks as well. This is interesting. I actually understand the technical aspect as far as the clipping of whites is concerned, but I'm still in a bit of a fog as to exactly when decoding/encoding takes place.

As I recall, I download a Panasonic codec to get avisynth to work. So are you are telling me not to use this codec (for decoding the DV?) when I frameserve to HCEnc with avisynth? For just cutting and pasting DV (or portions thereof), why is a DV codec needed except for maybe the preview window? And doesn't VD use direct-stream copy??

So then the next question.... If not the Panasonic DV codec, which one???

Thanks for your patience. The more I learn, the more questions I seem to have.

BTW, I am using a panny, GS-320 to be precise.

[towtruck]

EdDV

Thanks again. I think your second post and mine came in at the same time; I didn't know of it when replying to your first post.

So it seems that WMM won't lose much when doing basic cutting and pasting. But I should still get a better codec to use with avisynth??

Thanks for all your effort!

[edDV]

VirtualDub default is a 0-255 24bit RGB color space. The Panasonic DV codec converts incoming DV format to RGB by scaling 16 to 0 RGB black and 235 to RGB 255 white. At import any video in the 0-16 or 236-255 DV range is tossed (hard clipped).

If you use AVIsynth to filter the 0-255 RGB, you are working on the clipped import. Here is the 0-255 RGB (intermediate format) used in VirtualDub. This image shows a 24bit RGB export from VirtualDub that is then imported into a Vegas DV project timeline.
It would look the same if the Vegas project setting was RGB for reasons I will get to.



Premiere and Vegas use a different form of 24/32bit RGB space sometimes called "studio RGB". This allows lossless conversion of 16-235 YCbCr to 0-255 RGB and back with the 16 level black, 235 level white and 0-15 236-255 overshoots preserved (no clipping).

For this reason, Vegas or Premiere can operate internally in RGB for filtering but still convert back to YCbCr for output.

It is possible to operate in proper YCbCr space in AVIsynth but I will defer to those with more detail on procedure.

[Cornucopia]

I'm having overall much better luck with the Cedocida DV codec (for VFW), though if you're using "DirectShowSource" in AVISynth, you ought to be just fine with the stock MS codec (no downloads/installs needed).

Scott

[edDV]

BTW, I am using a panny, GS-320 to be precise.

I'm talking about the old "Video for Windows" Panasonic DV codec that is now in the public domain not the camcorder codec.

Video for Windows has been abandoned by Microsoft in favor of WDM and DirectShow. VirtualDub is still using the older Windows video system.

DV is a standardized format. Codecs convert DV to other formats so can vary.

The Panasonic DV codec converts DV to 0-255 24bit linear RGB and back. This was where PC video processing was in 1998.

[tranbuimd]

Hi edDV,
Thank you for directing me to this thread.
Since I like the best (closest to the source, .avi) video quality in DVD format(mpeg-2?), it seems Lagarith lossless codec is what I need. However, could you tell me what exactly should I do in order to make Lagarith/Avisynth/HCEncoder (or any encoder) work together? Or I just need to install the Lagarith codec then any encoder will automatically pick it up (but I don't think so)?
I did and read many searches but still could not figure out how to actually use a codec!!!
Thank you in advance!
TB

[edDV]

Hi edDV,
Thank you for directing me to this thread.
Since I like the best (closest to the source, .avi) video quality in DVD format(mpeg-2?), it seems Lagarith lossless codec is what I need. However, could you tell me what exactly should I do in order to make Lagarith/Avisynth/HCEncoder (or any encoder) work together? Or I just need to install the Lagarith codec then any encoder will automatically pick it up (but I don't think so)?
I did and read many searches but still could not figure out how to actually use a codec!!!
Thank you in advance!
TB

Are you capturing from a DV camcorder and want a DVD? Then you present DV format to the MPeg2 encoder without any conversion. If you want to use avisynth along the way then you are going to need to decode DV to base YCbCr or convert to RGB. If you do that you need to preserve the 0-16 and 235-255 overshoot regions. I haven't tested all the DV codecs. If I need to go to avisynth I use Vegas or Premiere Pro to export to uncompressed YCbCr or RGB as need be. The first thing I do is test the filter chain back to the waveform monitor with the belle nuite color bar to see how the overshoot region is passed.

http://www.belle-nuit.com/testchart.html

[tranbuimd]

Are you capturing from a DV camcorder and want a DVD? Then you present DV format to the MPeg2 encoder without any conversion.

No, I captured from a Sony Hi8 cam, via Canopus ADVC-110 and WinDV. The files are .avi. Now I only want to burn them to DVDs using DVD format (mpeg-2?) without any editing but with best quality possible regardless of size, because I just want the best quality.
Thank you in advance!
TB

[edDV]

Are you capturing from a DV camcorder and want a DVD? Then you present DV format to the MPeg2 encoder without any conversion.

No, I captured from a Sony Hi8 cam, via Canopus ADVC-110 and WinDV. The files are .avi. Now I only want to burn them to DVDs using DVD format (mpeg-2?) without any editing but with best quality possible regardless of size, because I just want the best quality.
Thank you in advance!
TB

Hi8 via the ADVC-110 gives you the same DV-AVI format file as a DV format camcorder. You have two main choices.

1. Encode directly to MPeg2 and author for DVD. Do you have DVD authoring software? Check out ULead Movie Factory 6 as an example. It has a built in MPeg2 encoder.
http://www.ulead.com/dmf/runme.htm

2. Decode to uncompressed YCbCr or RGB for processing with Avisynth filters and then encode to MPeg2 and then author for DVD. This is the complicated hobbiest path and requires much learning curve. Each step is separate software.

A mid course would be to use a consumer to prosumer editing package that provides more user friendly filters and editing tools. Many of these also include MPeg2 encoder and DVD authoring tools. A consumer level example is Adobe Premiere Elements. A Prosumer example is Sony Vegas + DVD.
http://www.adobe.com/products/premiereel/
http://www.sonycreativesoftware.com/products/product.asp?pid=404

Note that I find Vegas adequate for most DV to DVD tasks but sometimes use Virtualdubmod or avisynth for specialized conversions or filters. You can choose to use all non-commercial tools but that requires a more step by step process with different software used for each step.

[JohnnyMalaria]

For simple cutting and joining, WMM is fine. It does NOT re-encode in such scenarios - I am absolutely certain of that based on a variety of tests I have done.

Using WMM for capturing is fine, too. Nearly every capture tool uses the same underlying code provided by the OS.

[tranbuimd]

2. Decode to uncompressed YCbCr or RGB for processing with Avisynth filters and then encode to MPeg2 and then author for DVD. This is the complicated hobbiest path and requires much learning curve. Each step is separate software.

Hi edDV,
I guess this path will give a better result and it's worth learning in the long run.
I have tried Premiere Elements and like it a lot. I'll try Ulead then I'll get one of them. From what I have learned, if I want the best possible video quality, I must use some sort of combination such as Avisynth/Premiere or something.

Hi John,
Yes I also tried WMM but only for conversion/burning to DVD and it seems the video quality was not good, therefore I left it. I did not try the editing features though, just because right now my big concern is video (best) quality. Is there any way to configure WMM to do the conversion to give lossless output?

This is a great hobby, especially when you helpful guys are around!

Thanks all!
TB

[JohnnyMalaria]

Hi John,
Yes I also tried WMM but only for conversion/burning to DVD and it seems the video quality was not good, therefore I left it.

That's odd - I always get great results converting to WMV from DV using WMM. In fact, just today I posted a video:

http://www.enosoft.net/video/Shark%2...202007%202.wmv

encoded at 2.1Mbps. Incidentally, WMM can encode WMV *very* fast. The original DV was processed with Adobe Premiere Pro 1.5 but I couldn't get the WMV quality I needed, so I used WMM to do that.

I haven't tried encoding from WMV to MPEG-2, though. Not sure how well that would work. WMV is a great format for generating high quality files with small size.

(More info about the clip is at Shark Encounter Off North Carolina Coast)

[tranbuimd]

Hi John,
I did not try WMV, therefore I can't say about the quality, but from what I saw in your "Shark Encounter Off North Carolina Coast" clip, I must say it's good.
The "Shark Encounter Off North Carolina Coast" clip is unbelievable!!! Did you do some editings or just convert it to WMV? Did you make DVD and watched it on normal player? I guess the camcorder is also a key element though?
Cheers,
TB

[edDV]

I agree that DV import seems to pass through WMM fine to DV-AVI export. I haven't really evaluated their filters and transitions but I read somewhere they are processed at 4:3 640x480 even for wide source. I haven't verrified that.

2. Decode to uncompressed YCbCr or RGB for processing with Avisynth filters and then encode to MPeg2 and then author for DVD. This is the complicated hobbiest path and requires much learning curve. Each step is separate software.

Hi edDV,
I guess this path will give a better result and it's worth learning in the long run.
I have tried Premiere Elements and like it a lot. I'll try Ulead then I'll get one of them. From what I have learned, if I want the best possible video quality, I must use some sort of combination such as Avisynth/Premiere or something.

This all depends if your Hi8 capture needs levels correction or filtering before encoding.

I find with the ADVC-1xx box correctly set to 7.5 IRE (for NTSC) levels are mostly usable as captured. Premiere or Vegas have various filters for color correction of sharpening that are good enough most of the time. Generally, the worse the source the more one feels they need to filter. Most of my Hi8 came from a reasonable quality camcorder so it doesn't need much filtering. VHS/S-VHS is another story.

If you do decide to filter, try the same clip with different filters and evaluate on a TV. The computer monitor gives poor prediction for what it will look like played from a DVD player.

[JohnnyMalaria]

Did you do some editings or just convert it to WMV?

The raw DV footage shows a little vignetting due to the camcorder seeing the lens port on the housing when recording at the widest zoom. On a standard TV, this isn't visible since it falls in the overscan range. For use on a PC, I stretched the video horizontally by about 10%. Since the subject is underwater you can't really tell. The stretching was done in Premiere, as was the addition of the text.

Did you make DVD and watched it on normal player? I guess the camcorder is also a key element though?

Yes. Premiere can create a DVD from the timeline. Hence, it uses the DV footage as the source.

The camcorder is a fairly old (~1998) Sony DCR-SC100. A no-frills consumer unit that is very compact. I'd love to try my 3-CCD PDX-10 - but I would have to spend a lot of money getting a housing for it that isn't clunky and cumbersome.

You can see my underwater system here:

http://www.uwimaging.com/support/downloads/pdfs/Mako_SC100.pdf

(It doesn't show the lights and their battery pods.)

[tranbuimd]

I find with the ADVC-1xx box correctly set to 7.5 IRE (for NTSC) levels are mostly usable as captured.

Hi edDV,
That's what my box was set, and from my eyes, I think the quality is very good, very close to the sources.
I plan to have 2 "paths": 1/ I will make DVDs with "reporter" style content, that is, leaving the images as-is, reflecting the shooting conditions. I call it's the "historical path." On the other hand, I'll make "movie style" DVDs telling some story. This is where I would like to manipulate the clips a little. At first I guess I'll go with Premiere Elements as core learning tool and I hope I could go further. I now have AvidFree installed in my system, however it behaves a bit strange here and there, so I guess Elements is a way to go.

Since the subject is underwater you can't really tell.

I agree. Great job. The clip reminds me of J.J.Cousteau ocean series. Yours is very close!
Just send the links to my brother-in-law. He's diving and shooting in Sydney.

Cheers,
TB

[towtruck]

Hello Everyone

I started this crazy post, so I thought I'd weigh in....

WMM....
I would have to agree with JM and edDV in that using WMM to transfer DV from miniDV camera (lets use more technically accurate lingo here) and basic cutting and pasting operations does not incur any losses. There may be some losses using other filters, but if you are just doing transitions/fades etc, it really doesn't matter much because these are very short segments that you are adulterating already.

If you want to go directly from transferred DV to an MPE2 encoder using AVISYNTH, I have had great success with HCENC (great video quality). I use the canopus DV codec to prevent the white clip problem.

I have aslo tried AVSfilmCutter to do the editing, and once you figure it out, you can edit quite quickly. One of the nice things about it is that it will also produce the *.wav from the trimmed source.

So many ways to do the same thing....

[tranbuimd]

Hi towtruck,

If you want to go directly from transferred DV to an MPE2 encoder using AVISYNTH, I have had great success with HCENC (great video quality). I use the canopus DV codec to prevent the white clip problem.

This is also what I would like to know more about. Could you please describe exactly step by step what you have done?
I did the conversion (.avi to .m2v) using Avisynth/HCEnc and got great, very close to the source, video, too. The properties of the output file show codec = "Sonic CM(R) DS VideoDecoder 4.0."
How could we use the Canopus DV codec instead?
Thank you in advance!
Cheers,
TB
PS: If you see any sign that I tend to hi-jack your post please correct me. Most of the cases I even did not realize that! Just because I tend to ask a lot! Sorry!!!

[towtruck]

use the fourCC parameter in avisource.

See:
http://avisynth.org/mediawiki/AviSource
and search for canopus to get the correct syntax.

[towtruck]

BTW, hi-jacking is fine by me. I'm happy this thread can help others

[tranbuimd]

Hi Towtruck,
Where could we get the Canopus codec? As I know, the Canopus codec on this site is for playback only.
Last night I did a little experiment using WMM to split and put some clips on the storyboard and "save to my computer" as DV-AVI (NTSC). The output file, which is DV-AVI type 1, looks very identical with the sources in term of video quality. However, it could not be used to make MP4 file using MPEG StreamClip which did a great job with the original clips. The original clips are DV-AVI type 2 captured by Canopus and WinDV. To my eyes there seems no loss in video quality, but I believe the file structured is altered in somehow otherwise the MP4 converter could have processed the file.
Cheers,
TB

[JohnnyMalaria]

With simple cuts/splits, there will be no loss in quality since WMM simple spews out what comes in.

To use the files from WMM with MPEG StreamClip, you need to convert the Type-1 from WMM to Type-2.

(You can use our Enosoft DV Processor to do that.)

[towtruck]

Playback Only means that the codec will DECODE the DV. It will not ENCODE to DV. It doesn't matter if you will be viewing the decoded material or sending it on for processing (encoding to MPEG2). So the decoder on this site does just fine.

I'm not sure, but I'm guessing StreamClip only handles type-1 files, (VFW, or VIdeo for Windows, the older format). If so, it would explain why your type-1 files don't work. An avi is a container file. The container file is structured a bit differently for type-1 or type 2, but the DV data inside should be the same. WMM only outputs type-1 (DirectSHow, the new format), as MicroSoft likes to force people to do things its way. But I understand there is also a canopus converter available that will convert between the two different types, or split from type-1 to type-2. I simply haven't needed it.

[towtruck]

With simple cuts/splits, there will be no loss in quality since WMM simply spews out what comes in.

Q for JM. Why does it take so bloody long to 'SAVE a MOVIE' in WMM w/simple cuts/splits (DV in, DV out). Does it really take that long to read the required frames and write them back out in the requested sequence?

[JohnnyMalaria]

How long is "bloody long"?!! Is it slower than the duration of the timeline? i.e., if you have 5 minutes on the timeline, does it take much longer than 5 minutes to create the file? I find it takes about half the length - i.e, 2.5 minutes to create a 5 minute file.

The cause is most likely your hard drive data transfer rate. DV files are BIG - 3.5MB per second. Even just creating a simple cut/split movie requires that the source data are read from the hard drive, stripped all the AVI stuff, put into a new AVI wrapper (for the output) and written back to the hard drive. Just the raw data transfer is 3.5MB/s read and 3.5MB/s write. If you are using the same drive for source and edited, you'll also take a hit, especially if the heads have to keep switching from one part of the drive (where the source files are) to another (where destination file is).

If you'd like to do a comparison, using Windows' copy function to make a copy of an existing DV AVI file on the same hard drive and time the process. See how long it takes compared to the duration of the DV. (I'd be interested to find out what you get).

[Cornucopia]

Playback Only means that the codec will DECODE the DV. It will not ENCODE to DV. It doesn't matter if you will be viewing the decoded material or sending it on for processing (encoding to MPEG2). So the decoder on this site does just fine.

I'm not sure, but I'm guessing StreamClip only handles type-1 files, (VFW, or VIdeo for Windows, the older format). If so, it would explain why your type-1 files don't work. An avi is a container file. The container file is structured a bit differently for type-1 or type 2, but the DV data inside should be the same. WMM only outputs type-1 (DirectSHow, the new format), as MicroSoft likes to force people to do things its way. But I understand there is also a canopus converter available that will convert between the two different types, or split from type-1 to type-2. I simply haven't needed it.

Sorry to correct you, but both VFW and Directshow apps have been known to work with both Type1 and Type2. Type1 vs. 2 has little to do with VFW vs. Directshow (there is a historical connection, but that's for another time). The difference in a nutshell is this:

Type1 Container:
1 stream--
("IVAS" = Interleaved Video+Audio)
more compact than Type2, but onus of recognizing the stream is upon the App, not the API/OS.

Type2 Container:
2 streams--
("VIDS" =Video stream-->Actually just renamed header from IVAS; the audio is still interleaved in there in DV streams)
("AUDS"=Audio stream-->added/duplicated from the audio portion of the DV IVAS during capture)
Easier for most apps to understand without extra recognition code, little bit heavier CPU use and larger filesize though.

Streamclip probably only handles Type2. Quicktime, I know, can't handle Type1.

Scott