file in Virtualdub has strange colors when opened

[poisondeathray]

ffdshow will have higher priority than anything else, so disable it

in ffdshow you need to use the vfw configuration (yellow icon, not red icon) , and set DV to disabled , then check vdub again to see if cedocida is the decompressor, and set that to yuy2. you can also access cedocida configuration in vcswap

[sasuweh]

Maybe another stupid question, but how do I bring up the vfw configuration? I don't see any way to switch.

[jagabo]

Start -> All Programs -> ffdshow -> VFW Configuration. Decoder tab, Codecs in the left pane. Set DV to Disabled in the right pane.

[sasuweh]

Okay, here's where I'm at. I've disabled ffdshow DV under VFW Configuration. Now, when I load the DV AVI files into VirtualDub, it doesn't even have a listing for FourCC code. or a Pixel Format. All it says under Decompressor is:

Cedocida DV Codec (SD format) v0.2.2

If I open the same DV file in VirtualDubMod, it gives a little bit more info. Along with decompressor, it lists:

FourCC code: FourCC: [dvsd] Sony Digital Video (DV)

Mediainfo also listed the DV files as Digital Video (Sony). But when I try to play the DV files in VirtualDub or VirtualDubMod, it says:

"VideoSourceAVI error: The source image format is not acceptable. (error code - 2)". Below is a screenshot of the current Cedocida configuration." So right now the AVIs won't even play at all in VirtualDub.



Now, at this point, I'd like to not worry about editing the files in VirtualDub - that seems like extra steps at this point that I may not even want to do. If I just want to be able to get the DV AVIs into CCE using as few conversions as possible, how would I configure Cedocida? Jagabo, you mentioned above,

"If you open a DV AVI directly in VirtualDub you can use File -> File Information to see what VFW decoder is the default (unless you've set VirtualDub to use its internal DV decoder). AviSource() in AviSynth should use that same decoder."

and poisondeathray said,:

If you are using avisynth, you should set your decoder to output YUY2, otherwise you will incur YUY2=>RGB=>YUY2=>YV12 conversion . If you set it to output YUY2, you won't incur that extra RGB conversion. Since this is NTSC DV, you shouldn't even need ConvertToYUY2(interlaced=true), because it should already by YUY2 if the decoder is set properly ."

So what should I do in the config window for Cedocida to allow the DV files to run in CCE (and also play in VirtualDub)? Also, I'm worried about the Chroma problems that I have been reading about where most decoders incorrectly assume progressive files and mess up the chroma. Will Cedocida avoid that?

[poisondeathray]

That's the way I have it setup for the DEcoder , and there are no chroma issues

If you wanted to force a colorspace, you could uncheckmark the various output colorspace boxes . E.g. you can uncheckmark everything except YUY2 for the output format. This would mean you allow CCE to do the YUY2 => YV12 interlaced conversion (which works fine, it does it properly)

I would still use avisynth to feed CCE, that way you can adjust the levels and do any filtering in the same colorspace. You can double check the colorspace by using info() or looking at the bottom info panel in avsp

[sasuweh]

Okay, I tried unchecking all output colorspace boxes except YUY2 under Output:



When I try to play the DV files in VirtualDub now, it says:

"The decompression codec cannot decompress to an RGB format. This is very unusual. Check that any "Force YUY2" options are not enabled in the codec's properties."

Okay, I'm guessing that makes sense because I have just told Cedocida not to output anything but YUY2, correct? But if I go back and check all the checkboxes again (RGB24,RGB32,YV12), then VirtualDub just says the same thing I wrote above:


"VideoSourceAVI error: The source image format is not acceptable. (error code - 2)".

So now I can't seem to play the DV files in VirtualDub no matter what I set. Something must be wrong here.

[poisondeathray]

are you loading it directly into vdub?

I have it set now to YUY2 only output and it work fine here

what version of vdub ? close & restart it

[sasuweh]

Yes, I'm loading it directly into vdub via file>open video file. I have version 1.9.10 (build 32839/release). For vdubmod I have 1.5.10.2 (build 2540/release).

If I select "prefer internal video decoders" in vdub then it plays okay, but if I leave it unselected to use Cedocida then it just gives the error message when trying to play.

I've tried to close and restart it repeadedly after each change, still no good. I even unzipped a fresh version of vdub to start fresh, same thing.

[poisondeathray]

i get that error with vdubmod when yuy2 is forced, but it's ok when set to default

vdub works either way for me. Not sure what's happening

[sasuweh]

Okay, I have some new information that is totally baffling to me now. I have 2 different sets of these DV AVI files. One set is the original VHS captures which I made using Pinnacle Studio 12 and Dazzle. The other set is the same set of captures, but I used Studio to trim the beginning and ends and export the trimmed DV files so I would have the neatly trimmed DV AVIs to work with. The files that will not play now in vdub are the trimmed versions. If I open the original captures, untrimmed, they play fine now.

So what is going on with these trimmed versions? I asked last week over at the Pinnacle Studio forums about exporting the trimmed DV files and I was told by a couple different people that as long as I only made trims, I could just select "DV AVI" under file export and it would export the DV AVIs untouched other than being trimmed. I am beginning to wonder now of that is truly the case, or did these trimmed files go through some type of processing? There is something definitely different between the original captures and the trimmed versions if vdub won't play the trimmed but will play the originals.

I am now very hesitant to use these trimmed files, and I would feel better using the original captures. However, how could I trim them without affecting the quality and also get them into CCE to output for DVD?

[poisondeathray]

I don't use Pinnacle Studio, so I cannot comment. Trimming shouldn't cause a complete re-encode because DV-AVI is I-frame only. If you added other effects like color correction, then they might have been re-encoded. You can even trim in vdub (using direct stream copy mode)

Can you open the unprocessed and processed files in gspot? to see if there is a difference. Some programs may output type1 vs. type2 dv, but AFAIK vdub can open both kinds. There might be something else it reveals

[sasuweh]

Here's what Gspot says about the original file:

Note: 7.71 GB unneeded bytes at end of file
DV Type 2 AVI
Multipart OpenDML AVI (5 parts)
107208 frames in first part, -2 frames follow)

And the trimmed:

Note: 7.50 GB unneeded bytes at end of file
DV Type 2 AVI
Multipart OpenDML AVI (5 parts)
106013 frames in first part, -2 frames follow)

and MediaInfo says the same about both versions:

AVI (OpenDML)
28.8 Mbps, 720*480 (1.500), at 29.970 fps, Digital Video (Sony) (NTSC)
Frame Rate Mode - Constant
Frame Rate - 29.970 fps
Standard - NTSC
Resolution - 24 bits
Colorimetry - 4:1:1
Scan Type - Interlaced

I already racked my brain worrying about the "unneeded bytes" at end of file warning on spot, it looks like it's just a glitch on Gspot's end (I hope).

Maybe I'm better off trimming in vdub then and using direct stream copy mode? I'm giving that a try now. It looks like cutting is very simple to do. I have read a few things I should do when trimming in vdub:

1) always select cuts on a keyframe. (It looks like all the frames in my DV AVis are k frames, is that right? All the frames seem to have a "k")

2) make sure to select direct stream copy for both audio and video

3) make sure to uncheck the "Enable audio/video interleaving" option under audio and make sure the audio track delay setting is at 0

Are these correct? And does it matter what Cedocida is set at while trimming in vdub?

[jagabo]

Yes, all frames are keyframes in DV AVI. It doesn't matter what Cedocida's settings are when using Direct Stream Copy mode -- Cedocida will only be used to decompress the video for display. The actual editing will simply copy compressed frames from the input file to the output file.

I don't know about that third recommendation. I leave the audio set to default -- interleave every 1 frame, 0 delay.

The YUY2 problem (unable to preview with a DV codec set to output YUY2) is a longstanding bug in VirtualDub, peculiar to DV AVI files. If you use AviSynth to open the file and pass YUY2 data (ie, a simple AviSource() script) it previews the file just fine. It also previews other YUY2 format sources just fine. The bug only effects the preview function, and only DV AVI files. Otherwise the video is handled properly.

[sasuweh]

Well, I tried opening the problematic trimmed files in vdub using this AviSynth script:

AviSource("C:\file.avi")
ConvertToYUY2(interlaced=true)

but it still won't open. I get the following error:

"Avisynth open failure:
AviSource: Could not decompress frame 0"

Using the same above script on the original captures or the file i just trimmed in vdub opens fine. There must be something wrong with those trimmed file, no? It doesn't matter - I can easily trim them again in vdub.


I'm really sorry to be asking a thousand questions. I really appreciate all this information. You've all been very very helpful in helping me understand these things. I think I'm almost set with what I need to do (almost).

I just trimmed one of the original DV captures in vdub, and output it using direct stream copy following the tips I listed above. I hope those are correct. The trimmed avi now opens up fine in vdub as long as I have ticked all decoder boxes in Cedocida.

If I leave all output boxes ticked in Cedocida and open the original file and the newly trimmed file in avspmod as mentioned above, here is what it says about the files under "video information":

Video:
Frame Size: 720x480
Colorspace: YV12
Field or frame based: Frame Based
Parity: Bottom Field First

It seems by unticking different output boxes in Cedocida it changes the colorspace results. So I guess Cedocida is working. How come it selects YV12 when all the boxes are ticked?



So, I guess, the only thing I'm really a bit confused about now is if I should import the newly trimmed DV AVIs directly into CCE or should I use the script. You mentioned above that I could either:

1) "uncheckmark everything except YUY2 for the output format. This would mean you allow CCE to do the YUY2 => YV12 interlaced conversion (which works fine, it does it properly)"

or

2) Use the AviSynth script (which would allow me to use filters like Peachsmoother). If I do this, do I need to uncheck all the output boxes except YUY2 in Cedocida? you mentioned earlier "If you are using avisynth, you should set your decoder to output YUY2, otherwise you will incur YUY2=>RGB=>YUY2=>YV12 conversion . If you set it to output YUY2, you won't incur that extra RGB conversion. Since this is NTSC DV, you shouldn't even need ConvertToYUY2(interlaced=true), because it should already by YUY2 if the decoder is set properly."

Either way I'll be avoiding the nasty chroma problems during conversion, right? What about the "YV12 chroma sampling" options in Cedocida? Is that what affects the chroma problems that I've been reading about? I have a choice of "DV, MPEG 2 non interlaced, and MPEG 2 interlaced".
So this is the script if i do it that way:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\PeachSmoother.dll")
AviSource("C:\file.avi")
ConvertToYUY2(interlaced=true)
PeachSmoother()

Is that correct? And you said that if I force output to YUY2 in Cedocida, I wouldn't even need the "ConvertToYUY2(interlaced=true" part because it is done already by Cedocida?



It also seems that most sites recommend leaving "Enable Audio/Video Interleaving" checked in vdub. No wonder I'm confused.

[jagabo]

Well, I tried opening the problematic trimmed files in vdub using this AviSynth script:

AviSource("C:\file.avi")
ConvertToYUY2(interlaced=true)

but it still won't open. I get the following error:

"Avisynth open failure:
AviSource: Could not decompress frame 0"

I don't know what's going on there. Maybe you should post a short non-working sample.

It seems by unticking different output boxes in Cedocida it changes the colorspace results. So I guess Cedocida is working. How come it selects YV12 when all the boxes are ticked?

(I verified that you get YV12 in this circumstance.) AviSource() must tell Cedocida it prefers YV12 over the other colorspaces. This is a mistake, in my opinion. YUY2 is a closer match to NTSC DV. I guess they're assuming you're going to make DVDs.

1) "uncheckmark everything except YUY2 for the output format. This would mean you allow CCE to do the YUY2 => YV12 interlaced conversion (which works fine, it does it properly)"

That will work.

2) Use the AviSynth script (which would allow me to use filters like Peachsmoother). If I do this, do I need to uncheck all the output boxes except YUY2 in Cedocida?

If you only have YUY2 as an output option you know you will get YUY2. Otherwise Cecocida and the caller will negotiate a colorspace. You don't necessarily know what colorspace they will decide on.

Either way I'll be avoiding the nasty chroma problems during conversion, right?

Yes.

What about the "YV12 chroma sampling" options in Cedocida? Is that what affects the chroma problems that I've been reading about? I have a choice of "DV, MPEG 2 non interlaced, and MPEG 2 interlaced".

You should not be using YV12 for NTSC DV.

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\PeachSmoother.dll")
AviSource("C:\file.avi")
PeachSmoother()

With Cedocida set to output only YUY2 that's all the script you need (I'm assuming PeachSmoother works in YUY2).

[sasuweh]

Sorry for another big post, I'm still trying to clear things up .So if all colorspaces are checked in Cedocida, AviSource() demands YV12 is preferred, is that right?

Okay, so whether I go with option 1 or 2, either way I will want Cedocida to look like this:



right? That way:

- If I go with option 1, Cedocida will not do anything, and CCE will do the conversion as required.
- If I go with option 2 (the script), then Cedocida will automatically convert the colorspace to YUY2 in CCE.

Is that right?

I think I'm finally understanding about these different colorspaces. 4:1:1 is it's own colorspace, right? I wasn't understanding that. I know my captures are NTSC DV 4:1:1. NTSC DV is always 4:1:1 and YUY2 is 4:2:2. Is that correct? With Cedocida set as in the screenshot above (force YUY2), opening the captures in media player classic using the "info()" script shows the colorspace listed as YUY2. I was thinking previously that the original DV was captured in YV12, because that was what the "info()" script was showing. Now it looks like it was just displaying whatever the codec I originally had was decoding it to, and maybe it was set to YV12, I don't know. So if my DV files are really DV NTSC 4:1:1, then that has really nothing to do with YV12, right? I read that CCE will end up converting it to YV12 though for the >mv2 though because DVD is YV12 (4:2:0). Is that right?

So with Cedocida set like the screenshot above, I can use the script you posted, without the "ConvertToYUY2(interlaced=true)" line?

I think I might actually be getting this now (maybe).


I was also reading a DV to DVD guide on Doom 9, and the guide mentioned a couple things I haven't heard:

NTSC issues

As NTSC DV uses the 4:1:1 colorspace, there is an issue with the conversion to the 4:2:0 colorspace used in DVD MPEG-2. To address this issue use trbarry's filter ReInterpolate411().

To get improved color conversion, simply add a
ReInterpolate411()
command below the AviSource or DirectShowSource command, respectively.

It also says elsewhere on their site:

Reinterpolate411 should be used on all NTSC DV to DVD conversions when using AviSynth, no matter if you have a DV codec with the chroma upsampling bug or not.

If DV codec has Chroma bug:
AVISource("C:\file.avi").FixBrokenChromaUpsampling
ReInterpolate411()
#Other filters

If DV codec has no Chroma bug:
AVISource("C:\file.avi")
ReInterpolate411()
#Other filters


"This is a fast simple filter to correct the improper 4:1:1 => 4:2:2 conversion that seems to occur with some DV/4:1:1 codes when used with Avisynth. It assumes the odd chroma pixels are duplicates and discards them replacing them with the average of the two horizontally adjacent even chroma pixels. It doesn't matter whether the clip is interlaced though it must be in YUY2 format for Avisynth 2.5. There are no parameters, and currently no readme file."


This info is from 2004. Should I do that? And with Cedocida I don't have to worry about the chroma bug right?


They also suggested this as a script for AviSynth and CCE:

Interlaced DV

code:AviSource("C:\MyDVFiles\DV_type2_file.avi")
SeparateFields()
odd=SelectOdd.Convolution3D(1, 6, 10, 6, 8, 2.8, 0)
evn=SelectEven.Convolution3D(1, 6, 10, 6, 8, 2.8, 0)
Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd()

This way each field will be filtered separately, thus the interlaced structure will be preserved. Uncheck the Upper field first box in CCE (field order correction is performed in AviSynth script).


Seems a bit more complicated than my simple script. Is it necessary or should I just leave that alone?
Please correct me if I still have things confused at all.

It also said on that forum that the Sony DV codec only operates in RGB colorspace. That doesn't mean that my DV files are RGB does it? That' doesn't sound right.

"The Sony DV codec only operates in RGB colorspace. This is a major disadvantage because a color conversion has to be done for some AviSynth filters and input into CCE. Other DV codecs such as MainConcept, Canopus, and even Microsoft allow you the option to also operate in YUY2 colorspace. This will eliminate the need for colorspace conversions and prevent color shifts and clipping as a side effect of the colorspace conversions."

[jagabo]

So if all colorspaces are checked in Cedocida, AviSource() demands YV12 is preferred, is that right?

Yes, that's what I found.

I also noticed I have the Canopus DV codec on my computer. Will that cause problems? Should that be removed? Not sure if it came with something else or maybe a codec pack at some point.

It shouldn't cause problems except that may make it difficult to tell which DV codec is being used. Keep in mind there are two major video subsystems in Windows: VFW and DirectShow. Editors tend to use VFW, media players tend to use DirectShow. Each has their own codecs. Generally, VFW programs can't use DirectShow codecs and vice versa. To further complicate matters, if you are using a 64 bit version of Windows there are separate codecs for 32 bit and 64 bit programs. So you have four video subsystems. And any particular program can use its own private codecs. So things can get confusing.

Okay, so whether I go with option 1 or 2, either way I will want Cedocida to look like this:


[Attachment 3674 - Click to enlarge]

right?

Yes. It's not germane to this discussion but the YV12 chroma subsampling setting in the Encoder options should be left at DV.

That way:

- If I go with option 1, Cedocida will not do anything, and CCE will do the conversion as required.

Yes. Except Cedocida will perform the conversion from NTSC DV's internal 4:1:1 chroma subsampling to YUY2's 4:2:2 chroma subsampling.

- If I go with option 2 (the script), then Cedocida will automatically convert the colorspace to YUY2 in CCE.

More properly, Cedocida will convert to YUY2 before giving the frames to CCE.

I think I'm finally understanding about these different colorspaces. 4:1:1 is it's own colorspace, right?

4:1:1 is YUV samples with one U and one V sample for every four Y samples horizontally. Think of it like this: a 720x480 4:1:1 frame has a 720x480 Y (luma, grayscale) plane and 180x480 U and V (color) planes.

4:2:2 is YUV samples with one U and one V sample for every two Y samples. A 720x480 4:2:2 frame has a a 720x480 Y plane and 360x480 U and V planes. YUY2 refers to a specific order of the Y, U, and V samples (U and V samples are interleaved between Y samples: Y U Y V Y U Y V...).

4:2:0 is a type of 4:1:1 chroma subsampling but the colors are distributed differently. There are still one U and V samples for every four Y samples, but they come from 2x2 arrays of the image, not 4:1 arrays. Ie, a 4:2:0 frame has a a 720x480 Y plane and 360x240 U and V planes.

Interlaced 4:2:0 is similar to progressive 4:2:0 in that it has one U and one V for every four Y samples. But the U and V planes are interlaced. So a 720x480 interlaced frame contains two 720x240 images. Each of those images has a 720x240 Y plane and 360x120 U and V planes.

I wasn't understanding that. I know my captures are NTSC DV 4:1:1. NTSC DV is always 4:1:1 and YUY2 is 4:2:2. Is that correct?

Yes.

So if my DV files are really DV NTSC 4:1:1, then that has really nothing to do with YV12, right?

Right.

I read that CCE will end up converting it to YV12 though for the >mv2 though because DVD is YV12 (4:2:0). Is that right?

Yes.

So with Cedocida set like the screenshot above, I can use the script you posted, without the "ConvertToYUY2(interlaced=true)" line?

Yes.

I was also reading a DV to DVD guide on Doom 9, and the guide mentioned a couple things I haven't heard:

NTSC issues

As NTSC DV uses the 4:1:1 colorspace, there is an issue with the conversion to the 4:2:0 colorspace used in DVD MPEG-2. To address this issue use trbarry's filter ReInterpolate411().

To get improved color conversion, simply add a
ReInterpolate411()
command below the AviSource or DirectShowSource command, respectively.

I'll have tor read up on what that function does. I suspect it "smooths" the U and V samples. Some decoders will duplicate samples rather than interpolate when converting 4:1:1 to 4:2:2. For example, say four consecutive 4:1:1 U values are 100, 110, 120, 130. To turn that in to 4:2:2 you have to turn those four samples into eight. A simple duplication algorithm will give 100, 100, 110, 110, 120, 120, 130, 130. A smoother algorithm might give 100, 105, 110, 115, 120, 125, 130, 135.

They also suggested this as a script for AviSynth and CCE:

Interlaced DV

code:AviSource("C:\MyDVFiles\DV_type2_file.avi")
SeparateFields()
odd=SelectOdd.Convolution3D(1, 6, 10, 6, 8, 2.8, 0)
evn=SelectEven.Convolution3D(1, 6, 10, 6, 8, 2.8, 0)
Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd()

This way each field will be filtered separately, thus the interlaced structure will be preserved. Replace AviSource with DirectShowSource like above if your DV AVI is type-1.

Seems a bit more complicated than my simple script. Is it necessary or should I just leave that alone?

If you're going to do some noise reduction you have to seperate the fields (or use special filters that are interlace aware). Otherwise information from the two fields will contaminate each other. Remember, in interlaced digital video each frame contains two separate half pictures (aka fields), intended to be viewed separately.

It also said on that forum that the Sony DV codec only operates in RGB colorspace. That makes me even more confused since mediainfo says my files are Sony. Wouldn't that mean my DV files are RGB?

No, the color inside the DV data is always YUV 4:1:1. I don't know the Sony DV decoder but I suspect whoever said that is claiming it always decompresses to RGB. I know the Pansonic DV codec does.

[sasuweh]

Okay, so I'm going to use option 2, the script. That will allow me to use some filters that I would like to use. I am trying not to overdo the use of filters. I have read in several places that Guavacomb and PeachSmoother are often used well together. Especially since these are transfers from VHS tapes, I would like to try that. But, I'm a bit unsure of how to assemble the script in order to use these filters. I already have the PeachSmoother and Guavacomb dll's in the Avisynth plugins folder. So here is how I think my script will be looking:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\GuavaComb.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\PeachSmoother.dll")
AviSource("C:\file.avi")
SeparateFields()
ConvertToYUY2(interlaced=true) << I don't need this now since I'ce got Cedocida set up, correct?
odd=SelectOdd.GuavaComb(mode = "NTSC")
evn=SelectEven.GuavaComb(mode = "NTSC")
odd=SelectOdd.PeachSmoother()
evn=SelectEven.PeachSmoother()
Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd() << I should leave this out, right (read below, last paragraph)

I'm not sure if I've got things in the right order, especially interleave, weave, doubleweave, and separatefields. And are my GuavaComb and Peachsmoother filters set up right for defaults (is odd, evn lines correctly formatted)?

From what I've read, Peachsmoother works best if it comes right after GuavaComb.

From ArsTechnica:
" It is recommended that Peach Smoother come immediately after comb filters (like Guava Comb), but before de-interlacing/IVTC filters."

I've read in a couple places that Reinterpolate411 is not needed when using Cedocida, so I guess that can be left out.


My captures are BFF (bottom field first) . I've read that All DV is BFF, and Avisynth defaults to AssumeBFF. In CCE I will be leaving the "Offset Line" to 1, since I have BFF. It seems DoubleWeave.SelectOdd() changes the field order, in this case from BFF to TFF. So that line is unnecessary, isn't it?

[jagabo]

I don't use either of those filters so I can't say much about them. But there are problems with your script. Keep in mind that when you don't specify a video stream AviSynth defaults to using the video stream called "last". For example, from your script:

AviSource("C:\file.avi")
SeparateFields()

is equivalent to:

last = AviSource("C:\file.avi")
last = SeparateFields(last)

The first line is creating a named stream called "last". The second line takes "last" runs it through SeparateFields() and creates a new stream called "last", replacing the old one.

You don't need the ConvertToYUY2() because your video is already YUY2. And if you did need it, you wouldn't use the interlaced=true argument because after SeparateFields() your video is no longer interlaced. If you used it before SeparateFields() you would use the interlaced=true argument.

Regarding:

odd=SelectOdd.GuavaComb(mode = "NTSC")
evn=SelectEven.GuavaComb(mode = "NTSC")
odd=SelectOdd.PeachSmoother()
evn=SelectEven.PeachSmoother()

In the first line you are creating a named stream, "odd", from "last". In the third line you are doing the same thing. So you are throwing out the results of the first line. The same is happening with the "even" stream. Your script should be modified like this to use both GuavaComb() and PeachSmoother():

odd=SelectOdd.GuavaComb(mode = "NTSC")
evn=SelectEven.GuavaComb(mode = "NTSC")
odd=PeachSmoother(odd)
evn=PeachSmoother(even)

Or simpler:

odd=SelectOdd.GuavaComb(mode = "NTSC").PeachSmoother()
evn=SelectEven.GuavaComb(mode = "NTSC").PeachSmoother()

And, yes, DoubleWeave.SelectOdd() was used to reverse the field order. Since you are encoding with CCE set to TFF and using the offset to reverse the field order, you don't need the DoubleWeave.SelectOdd().

So your final script should look like:

AviSource("C:\file.avi")
SeparateFields()
odd=SelectOdd().GuavaComb(mode = "NTSC").PeachSmoother()
evn=SelectEven().GuavaComb(mode = "NTSC").PeachSmoother()
Interleave(evn,odd)
Weave()

I tossed in the "()" after SelectOdd and SelectEven. It's not necessary but I find it's easier to read with consistent syntax.

[sasuweh]

So I don't need the "LoadPlugin" lines at the beginning then?

Thanks a million for helping with the script. I had a feeling it wasn't going to be quite right. Regarding CCE set to TFF. I have been reading lots of confusing statements regarding how to set CCE for this. There are two things to pay attention to here:

1) "Output Top Field First" can be checked or unchecked
2) "Offset Line" can be 0 or 1

From the CCE faq:

CCE 2.5
So here is the rule of thumb: Always uncheck "Upper Field First" unless your video is interlaced AND bottom field first. Progressive material is always top field first.

CCE 2.67
Again here is the rule of thumb: Always set "Offset Line" to 0 unless your video is interlaced AND bottom field first in which case you set it to 1. Progressive material is always top field first.

So, since my captures are bottom field first, it sounds like I would want"

1) "Output Top Field First" - checked
2) "Offset Line" - 1

This would give top field first output I think. Is that what I would want?

And if it turns out that I don't like how the filters are working, and just want to skip them, then I wouldn't need any of the "SeparateFields, Interleave, or Weave" lines, right? If I just wanted to get the file into CCE as is, my script would simply be:


AviSource("C:\file.avi")

and nothing else?

There's 2 other things I'm a bit unsure how to set in CCE. Under Audio Sampling Frequency, there is a box that says "Sync with Video" which is checked by default. I will be letting CCE output Wave so I can run it through BeHappy afterwards, so do I want to leave this box checked or unchecked?

And, under framerate, 29.97 is checked, and there is a box called "Rate conv" that is checked by default. Should I just leave it checked or does it not matter since my framerate is already 29.970?

Also, under pulldown, 2:3 is selected by default, and Inverse 3:2 pulldown is unchecked. I'm pretty sure I don't want to mess around with anything like that? I think my captures are mostly telecine - I'm new to all this, but when going through the frames in vdub, I see a pattern of 5 frames - 2 have lines throughout the image, then 3 frames are clear, then 2 have lines, then 3 clear, etc. So I think that means they are telecine? Though 1 video looks like it is a mix, because some scenes are all lines, with no clear frames. I think it is best to just keep my DVDs interlaced though, so I should just skip all that, and keep it simple, right?

[poisondeathray]

So I don't need the "LoadPlugin" lines at the beginning then?

Some dll's are autoloading when placed in the plugins folder, some need to be loaded manually. If you get an error message about missing plugin then load it manually

Thanks a million for helping with the script. I had a feeling it wasn't going to be quite right. Regarding CCE set to TFF. I have been reading lots of confusing statements regarding how to set CCE for this. There are two things to pay attention to here:

1) "Output Top Field First" can be checked or unchecked
2) "Offset Line" can be 0 or 1

From the CCE faq:

CCE 2.5
So here is the rule of thumb: Always uncheck "Upper Field First" unless your video is interlaced AND bottom field first. Progressive material is always top field first.

CCE 2.67
Again here is the rule of thumb: Always set "Offset Line" to 0 unless your video is interlaced AND bottom field first in which case you set it to 1. Progressive material is always top field first.

So, since my captures are bottom field first, it sounds like I would want"

1) "Output Top Field First" - checked
2) "Offset Line" - 1

This would give top field first output I think. Is that what I would want?

I think so. I would PM "manono" and ask him. I know newer versions like SP2 have that fixed (i.e. it behaves as you would expect it to. For a BFF source, you would uncheck TFF and offset 0 using the newer versions, those older versions were confusing but I think those instructions above are what you should do )

And if it turns out that I don't like how the filters are working, and just want to skip them, then I wouldn't need any of the "SeparateFields, Interleave, or Weave" lines, right? If I just wanted to get the file into CCE as is, my script would simply be:


AviSource("C:\file.avi")

and nothing else?

yes

There's 2 other things I'm a bit unsure how to set in CCE. Under Audio Sampling Frequency, there is a box that says "Sync with Video" which is checked by default. I will be letting CCE output Wave so I can run it through BeHappy afterwards, so do I want to leave this box checked or unchecked?

most people don't use CCE for audio, so you can uncheck audio if you are using another audio program. I've heard there are bugs when you use CCE for audio

And, under framerate, 29.97 is checked, and there is a box called "Rate conv" that is checked by default. Should I just leave it checked or does it not matter since my framerate is already 29.970?

Also, under pulldown, 2:3 is selected by default, and Inverse 3:2 pulldown is unchecked. I'm pretty sure I don't want to mess around with anything like that? I think my captures are telecine - I'm new to all this, but when going through the frames in vdub, I see a pattern of 5 frames - 2 have lines throughout the image, then 3 frames are clear, then 2 have lines, then 3 clear, etc. So I think that means they are telecine? I think it is best to just keep my DVDs interlaced though, so I should just skip all that, right?

You should post a sample . If you have a telecined source, it would be better to encode progressive and use soft pulldown flags. Encoding interlaced will always give lower quality. If you want a simple guide to examining fields
http://neuron2.net/faq.html#analysis

[manono]

He could always read the CCE manual for clear instructions as to how to handle the Output Top Field First box. For the older versions he quoted, the instructions for the two versions weren't only confusing but contradictory. For my CCE 2.70.02.12 it clearly states:

3.10 Output top field first stream
Specify the field order of output stream. If it is selected, an output
stream is top field first. If not, an output is bottom field first. This
setting works on MPEG-2 output.
It is important to set correct field order, because incorrect field
order causes stuttered motion. To set field order correctly, you have
to care about Offset line setting.
• If the field order is same between the source and the output,
Offset line should be 0 or even number.
• If the field order is different between the source and the output,
Offset line should be 1 or odd number.

For my TFF interlaced streams I have the box checked and the Offset line at zero. For BFF sources, assuming you want to keep them BFF, uncheck the box and keep the Offset line at zero.

[sasuweh]

Okay, I read that guide poisondeathray posted. My captures are definitely BFF. I am posting a couple short clips here that I know may be a mix of telecined and other. It's from an old VHS never released on DVD I'm trying to preserve. The first clip, test1, was material shot on video. The second clip, test2, was material shot on film. The whole video is a mix of different footage.

The manual was very confusing to me regarding the Output Top Field First box. I honestly don't know if there is any benefit to outputting top field first or bottom field first. I don't really know what difference it will make. Since my captures are BFF, is it preferable to keep it that way, or output top field first?

I was always under the impression that all DVDs are interlaced, but now I know they can be interlaced or progressive. I would of course like to have these DVDs looking as good as possible. I thought it was just standard (and easier) to output as interlaced. But if progressive is recommended, I'll do that. I'm just unsure how to go about this.

[sasuweh]

Here's the 2nd clip.

[jagabo]

Test1.avi is interlaced, BFF. Test2.avi is telecined film (interlaced, BFF).

Sorry about leaving the LoadPlugin() line out earlier. I have all my plugins in AviSynth's Plugins folder so they load automatically.

Note CCE's output field order setting doesn't effect the video itself, only what is flagged in the MPEG file. It's either marked TFF or BFF. The Offset Line value determines whether the field order is inverted by the encoder. 0 leaves it unchanged, 1 inverts it. (The field order is reversed by shifting the frame up by one scanline.) So with your BFF sources you should flag the video BFF (disable Output TFF) and set Offset Line set to 0. Or you can flag it TFF and set Offset Line to 1.

[sasuweh]

Okay, so I think I will check the "Output Top Field First" and set the Offset Line to 1. I'm pretty sure that is what had worked previously when I ran an early test. I tried changing the settings and ended up with choppy video output.

I keep my plugins in the AviSynth plugins folder. So if I do that I can skip the "LoadPlugin" lines?

So after seeing some samples, with a video that is mixed like this, am I better off to leave the DVD interlaced? Or am I still better off encoding progressive and using soft pulldown flags as mentioned above? I'm not sure how to do that part. I think most of my VHS tapes are telecined (I'm only doing about 6 or 7), but I would have to double-check them all to be sure.

[jagabo]

Okay, so I think I will check the "Output Top Field First" and set the Offset Line to 1. I'm pretty sure that is what had worked previously when I ran an early test. I tried changing the settings and ended up with choppy video output.

If you changed both settings you should have gotten smooth results. If you changed only one you would get very fast-jerky motion.

I keep my plugins in the AviSynth plugins folder. So if I do that I can skip the "LoadPlugin" lines?

Yes. You'll get an error message if a filter doesn't auto load.

So after seeing some samples, with a video that is mixed like this, am I better off to leave the DVD interlaced? Or am I still better off encoding progressive and using soft pulldown flags as mentioned above? I'm not sure how to do that part. I think most of my VHS tapes are telecined (I'm only doing about 6 or 7), but I would have to double-check them all to be sure.

Test1 should be left interlaced. Test2 can be inverse telecined back to the original film frames, then encoded progressive with pulldown flags. That will give less MPEG compression artifacts because 23.976 fps progressive compresses better than 29.97 hard telecine. Try:

AviSource("filename.avi")
TFM()
TDecimate()

You'll need the TIVTC package: http://bengal.missouri.edu/~kes25c/

[sasuweh]

Allright, so any transfers that match test1 (true interlaced) should stay interlaced and I should just make the DVD interlaced? And any transfers that match test2 (telecined) should be inverse telecined, then encoded as progressive with pulldown flags? After reading, I understand it that by inverse telecining the video in clip2 it would be reverting the footage back to progressive 23.976, saving video frames, then the pulldown flag would tell the player to create the missing frames on-the-fly to bring it to 29.970 fps? Am I making sense?

So then what would I need to select in CCE when working with telecined video? Under "pulldown" there's choices for 2:3 and 3:2, along with Inv 3:2 pulldown and Auto 2-pass detection, plus a "Deinterlacing" option under Preprocess and a "Progressive Frame" option under Picture Quality. And the TFM()/TDecimate() lines you mentioned above, would that be used only for a video like the one I posted that is mixed (both interlaced and telecined), or would that be used for videos that are all telecined also? Do those additional lines handle both types together in 1 video like I posted?

Sorry for another batch of questions, but this telecine/progressive stuff is all new to me and I don't want to use those additional lines and then set things in CCE incorrectly and not know it.

I've read that CCE SP2 3:2 detection (in 2 pass) works great for hard telecine & hybrid sources. I also read "
To sum it up cce's auto detect is safe but not very effective, it also adds quite a bit of encoding time. TFM().TDecimate() is very effective but not always safe with hybrid content. TFM().TDecimate(hybrid=1) is very effective and safer with hybrid content. For auto detection cce sp2's would be good, however I would still opt for the manual setting to ivtc with probably TFM().TDecimate(hybrid=1) just to be a little safer.".

Another comment said, "If you try to inverse telecine a hybrid source you will get poor motion (jumpiness) in the non-telecined sections because they will be improperly decimated. It's really a bad idea."

I read that using TFM()/TDecimate(hybrid=1) creates a non-compliant DVD. Again, I don't really know anything about this, just what I've been reading.

Somebody else mentioned using TFM(pp=0) also. "This is the IVTC operation, TFM identifies matches, and TDecimate removes the duplicate frames. The "pp" parameter is post processing, which I often turn off simply because it can detect many correct frames as "problem" frames and attempt to deinterlace them (which is DIFFERENT than IVTC)".


It looks like I have 3 choices:

1) Just encode as interlaced and leave it as interlaced/telecined >> I know, it's lazy and wrong, but at least I don't run the risk of screwing things up with the wrong settings and losing frames.

2) Use TFM(pp=0) / TDecimate(hybrid=1) >> but I'm not sure what to set in CCE in conjunction with this and where it belongs in my previous script

3) Just set CCE to do it using "Inv. 3:2 pulldown" & "auto 2-pass detection" >> I wish I could just trust CCE to do it, but I don't know how to set some settings (2:3 or 3:2 under pulldown, "progressive" under picture quality, audio sync with video -- and "rate conv" under Framerate seems especially confusing to me, and I've read about it in the manual)

It's all so very confusing. Thanks for being so patient.

[jagabo]

Allright, so any transfers that match test1 (true interlaced) should stay interlaced and I should just make the DVD interlaced?

Yes.

And any transfers that match test2 (telecined) should be inverse telecined, then encoded as progressive with pulldown flags?

Yes.

After reading, I understand it that by inverse telecining the video in clip2 it would be reverting the footage back to progressive 23.976, saving video frames, then the pulldown flag would tell the player to create the missing frames on-the-fly to bring it to 29.970 fps? Am I making sense?

Yes.

So then what would I need to select in CCE when working with telecined video? Under "pulldown" there's choices for 2:3 and 3:2, along with Inv 3:2 pulldown and Auto 2-pass detection, plus a "Deinterlacing" option under Preprocess and a "Progressive Frame" option under Picture Quality.

Select Progressive Frame, and either 2:3 or 3:2 pulldown. Those are short for 2,3,2,3,2,3,2,3... and 3,2,3,2,3,2,3,2... field repeats. Ie, the only difference is whether you start repeating with a 2 or a 3.

To understand what's going on here you need to know that TV was originally broadcast as 60 fields per second. When black and white TV was developed they wanted the smoothness of motion and low flicker of 60 pictures per second. But that required too much bandwidth for broadcast. Reducing the frame rate to 30 pictures per second would fit within the bandwidth requirements but it was jerky and flickered too much. So they decided to compromise, broadcast 60 half pictures (fields) per second. That gave the benefits of 60 different pictures per second but only required the bandwidth of 30 pictures per second. During each field only half the scanlines of the screen are displaed, every other scan line. So during one field all the even numbered scanlines (0,2,4,6..., the top field) are displaed, then 1/60 of a second later all the odd numbered scanlines (1,3,5,7..., the bottom field) are displayed. By the time one field is being displayed the previous field has faded away (the emissions from phosphors on the CRT screen were designed to decay in about 1/60 second). So you see 60 half pictures every second on the screen. Later, when color TV was introduced, for technical reasons they slowed the field rate down from 60 to 59.94 fields per second. I will use 60, 30, and 24 instead of 59.94, 29.97 and 23.976 for the rest of this discussion.

Think of it like this: a video camera takes 60 snapshots a second. But it only sends half of each snapshot to the TV, always alternating between the top field and bottom fields.

When digital video came around it was convenient to pack two successive fields into a frame. 60 fields per second of broadcast video becomes 30 frames per second of digital video. During a still shot the two fields simply form the two halves of the full picture. That is, the 60 pictures per second the video camera saw were all identical. So you can take two successive fields, weave them together, and you have reconstructed those full pictures. But if something moves during the 1/60 second between fields you will see comb artifacts -- the familiar "interlaced look" of video viewed on a progressive computer screen. TFF and BFF refer to which of the two fields was captured first. A capture device can start with a top field, then weave in the next field, a bottom field, to produce the full frame (TFF). Or it can start with a bottom field, then weave in the next top field to produce BFF frames. When these frames are separated into fields for display you need to know which of the two fields to display first. Ie, You have to keep track of the field order in which they were captured.

So how do you get from the 24 frames per second of film to the 60 fields per second of broadcast TV? First you alternate repeating film frames in a 2x, 3x repeat pattern. (So, on average, every frame is repeated 2.5 times. 2.5 * 24 = 60). Then you pull fields out of those 60 pictures per second just like a video camera would, alternating between top and bottom fields. So the 2:3 or 3:2 notation simply refers to the fact that frames are repeated in a sequence of 2,3,2,3,2,3,2,3,2,3... repeats. The only difference being whether you start with a 2 or a 3. In general, it doesn't matter which. It's offered as an option simply because they can. This process is called "telecine".

When those telecined fields are packaged into digital frames, by weaving pairs of fields into frames, some of those frames will contain two fields from the same film frame (and look like a progressive image). Other frames will contain fields from two different film frames (and will look like an interlaced image). If you step through the 30 fps telecined digital video you will see a long term pattern of 3 progressive looking frames (both fields come from the the same film frame) and 2 interlaced looking frames (each field comes from a different film frame) when there is motion. This is an easy way to determine if you have telecined film vs fully interlaced video (where every frame looks interlaced). It's possible to go through all those fields, pick out pairs that come from the same film frame and weave those back together to restore the original 24 fps film frames. This processes is called an "inverse telecine".

And the TFM()/TDecimate() lines you mentioned above, would that be used only for a video like the one I posted that is mixed (both interlaced and telecined), or would that be used for videos that are all telecined also?

You use TFM().TDecimate() only with telecined film sources. Fully interlaced video (Test1) did not come from 24 fps film so it would be inappropriate to make 24 fps progressive frames from it.

I've read that CCE SP2 3:2 detection (in 2 pass) works great for hard telecine & hybrid sources. I also read "
To sum it up cce's auto detect is safe but not very effective, it also adds quite a bit of encoding time. TFM().TDecimate() is very effective but not always safe with hybrid content. TFM().TDecimate(hybrid=1) is very effective and safer with hybrid content. For auto detection cce sp2's would be good, however I would still opt for the manual setting to ivtc with probably TFM().TDecimate(hybrid=1) just to be a little safer.".

Another comment said, "If you try to inverse telecine a hybrid source you will get poor motion (jumpiness) in the non-telecined sections because they will be improperly decimated. It's really a bad idea."

I read that using TFM()/TDecimate(hybrid=1) creates a non-compliant DVD. Again, I don't really know anything about this, just what I've been reading.

Somebody else mentioned using TFM(pp=0) also. "This is the IVTC operation, TFM identifies matches, and TDecimate removes the duplicate frames. The "pp" parameter is post processing, which I often turn off simply because it can detect many correct frames as "problem" frames and attempt to deinterlace them (which is DIFFERENT than IVTC)".


It looks like I have 3 choices:

1) Just encode as interlaced and leave it as interlaced/telecined >> I know, it's lazy and wrong, but at least I don't run the risk of screwing things up with the wrong settings and losing frames.

2) Use TFM(pp=0) / TDecimate(hybrid=1) >> but I'm not sure what to set in CCE in conjunction with this and where it belongs in my previous script

3) Just set CCE to do it using "Inv. 3:2 pulldown" & "auto 2-pass detection" >> I wish I could just trust CCE to do it, but I don't know how to set some settings (2:3 or 3:2 under pulldown, "progressive" under picture quality, audio sync with video -- and "rate conv" under Framerate seems especially confusing to me, and I've read about it in the manual)

It's all so very confusing. Thanks for being so patient.

The simplest and safest thing to do is simply encode as 29.97 fps interlaced. Hybrid video (mix of 24 fps film source and 30 fps or 60 fields per second video video sources) aren't all that common (most often found in anime). You might with a DVD rip where maybe the opening studio logo is 30 fps progressive and the main movie is 24 fps telecined film. Or a TV recording where the movie is telecined film but that ads are interlaced video. And you often find that movies are telecined into 29.97 fps interlaced frames then edited for broadcast. Those edits can cause discontinuities in the 3:2 repeat pattern, even orphaned fields.

It's worth keeping in mind that IVTC isn't always perfect, even with 100 percent film sources. But TFM().TDecimate() works pretty well and the benefits gained from encoding 24 fps progressive vs 30 fps interlaced is worth the tradeoff.

[sasuweh]

That's an awesome in-depth explanation. Thanks very much for taking the time to explain this in such detail. So you think the benefits gained by doing the IVTC is worth the tradeoff of it perhaps not being perfect? More than anything else I'm just afraid of it deciding to remove the wrong frames, especially on hybrid video like the one I posted. The idea is TFM(pp=0).TDecimate(hybrid=1) is supposed to be a safer setting for hybrid though, right? Would there be any harm in leaving it like that if my source was fully telecined with no true interlaced? I might just give TFM(pp=0).TDecimate(hybrid=1) a try and see how it turns out. The thing is though, I don't think I'd know it if it made some mistakes unless they are major mistakes. Little mistakes would probably go unnoticed, but that's where you say the tradeoff is, right?


Could you tell me then what to set in CCE if I use TFM(pp=0).TDecimate(hybrid=1) to assure a compliant DVD? Obviously I would be leaving "Inv. 3:2 pulldown" unchecked because that's what TFM().TDecimate() is doing, but what about these:


Under pulldown, Inv. 3:2 pulldown is unselected by default, 2:3 is ticked, and "deinterlacing is unticked. . Should I change anything here?




Under framerate, 29.97 is ticked by default for me, and Rate conv is ticked. I'm very confused over what Rate conv will do. And I will be putting out 23.976 right? So what do I do here?




"Sync with video" is checked by default here. I'll just be outputting the Wav to convert in BeHappy, but now I don't know aht to set here because of the inverse telecine.

"Output top field first stream" is checked by default and Offset is set to 1. Again, my captures are BFF. I could just leave it like this?




Under "picture quality", Progressive frame is unchecked and Block Scan Order is set to auto? Do I select Progressive here now?




Lastly, can low stay ticked here or do I want to untick it?



If I know how to set these options to work right using TFM(pp=0).TDecimate(hybrid=1) I think I'll give it a try. Also, how should my script look then? Where do I put this? Here is my script as above:

AviSource("C:\file.avi")
SeparateFields()
odd=SelectOdd().GuavaComb(mode = "NTSC").PeachSmoother()
evn=SelectEven().GuavaComb(mode = "NTSC").PeachSmoother()
Interleave(evn,odd)
Weave()


I can't tell you how helpful this is. I've tried to consult the CCE manual, but it really doesn't seem to help me much, especially since it doesn't address how to set things when using AviSynth.


Out of curiosity thoguh, if the dvd's are left telecined, can't a progresive dvd player apply 3:2 pulldown and inverse telecine as it is playing? Or is that not ideal?