VHS to DVD with Canopus ADVC110 - mostly filter/cleaning questions

[jagabo]

Trim (clip, int first_frame, int last_frame [, bool pad_audio])

Trim trims a video clip so that it includes only the frames first_frame through last_frame. The audio is similarly trimmed so that it stays synchronized. If you pass 0 for last_frame it means "end of clip." A negative value of last_frame returns the frames from first_frame to first_frame + (- last_frame-1). This is the only way to get the very first frame!

[SerbianBoss]

DVD. These came from VHS source, capped through a TBC-1000 and then A/D pass-though on my Panasonic DV camera to the PC as interlaced PAL DV avi. These were processed through avisynth encoded at CBR 7600 kbps as they are part of a short multi-angle test. The final clips are encoded at around CBR 8700 kbps + audio.

The screen grabs were taken with VLC from the interlaced PAL mpeg2 footage. The tape is ten years old, but some of the footage on it is 5 - 6 years older than that, so this is 2nd and 3rd generation dub in some cases.

Good explanation.Did you use some more filters for that cassete(like sharpening or color correction in avisynth)?

[FulciLives]

May I ask a related question?
@FulciLives -- Re Convolution3d --
every sample script I have seen posted on several forums includes the commands to SeparateFrames / Weave
I think this has caused most of us newbies to think this is required for all video, not just interlaced.
Convolution3d ReadMe doesn't discuss Interlaced / Progressive, but your post here implies that progressive doesn't need the Separate / Weave commands.
Z-C apparently found that out in practice.
Or did I mis-understand ?

Your understanding is correct. You would use the Separate Fields technique for interlaced video. For progressive video that is not needed.

However if you have a NTSC source and you perform an IVTC first in your AviSynth script then you can treat it as progressive video (after the IVTC stuff). However doing an IVTC can be tricky and more often than not it is better to not even bother with it. So even if you have a NTSC video with a 3-2 pulldown pattern ... if you encode it as 29.970fps without doing an IVTC then you treat it as interlaced video.

- John "FulciLives" Coleman

[SerbianBoss]

But if you use degrain median filter on interlaced Pal you can use witjout separate field/weave:

degrainmedian(5,7,0,interlaced=true)

Every filter who have "interlaced flag" can use without separate fields/weave

[guns1inger]

Good explanation.Did you use some more filters for that cassete(like sharpening or color correction in avisynth)?

I found that even a small amount of sharpening was detrimental in this case, mostly because there is already some ghosting/haloing from the exposure in the original shoot. Shartening just enhanced this effect, which I didn't want.

I used the virtualdub filter colormill (called using LoadVirtualdubFilter in avisynth) for contrast and colour correction. GrannyGeek mentioned in another post, and it is very good, although somewhat slow. And because it is a vdub filter, it does require a colourspace change to RGB32. I set it up in virtualdubmod, get the colours/brightness etc right, and save the process settings. The vcf file can then be opened in notepad to extract the settings for use in the avs script.

So I load my DV avi, convert the RGB32 and process with colormill, then convert to yuy2 for everything else.

This is the colormill post I mentioned

https://forum.videohelp.com/viewtopic.php?t=309062

[Z-C]

Indeed this thread is very helpful!

guns1inger: I tried using the ReadOut option. But for some reason the numbers keep going up! I even tried scrolling back and
replaying the movie and the numbers keep going up from the point they stopped. Is this a bug?

Very nice improvement by the way according to the pics. It's pretty close to the results I get so far.

lordsmurf: I have very old tapes that I don't want to degrade more. Besides, I really don't see that the filters do that much of a difference. It's a nice improvement - but if the difference is mostly theortical then I really shouldn't consume all my time on minor changes. Surely, if the result looks like crap on a certain film - I will do the best I can to fix it.


By the way, about encoding to MPEG 2 - should I use 3 pass VBR like in doom9 guides, or just CBR? I thought I'd go with CBR, but the guides confused me on this.

grannyGeek: Many filters state whether they handle interlaced footage well or not(interlaced-true/false option or something similar). Some don't, and I guess the solution is either asking here or just trying for yourself(I did both actually ).
About applying different settings for different parts of the video - this sounds like a great idea! could you give example of how you do that? I didn't understand whether you physically cut the footage to different files and work on them separately, or if there is a way to apply an avisynth script on each part of the movie(E.g: script1 - from 0:00:00 to 0:30:00, script 2 - from 0:30:00 to 1:00:00)

I found that even a small amount of sharpening was detrimental in this case, mostly because there is already some ghosting/haloing from the exposure in the original shoot. Shartening just enhanced this effect, which I didn't want.

Could someone explain what ghosting/haloing is? I understand it is something where 2 frames blend together, but is it noticeable at real-time movie playback?

[jagabo]

AVISource("file.avi")
SEG1=Trim(1, 1000).YourFilterHere()
SEG2=Trim(1001, 2000).YourOtherFilterHere()
SEG3=Trim(2001, 3000).AnotherFilterHere()
SEG1+SEG2+SEG3

[SerbianBoss]

guns1inger, what you think about tweakcolor filter for avisynth?

What colormil filter have that tweakcolor dosnt?

[Z-C]

jagabo: That's great! thanks .

Here's an example of my progress on the testing clip I'm currently working on.
This is just a picture, but I chose one that you can easily see a character's face to see the difference.
For this movie anyway, this is the best results I was able to achieve when taking into consideration both reducing noise and keeping quality(less blur).
I'd be happy to hear your comments, because this seems like a serious enhancement to me.
I'd like to mention that this is a closeup stilled frame. So the amount of blurring you do see is hardly(if at all) noticeable when the movie is streaming.





EDIT: It seems that the satisfaction I had from using both PeachSmoother+Vague was premature. For some reason, after encoding to mpeg2, the result is jumping and moves like in slow motion. The same happened with the use of MSU denoiser filter, but has never happened with Con3d.
The other dissappointment is this: The encoded file seems to have a lot more noise than when previewing the same movie with the same filters on Vdub. I also noticed that when I use peach smoother + vague I really have to play the movie once in vdub and then on the second playback I onlu see the substantial results. Maybe this is the cause of the bad quality encoded file? It wasn't fully filtered?

I can upload a short clip from the bad encoded clip if you want.

[jagabo]

It's just my opinion, but I like the unfiltered version better. The filtered one is too obviously filtered.

[lordsmurf]

I like the filtered one. Don't confuse noise with detail.

[jagabo]

Don't confuse noise with detail.

I wasn't.

[DVWannaB]

edited.....

[Z-C]

I uploaded a clip in here:
http://www.youtube.com/watch?v=EFgHE0qFuFM

but since it's a crappy quality anyway because of YouTube you can't see all the noise that was left.
The weirdest thing is that for some reason after uploading the video seems to run fine. So I guess that's not much of a help to you guys. hope you can help me even though the clip isn't too clear with describing the problem.

But when I play the video on my PC or try to burn it, the motion isn't clean. It's like it's moving on slow motion. Or maybe a better description would be that it looks similar to running low FPS(like when games look bad because they run at like 7 FPS).

[jagabo]

Are you talking about playing your videos within VirtualDub? VirtualDub isn't designed as a player and often plays with very jerky frame rates, especially if you are filtering.

If you are talking about playing MPEG files created after VirtualDub filtering (via a media player or DVD player) you shouldn't have any problems.

[Z-C]

I'm aware of that. I am referring to playing the encoded mpeg on other players - like WMP or PowerDVD. And I'm afraid I am encountering those problems upon playback. What could be the reason?

[jagabo]

Check the MPEG properties with GSpot to see what might be different between the VirtualDub and AVISynth processed files.

Also open the problematic MPEG file with VirtualDubMod or VirtualDubMPEG2 and step through frames one-by-one to see if any are missing or duplicated -- ie, determine if the problem is in the file or is caused by the player software.

[grannyGeek]

Z-C, You don't have to capture or encode the clip pieces separately.
You can just create your aviSynth script to *trim* the section of video, and it calls only those frames.
unfortunately, there's no way to call it by time, only by frame numbers.
Like so --
AviSource("C:\thisCap.avi")trim(15,1500)
Convolution3d(preset="movieLQ")


My next goal is to try to get comfortable enough in aviSynth to write a function that will include my most frequent filters, so I can just list out my trimmed clips, and call the function to each.
like this
clipA = aviSource("C:\thisCap.avi")trim(15,1500).multiFunc tion(params)
clipB = aviSource("C:\thisCap.avi")trim(1501,6500).multiFu nction(different params)
etc, etc
UnalignedSplice(clipA, clipB, etc, etc)

But first I have to make my head stop reverting to Visual Basic whenever I start to type
too many years of working with Excel, I suspect.

Now, if anyone has a sample of what I'm talking about, and wants to share, I wouldn't be too proud to pirate that . . .



SOME EDITS ADDED - FIRST PARAGRAPH

[Z-C]

good ideas jagabo. thanks .

Everything seems to be fine with Gspot.

It seems like Vdub is able to play to movie correctly. I will burn and check the movie on my standalone DVD player.

But one of the problems still remain - how come when previewing the movie filtered by the avisynth script, I get the nice and clear image like above. But after encoding it's like the filter hardly changed anything - I get the noisy picture which is almost, if not exactly, like the non-filtered movie.

If it helps - I'm not going through Vdub while encoding. I simply load the .avs file into CCE and encode. However, this has worked just fine with other filters like Convolution3d.

Any ideas?

[grannyGeek]

Could you post your script?

[grannyGeek]

Z-C -
kind of a ps
I had a capture recently that gave me fits.
I used Convolution3d, RemoveDirt, and aviSynth internal filter Tweak to adjust bright and contrast.
It looked really good in preview, but when I encoded to avi using Huffyuv lossless codec, it developed a lot of "banding" artifacts.
Apparently I had over-filtered, and stretched it beyond its capability.

Maybe this is what is happening to your clip.

I ended up re-capturing, and adjusted the Bright and Contrast settings in the capture filter.
Applied Conv3d and RemoveDirt filters, and encoded with Huffyuv with only very minor artifacts.

[jagabo]

Everything seems to be fine with Gspot.

It seems like Vdub is able to play to movie correctly. I will burn and check the movie on my standalone DVD player.

It may be that your MPEG 2 decoder is doing some processing (deinterlacing? brightness?) while playing the video (VirtualDubMPEG2 and Mod use their own MPEG 2 decoders). That might explain why you don't get smooth playback -- but you'd expect it with both the AVISynth and VirtualDub processed files.

But one of the problems still remain - how come when previewing the movie filtered by the avisynth script, I get the nice and clear image like above. But after encoding it's like the filter hardly changed anything - I get the noisy picture which is almost, if not exactly, like the non-filtered movie.

Maybe you're not using enough bitrate in the MPEG 2 file? That would cause macroblocks which some people describe as noise. I don't really think that's your problem though.

[grannyGeek]

Whew- finally caught up and read all the posts I missed while I slept,
sorry that I repeated info that jagabo had already provided about "Trim"

@SerbianBoss---
ColorMill is different than Tweak in that it lets you adjust levels, gamma, and RGB for dark, mid, and light ranges. So if you only want to lower the red tones in the "highlights", you can go into the "light" range to adjust the red.
Then maybe you want to lower saturation in the Mid range, and adjust the Hue in the Dark range.
etc.
RGB Equalizer is similar, but adds more bands for adjustment (like an audio equalizer)

Hard to explain in words, you just have to download and try them.
You can tweak color correction very precisely, but of course, they only work in RGB colorspace.


@Barnabas
I wonder if the cleaning filter you recall might be NeatVideo? $99.90 USD.
they also have a free demo, if I remember right the demo watermarks the video.
http://www.neatvideo.com/

[Barnabas]

grannyGeek,

NeatVideo, that's the one!. Been trying to think of that for a couple of days now, Thanks!

[Z-C]

The problem is Not with newly generated artifacts or macroblocks, so it's not a conversion/bitrate problem.

About the other suggestion you made about the decoder - If that was the case then I'd expect it to happen(like you said) with all the files I encoded. It seems to only happen with files encoded using PeachSmoother/Vague filters. It seems like it skips the filters or it almost doesn't use them.
When using convolution3d everything is fine, but I'd prefer to avoid using that because although it's a great noise reduction, it's also adds a 'nice' blur.

EDIT: This is just getting weirder and weirder. Instead of directly sending the .avs file to CCE, I decided to first open the .avs file in Vdubmod for preview. Like I said before, the movie reduces noise after previewing it once. So I played it fully for one time. Then I started frameserving with Vdub and encoded with CCE.

The result is finally like it should be - same as the preview! I even compared the two MPEG2 files - the one going through preview in Vdub and the one isn't, and the change is obvious!

How could you guys explain that? I'm lost..

[FulciLives]

Post your AviSynth AVS script please.

Some of the things you are saying makes no sense unless maybe you have "errors" in your script.

- John "FulciLives" Coleman

[Z-C]

I got NeatVideo demo and tried playing around with it.
Besides the fact that it's a great noise reducer, that slow motion/looking like low fps movie issue occured again after encoding.
I also tried burning these clips to a DVD and tried using them on my standalone DVD. They show the same problems there too, so I guess it's not a decoder issue.

So far it seems that only Convolution3D encoded movies work smoothly. Using every one of the following filters: Neatvideo, MSU denoiser, PeachSmoother, Vague denoiser causes these problems.

If this was an AviSynth problem, then the fully Vdub filters(MSU, NeatVideo) should have worked right? If this was a Vdub problem, then skipping vdub and using the avisynth scripts directly on CCE should've worked.

Now that I come to think about it, maybe these filters cause problems because the footage is interlaced? Those filters do have interlaced flag so I dunno how this is possible. But, Con3D was the only filter I had to separate the feilds before filtering, and it's the only filter that works well.

Here is the Con3D script that works fine:
LoadPlugin("Convolution3D.dll")
AviSource("C:\vhstodvd\movies\scene0001.avi")
SeparateFields()
ConvertToYUY2(interlaced=true)
odd=SelectOdd.Convolution3D(1, 32, 128, 16, 64, 10, 0)
evn=SelectEven.Convolution3D(1, 32, 128, 16, 64, 10, 0)
Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd()

Here is the Peach/vague script that produces problems:
LoadPlugin("PeachSmoother.dll")
LoadPlugin("VagueDenoiser.dll")
AviSource("C:\vhstodvd\movies\scene0001.avi")
ConvertToYUY2(interlaced=true)
PeachSmoother(NoiseReduction=80)
VagueDenoiser(threshold=7, nsteps=8, interlaced=true)

But, Please take into account, that like I said before, NeatVideo and MSU denoiser produces the same issues, even though they use no AviSynth script - because they are Vdub filters.

[jagabo]

I've been playing around with all the same filters you have over the last few days, both VirtualDub and AVISynth, and my output files are all as smooth as the input files. But I've been using progressive sources to make it easier to see the differences.

Make sure AVISynth has correctly identified the field order. I believe your source is bottom field first, add AssumeBFF() before filtering. Or, if you source is top field first, add AssumeTFF().

[Z-C]

I'm not sure whether my source is BFF or TFF. how can I check?
I got a small script to test that from doom9. I used the script but I'm still not sure what I'm supposed to see to determine whether I have BFF or TFF.

However, since this has worked with the con3d filters, I assumed I have TFF? and used the compatible options for TFF in CCE. or could separating the fields and then combining them when using Con3d, change my source to TFF? Sorry if this doesn't make sense... I might be confused from all of these new terms I'm learning.

[FulciLives]

DV AVI is always BOTTOM FIELD FIRST so you would use AssumeBFF() in the script.

For instance ... try this:

LoadPlugin("PeachSmoother.dll") 
LoadPlugin("VagueDenoiser.dll") 
AviSource("C:\vhstodvd\movies\scene0001.avi") 
AssumeBFF()
ConvertToYUY2(interlaced=true) 
PeachSmoother(NoiseReduction=80) 
VagueDenoiser(threshold=7, nsteps=8, interlaced=true)

Open this up directly with CCE and be sure to properly set up CCE for an interlaced bottom field first video.

- John "FulciLives" Coleman

P.S.
I've never used PeachSmoother nor VagueDenoiser but I notice in your sample script that you have an interlace option for VagueDenoiser but I don't see an option for that with your PeachSmoother function.