Here is a several seconds of my video. It's from an interlaced VHS-to-DVD conversion which I demuxed to MPEG2. I don't have the original VHS tape. This is all I have. I have not done any work to the sample I am giving you.
Below is the script I used to remove most of the problems. It deinterlaced and denoised it and everything looks good apart from these dancing pixels which are clearly seen in the first few seconds of the video in the wrestlemania logo. In the 2nd part of the video with the man you can also see some flicker which I would also like to remove.
Can anyone suggest a script which will fix the dancing pixels and remove the flicker?Code:Mpeg2Source("L:\2 = New\Raw 1996 March 11 Still.d2v", CPU=6) Load_Stdcall_plugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll") QTGMC(Preset="Ultra Fast") Vinverse() TTempSmoothF(maxr=3, lthresh=8, cthresh=5, strength=4, interlaced=false)
+ Reply to Thread
Results 1 to 30 of 62
Last edited by VideoFanatic; 26th Jul 2012 at 17:10.
You can try a software TBC but I don't think it will work well with a DVD rip.
Otherwise, for the still shot you can use a very strong temporal smoother (like Temporal Smoother in VirtualDub, turned up all the way). Or even better, use a still frame. Obviously, that won't work with motion shots.
I gave the sample a quick try, but couldn't do that much with video already processed. The dot crawl, aliasing, flicker, and other junk would have responded better -- with less loss of detail -- had QTGMC not been underused. TTempSmooth alone won't do much for the problems you mentioned. Much of the flicker is gone, and some of the ghosting. Yadif is clean and fast but IMHO it softens video that's already too soft. The dancing pixels aren't just in the early part; they're all over the video. I tried to restore some of the dark details but they were already crushed by previous processing. I also centered the image a little and covered the 12 pixels of head switching noise at the bottom. Most of the time you can't take the easy way out with low quality video. It takes patience, or you just have to live with the results. Would have preferred to work with the original unprocessed sample, as it would allow using stronger filters and retaining more detail.
Ed: Just not satisfied with the first effort, so I'm posting a newer version "DancingPixelsA", attached. All I added to the previous procedure was RemoveSpots() in the last step. I find it does more than just remove spots, also cleans up mild flicker. Unfortunately the clip is starting to look overprocessed, so if you want to try RemoveSpots for effect, go back to the unprocessed original.
Last edited by sanlyn; 27th Jul 2012 at 05:44. Reason: remove earlier mpgOur inventions are wont to be pretty toys, which distract our attention from serious things. They are but improved means to an unimproved end. -- Henry David Thoreau
By dancing pixels, I think you mean horizontal line jitter. Since you have good black borders, software tbc might work, but I should make some modificiations to work only on certain lines, and to have sub-pixel precision, and a mode for static images. Hmm...
And you don't need Vinverse in the script as there's no interlacing after QTGMC. If you're using it to control something else, then use a filter designed for that purpose.
vidpath="E:\forum\dancingPixels\" ppath="D:\AviSynth 2.5\plugins\" Import(ppath+"QTGMC-3.32.avs") Import(ppath+"RemoveSpots.avs") MPEG2Source(vidpath+"Raw 1996 March 11 Still.d2v",CPU=4) Crop(2,0,0,-12) AddBorders(0,6,2,6) ColorYUV(off_y=10,gamma_y=-15) ColorYUV(cont_v=-12) MCTemporalDenoise(settings="low",sigma=5,AA=true,useEEDI2=false,enhance=true,DeBlock=true,interlaced=false) ConvertToYUY2(interlaced=false) TemporalSoften(4,4,8,15,2) ConvertToYV12(interlaced=false) QTGMC(Preset="medium",InputType=1) # -------------------------------------- # --- RemoveSpots() for version 'A" only # -------------------------------------- RemoveSpots() LSFMod(strength=50) AddGrainC(var=1.0,uvar=1.0)
ED: holygamer, the basic idea here was to use two types of filters. One for line and edge noise (some of MCTD's modified parameters). The other to address flicker and shimmer (smoothers and a stronger QTGMC). RemoveSpots() helped, too. You might add that to your arsenal. It's a pretty fast script, as well.
Last edited by sanlyn; 27th Jul 2012 at 08:47.Our inventions are wont to be pretty toys, which distract our attention from serious things. They are but improved means to an unimproved end. -- Henry David Thoreau
For the record the sample I gave you was interlaced and I didn't do any work to it. I was just saying that when I then did work on it using my script, I still have dancing pixels which I didn't know how to remove. The sample I gave is interlaced and it has really obvious interlacing artefacts when watched on a TV. QTGMC removes those interlacing artefacts completely. I only have a Dual Core with Vista 32-bit at the moment. On Ultra Fast QTGMC takes about 11 hours with Avisynth MT. I noticed that on other videos with worse interlacing artefacts that Ultra Fast doesn't completely remove all of those interlacing artefacts but if you add Vinverse then it does. So surely Vinverse doesn't do any harm? I have thousands of VHS-to-DVD videos similar to this so I'm just looking for a script that would work on all of them. If some of my videos didn't have this dancing pixels problem then using your script and Vinverse wouldn't cause any harm would it?
I will be buying an 8 core (8 x 3Ghz) PC with Windows 7 64-bit very soon. If a dual core takes 11 hours to encode a video then what's the slowest (better quality) QTGMC speed I could use which would take 4 hours or less for a 1 hour 30 minute video? Also MCTemporalDenoise takes over 24 hours on a Dual core. How long would it take with my new PC?
I noticed you mentioning interlaced=false in the script. Isn't the video interlaced before your ran QTGMC at the end so why did you say it was progressive when the video is still interlaced?
What doees InputType=1 do with QTGMC? I can't find any documentation telling me what it does.
That whole script you posted, did you use that on my original sample video?
I'm not bothered about the plastic look. I like the way it looks and would prefer as much of the noise removed as possible.
I tried using the TBC script that was mentioned in the TBC Doom thread but it took over a hour just to process several seconds and the video looked far worse than normal! Is that normal? Are you able to get better results yourself with TBC compared to the DancingPixelsA video that you made?
I watched the DancingPixelsA video you gave me and it looks great. Thanks. There are still some moving pixels though, however if that's the best that can be done then I'm very happy with that.
What plugin do I need for RemoveSpots to work?
Last edited by VideoFanatic; 28th Jul 2012 at 14:05.
Oops, my fault. Must be something wrong my eyes (I shouldn't do this work at 2:30 AM!). Just loaded it again and you're right. Interlaced. (Wonder what the hell I was thinking???). I'll have another go at it. Will probably use some of the same plugins, but the sequence will have to be changed. Back with you later on that, but to answer some questions:
InputType-1 is ostensibly for progressive video (or you can change some of the settings for interlaced), it has various parameters but it's there for types of damage that the other modes don't handle. I know what you mean about QTGMC and MCTD being slow, depending on how it's set up, but problem videos often require drastic treatment. I have plenty of those myself -- I'm working on one now that I started in 2007, this is my 4th attempt. In the strictly progressive video mode, QTGMC at some stage or other after some prepping treats it as interlaced and outputs the same fps. Notes on the QTGMC settings are in the html that comes with the QTGMC package, in the "instructions" folder.
Some plugins run pretty slow on my XP's as well, but not as slow as they seem to run on yours. I'll admit, I've done what many do: for a really slow process, I just cut the video into chunks and let a chunk run overnite. I can't count the number of times i've done that. I just spent 3 weeks building a new machine, and stubbornly I made it 32-bit XP with quad-core AMD. Yeah, runs some scripts 500% faster (or more). One trick I use with the slower machines is something I picked up from Doom9. The slower plugins monopolize CPU time; they run much faster if run in a script by themselves, then run the other plugins after. I do this especially if I plan on fiddling with parameters of the other plugins: that way, I have my slow-burner already "baked" so to speak, and I can run the quickies by themselves without having to hassle with the slow runners every time I change something.
Back later. SOrry for the delay. The wife is home today, so there are constant interruptions. I always say my retirement is suspended between 6-PM Friday and 8-AM Monday.Our inventions are wont to be pretty toys, which distract our attention from serious things. They are but improved means to an unimproved end. -- Henry David Thoreau
Strangely it doesn't look like you using progressive when you actually wanted interlaced did any harm as the video looks good!
I just checked to see what 30 FPS progressive looks like and it doesn't look good which is a shame. And Bluray doesn't support 60 FPS progressive so I'll have to keep the video interlaced. I look forward to seeing what you can do.
By the way where do I get the RemoveSpots plugin?
Last edited by VideoFanatic; 28th Jul 2012 at 18:31.
There must be dozens of versions of RemoveDirt() itself, which many users change to suit their purpose. This ability to modify the basic script+plugin resulted in RemoveSpots(), among many other toos.
I'll tackle this video in the morning again, getting late here. But I took a deeper look, and learned two things. (1) I should have learned to never make assumptions about sample clips. I saw no combing effects, so I assumed it was not interlaced. I should know better by now. (2) More importantly, this video at some time in the past was definitely de-interlaced, and incorrectly so. Then, it was either reinterlaced (again, incorrectly) and/or transcoded incorrectly. It's a totally fouled-up VHS->MPEG transfer. Most of the damage can't be repaired. The best one can do is smooth the hell out of it. I encountered a similar video earlier this year, but in far worse shape. Oh, well...up and at it tomorrow.
Could you please give me precise instruction as to how I get RemoveSpots to work and what plugins I need. I downloaded RemoveDirt 0.9 and put the SSE2 file in the plugins folder but the script still isn't working. I'm not sure what I'm supposed to do with the script in the post you showed me? It doesn't look like an avsi file so it doesn't look like it goes in the plugins folder. If it's a script, you didn't use that script, you just used "RemoveSpots()" and it worked for you! I'm confused!
How it's used is in the post he linked to:
Paste the whole RemoveSpots() function into your script like the script in that post. Or put it in its own AVS script an import it. Or name the script RemoveSpots.AVSI and put it in AviSynth's plugins folder so it's imported automatically.
Sorry, holygamer, jagabo beat me to the answer. Busy day today, but I'm looking over the video while I work on another PC.
On my PC I have the RemoveSpots script saved as "RemoveSpots.avs" in my Avisynth plugins folder. To bring it into a running script, I use this statement (change the hard drive and folder path to your Avisynth plugins folder to match your own setup):
Someone might have to add more detail to what I've found, but the clip sample shows a video that has alrady been deinterlaced, incorrectly. Here is an image from frame 30 (Even field only), and you can see the bad deinterlace damage (note the jaggies). It looks as if someone made a bad attempt at making this video progressive. Treating the video as interlaced or using SeparateFields for filtering just makes it worse:
You can bob-deinterlace the mpeg sample in VirtualDub and observe the problem. Open the mpg in VirtualDub. Under Video . . . Filters . . ., choose the "bob doubler". Set the bob dialog's "Field Order" to Top Field First and the "Deinterlacing method" to Bob. Play the video a frame at a time to see the damage. Rather than a case of dancing pixels, you have a case of dancing lines.
Last edited by sanlyn; 29th Jul 2012 at 18:19.
I've used the same simple bob with properly interlaced and progressive source. Yes, it always looks a little sci-fi, but I don't see jaggies, smearing and geometric tricks like these. If you filter this clip as if it's interlaced, it looks worse. But running QTGMC in repair mode makes significant improvement, to the point where other denoisers actually accomplish something afterward. The last time I saw serrations and wiggles like these after deinterlace, it was because I did something silly. Now, I can't recall what my error was. But it produced similar effects. Unfortunately I don't have the original VHS source to re-work. Only have the botched results. Anyway, I'll keep at it.
So do you guys see interlacing artefacts on the video in this thread when played on a TV? I do.
Thin horizontal lines:
Smarter deinterlacers will look at other fields, and other frames, for data to fill in those missing lines. And perform smarter interpolation methods (like nnedi3).
Last edited by jagabo; 29th Jul 2012 at 19:20.
OK guys. I've confirmed my other DVD clip doesn't have interlacing artects on my TV just like you said. I found a DVD player and it played without interlacing artefacts. I also contacted my friend who said a Bluray disc I gave him has interlacing artefacts (same sample we're talking about). He's just told me that it was his Bluray player that's the problem as he tried it on another player and it worked fine! Seems like a strange way for a Bluray player to break down!
So the clip in this thread, do you guys see any interlacing artefacts in it when watched on your TV? Here's a DVD ISO.
So do you guys see interlacing artefacts on that video when played on a TV? I do. I also put it on a Bluray and took to a professional TV shop. They played it on several Bluray players and there were interlacing artefacts visible.
If you do see interlacing artefacts, do you think it was because of a bad conversion?
A sloppy conversion, alright. But I get what jagabo says. I just looked over some interlaced stuff with simple bob() and it looks nicely bob-deinterlaced to me, smooth verticals, more grainy and softer than the original, but -- it's a much cleaner capture to begin with. I was hoping to avoid heavy time-consuming plugins at holygamer's request, but let's face it: sick video needs strong medicine. Will have some results later tonight.
Last edited by sanlyn; 29th Jul 2012 at 19:40.
So back onto the original subject of this thread! The original sample I posted in this thread I would like any suggestions on how to improve any part of the picture such as the dancing pixels. Most of the dancing pixels are gone with RemoveSpots but some still remain. I'm really just looking for a set of scripts I can use on any video to perform general cleanup. I've got thousands of similar videos like this. I'm getting an 8 core (8 x 3Ghz) PC with Windows 7 64-bit soon and I will be using the Multi-Threaded version of Avisynth. So with that configuration in mind if you can suggest scripts that would take up to 4-5 hours for a 1 hour 30 minute video with that PC that would help a lot.
Will MCTemporalDenoise take less than 5 hours for example?
Making RemoveSpots work
I copied this script into a file called "RemoveSpots.avsi" and put that in my plugins folder. I was using RemoveGrain-1.0 PreRelease and I copied all the SSE2 files from that into my plugins folder. I then get an error saying there is no function named Clense. After a Google search I find the RemoveGrain page and it says if you get an error like "there in no function named clense", then you need this: RemoveGrainT 1.0. So I tried that and I still got the error "there is no function named clense" in RemoveSpots.avsi. So I copied the SSE2 files and the DenoiseSharpen files from RemoveGrain 1.0b Beta into the plugins folder.
My encoder now encodes the file without giving an error and it removes the dancing pixels. However when previewing the video in AvsPmod, AvsPmod crashes. Was I correct to use the "RemoveGrain 1.0b Beta"? Any way I can get AvsPmod to stop crashing?
I don't know about you, but I find AvsPmod to be a big hassle. Good idea, but IMHO it needs work. It's OK for re-running one or two functions like ColorYUV to change levels and chroma over and over. But whenever I try complicated functions or scripts it gets in the way. But that's up to you. Many defend its use in any and all situations. It just slows me down.
Like many members here, I do wish there was one script I could use for everything...even for videos that at first glance look like 20 others I have. Some things like IVTC and TTempSmooth or LSFMOD are useful just about anywhere, but not everywhere. Every video is a learning experience.
With problem videos like these, it's a wonder you haven't looked into NeatVideo. It's used by pros pushing video thru Premiere and some even higher-end stuff. But I don't like to recommend scripts using plugins that users don't have.
Meanwhile I'm still working at this between bouts of repairing two PC's I have in front of me tonite. Sorry, but repairs are bread and butter around here (they paid for 3 of my PC's). I'll keep trying more tricks with this kinky clip.
DSepending on how it's set up, NeatVido runs somewhat faster than MCTemporalDenoise. That 1fps user must have NeatVideo turned up full power. Great way to ruin a video . But that's personal preference. One man's poison, and all that . . .
If you treat this clip as progressive, it looks better. If you try SeparateFields or bob or any deinterlace, then put it back together again, it's as if no filter had ever been run; comes out the door looking exactly the same way it went in. Amazing.
Bluray doesn't support 60 fps progressive and 30 fps looks like crap so I need to keep it interlaced. Were you being sarcastic when you said Amazing? Are you saying deinterlacing then re-interlacing is pointless or a good idea?
Last edited by VideoFanatic; 29th Jul 2012 at 23:49.
No fast fix for these clips, holygamer. They're a mess. The version "A" script is the cleanest so far, and I've done everything except wash this clip down with alcohol (Hmm, I wonder . . .? But, no.). Every time it's treated as "interlaced", it comes out looking just as bad. Every time I use the same or other filters and treat it as somehow-but-incorrectly progressive, it comes out cleaner. Mystery to me. But not the first one I've seen. You could make MCTD "Medium" instead of "Low", and use stronger QTGMC. Or throw more smoothers at it.
If you think this clip has problems, try this original post : AllInYourName.mkv . (35MB, 5-min 34-sec of pure noise, horrible camera work, and nightmare conversion).
3 and a half months later, the best we could get: post of 2.75 minute preview plus entire video http://forum.videohelp.com/threads/341917-remove-noise-and-block?p=2153660&viewfull=1#post2153660 . Broke a lot of rules, even then. A long way from what we wanted, and still looks like crap. Big download, yeah. Two and a half hours of processing for 5-1/2 minutes of video, then add another 5 hours for cutting and color correction of 19 different camera shots. For bad transfers, that's about par.
Meanwhile: for 16:9 HD 1-hour recordings I make off cable TV, it takes only 30 minutes to cut out the commercials, make a menu, and burn it to disc. Period. Welcome to VHS.
Last edited by sanlyn; 30th Jul 2012 at 15:39.