I posted this sample a while back and PDR, I believe cleaned it up nicely. Sanlyn offered some great suggestions as well. Can you guys take another look at it please? This is an official retail DVD rip which I am trying to restore.
I have gotten decent results with McTemporalDenoise, TTempSmooth along with chroma sharpening, Toon, a little warpsharp and dehalo_alpha. Desaturated as well (sat=80). But something seems..."off". Any suggestions are appreciated. This is the original UNFILTERED sample. The script I'm using is as follows:
Code:MPEG2Source("blah.d2v") AssumeTFF() TFM(d2v="blah.d2v") TDecimate(mode=1) McTemporalDenoise(settings="high", sigma=4, protect=true, edgeclean=true, Ecmode="dehalo_alpha_mt()", enhance=true, AGstr=1.0) Tweak(sat=0.80) aWarpSharp(depth=8) #thins lines. this setting was able to chisel out junk and show more small details## Mergechroma(awarpsharp2(depth=20)) #brings chroma inside lines# Toon(strength=1.0) Santiag(2,2)
+ Reply to Thread
Results 1 to 21 of 21
Thread: need help cleaning this up
The video is in sad shape. Large clumps of shifting and fluttering grain, banding, low-bitrate artifacts, luma and chroma out of range for DVD (and almost out of range for PC video), strong red color cast, crippled blue, spots, dropouts, projector hop...and some really bad flicker. The sample script posted looks a lot better than the original, but flicker and even some "flashing" and jitter are still there. I think I got most of it, but didn't address the remaining spots or hops (I was getting tired of finding new problems). I used a few goodies you probably haven't seen:
The filtered video used your same TIVTC, so it's not DVD-compliant (23.976 fps). If you prefer to blow out the darks, just increase Cont_y with ColorYUV.
# Scripts and Plugins:
# - QTGMC-3.32.avsi
# - daa3.avs
# - TIVTC.dll
# - SmoothAdjust.dll
# - SangNon.dll
# - FluxSmooth.dll
# - TTempSmooth.dll
# - DeHalo_alpha.avsi
# - aWarpSharp.dll
# - Toon-v1.1.dll
# - LSFMod.avsi
# - DeFlick.vdf
# (from the Dither plugin package):
# - Dither.avs
# - mt_xxpand_multi.avs
# - avstp.dll
# - dither.dll
# - mt_masktools-26.dll
MPEG2Source(vidpath+"VTS_01_1.d2v") TFM(order=1).TDecimate(mode=1) Trim(5,0) Tweak(sat=0.85) Crop(8,0,-4,-8) SmoothLevels(0, 0.9, 235, 12, 235, chroma=200, limiter=2, tvrange=true) ColorYUV(cont_y=15,off_y=1) ColorYUV(cont_v=-15) ColorYUV(Cont_u=30) AddBorders(8,0,4,8) daa3() SangNom(0,24) QTGMC(InputType=1,preset="faster",SourceMatch=3, TR2=2) FluxSmoothT() TurnRight() FluxSmoothT() TurnLeft() TTempSmooth() ConvertToRGB32(matrix="Rec601",interlaced=false) Crop(8,0,-4,-8).AddBorders(6,4,6,4) vdpath="D:\VirtualDub\plugins\" LoadVirtualDubPlugin(vdpath+"Deflick.vdf", "DeFlick",1) DeFlick(0, 10, 0, 0, 0) ## # <- '10' might be a little high. Try 8. Dither_convert_rgb_to_yuv(interlaced=false,tv_range=true) dehalo_alpha() mergechroma(aWarpSharp(depth=20.0, thresh=0.75, blurlevel=2, cm=1)) Toon(strength=1.0) LSFMod() # ### AddBorders(6,4,6,4) ### <- disabled return last
ED: A "fixed" version of the video using this script is posted in #10, below.
Last edited by sanlyn; 17th Jan 2013 at 10:09. Reason: corrected text of codeOur inventions are wont to be pretty toys, which distract our attention from serious things. They are but improved means to an unimproved end. -- Henry David Thoreau
That's not bad. Although it resulted in some jaggies in some areas (take a look at the girl's glasses). Also the grain appears stabilized but has a "fuzzy" texture, due to that. Usually results in using TTempSmooth alone. I've been preprocessing these clips with MvDegrain3 (along with mergechroma and the crew) and then sending it to NeatVideo for some light smoothing as a finishing touch. Along with Neat's sharpening at 40%
The wall behind the secretary is supposed to be slightly grainy. but not half-moon banded. It's also not supposed to move and change shape every 3 frames. Thought you were asking how to clean that up. Or were you talking about flicker? Crushed darks? Red color cast?
Last edited by sanlyn; 17th Jan 2013 at 03:09.Our inventions are wont to be pretty toys, which distract our attention from serious things. They are but improved means to an unimproved end. -- Henry David Thoreau
I came up with the MvDegrain/Neat combo solution AFTER I posted this. I forgot to mention that so I do apologize for that little missed piece of information. It was late. I'm on my way to work now so I'll post samples tonight. But the idea is to first eliminate the grain and stabilize the footage with MvDegrain3 (or 2). You then run it through Neat (at low settings) to smooth it out. It works wonderfully and you'll see exactly what I mean. No detail loss whatsoever, the banding is essentially ancient history and the image is not blurry, as would happen if you just used Neat by itself.
Mergechroma sharpens the chroma, what do you mean by "how does it come into the picture"?
I changed the script I posted in post #2. The corrections are in BLUE. Works OK now (Shucks, I'll have to replace posted v5 with a new one tonite). Thanks for spotting that one.
MergeChroma(fft3dfilter(sigma=.1, sigma2=.3, sigma3=.5, sigma4=45, plane=3))
That will help curb color fluctuations over several frames. Can de-rainbow a bit, too. But it won't sharpen anything. The following would soften some chroma noise without affecting luma:
Last edited by sanlyn; 17th Jan 2013 at 08:20.Our inventions are wont to be pretty toys, which distract our attention from serious things. They are but improved means to an unimproved end. -- Henry David Thoreau
MergeChroma(awarpsharp) puts the bleeding color back in the lines. That's the bottom line. And it works.
Let's stick to the subject and not go off into other tangeants, please.
But, scoob, we are sticking to the subject. Its not "mergechroma" that sharpens, it's "(awarpsharp)" that does the sharpening by tightening up on the color bleed. You can use mergechroma to do all kinds of things; this is just one of them. The sharpener is awarpsharp, not mergechroma. We're using mergechroma to limit the operation to chroma only, or you'd end up over sharpening the whole image including the overall noise level, not just tweaking chroma smear along edges.
Meanwhile I got up a fixed version of the video without the frame size throwing things off. I also stopped the frame hopping by replacing this part of the original script:
FluxSmoothT() TurnRight() FluxSmoothT() TurnLeft()
Stab(dxmax=8,dymax=8) FluxSmoothT() TurnRight() FluxSmoothT() TurnLeft()
Last edited by sanlyn; 17th Jan 2013 at 11:40.
What do you think?
Lookin' better all the time, unclescoob. I think you're down to the ittsy-bittsy tweaking stage. All looks pretty good. I note some persistent flutter in that pesky background behind the redhead, so I found a tweak that keeps it quite still without over smoothing background textures. Also, note how objects tend to jitter and look "nervous". You can see it in the two shots below, and elsewhere. In the image on the left: load your mp4 into VirtualDub, play it a frame at a time and watch the guy's face change shape when he's supposed to be still.
I used RemoveSpots() and QTGMC to calm this shimmering, and Stab() for the frame hopping. The attached mkv used your original m2v sample and matches frames 412-866 of your posted mp4.
Ran the script in two steps again so I could tweak parts as needed.
Sanlyn, I can't play your video. In any event, why do you always use these complex, overwhelming scripts? And then they're TWO-STEP complex, overwhelming scripts at that. Total overkill and not necessary. Wanna know what I did to get the video back in shape (sans stabilization on the shakies)? Look:
MPEG2Source("blah.d2v") AssumeTFF() TFM() TDecimate(mode=1) #and now, ladies and gentlemen, I present MVDegrain3 from MVTools: super = MSuper(planar=true) bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4) fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4) bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4) fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4) bv3 = MAnalyse(super, isb = true, delta = 3, overlap=4) fv3 = MAnalyse(super, isb = false, delta = 3, overlap=4) MDegrain3(super,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=400,planar=true) #Add a little TTempSmooth (fluxsmooth also works) for the remaining stubborn flicker like so: TTempSmooth(maxr=7, lthresh=25, strength=5) #let's fix the lines, the bleeding color, darken the new line and anti-alias, like so: aWarpSharp(depth=8) #thins lines. this setting was able to chisel out junk and show more small details## Mergechroma(awarpsharp2(depth=20)) #brings chroma inside lines# Toon(strength=1.0) Santiag(2,2) #let's convert to RGB32 to finalize with Virtualdub for NeatVideo and Color Correction ConvertToRGB32()
I was bustin' your chops a little bit on that last post. But these scripts should be simpler if possible. By the way, I checked the bitrate and was wrong, Sanlyn. The AVERAGE is 5,500kbps and it peaks at over 8,000kbps. Does that change the statement about this video being low bitrate?
Also, what does turn left and turn right do in terms of the effectiveness of cleaning? i'm trying it and it seems to work well. What's the secret behind it?
The mkv posted is plain AVCHD h264. You might be missing an mkv codec somewhere. But an mp4 version is attached.
What's going on with the frame rate in your mp4?
Frame rate mode: Variable
Frame rate: 23.864 fps
Minimum frame rate: 19.173 fps
Maximum frame rate: 47.974 fps
Your script is pretty much the same as posted earlier. Looks good , but still has flutter, shimmer, and frame hops. That's helped visibly by oversmoothing and running NeatVideo at fairly average y-channel settings, but much of the textured artwork in the background is oversmoothed. But some people like that effect. Basically you have the same thing you started with, but pumped up more smoothing. If that's what you want, don't change a thing.
BTW, I have no problem playing the mp4 or mkv, but if you still have a glitch let me know. I have other h264 encoders and can remount later after lunch. I had a problem here playing some h264's made with Handbrake and a couple of other apps, but found a fix for it. No problems since then.
No that's fine, I was able to play your sample. My first stage (after MvDegrain3 and the chroma sharpening and line darkening) looks pretty much like yours. I then ran it through virtualdub for color correction and some light NeatVideo settings.
I also got rid of some flutters with lower TTEmpSmooth settings and some Fluxsmooth combined.
I used stab, by the way and it works well. Except that now, instead of the image shaking, the whole frame itself shakes (note the borders). How can I get rid of that? Regarding my frame rate, take a look at this sample and let me know. The first sample was encoded with Handbrake. I used HcEnc for this one (it's supposed to be 23.97fps, as I IVTC and left it at that. I want it progressive).
Let me know what you think of this sample.
OK, glad it worked. Maybe the "mkv" container called for something that might not be on your PC (like Haali media splitter, maybe? Hell, who knows. One h264 encoder and umpteen containers and codecs!! Ya gotta love it).
restrored.m2v looks nice. Some of that grungy stuff in the flat backgrounds, it's a headache sometimes when it's in the original source. And stab does literally move the whole image, meaning that the borders get moved as well. I cropped off 2 pixels from the left. Still a little grayish noise on the lefthand side, but likely that will go unnoticed. (On second look, my mp4 borders need a touch-up, too). What you have to do is let Stab do its thing, then view the results, fix borders as needed in the script, and you're home free. Kinda clunky, but if you try to anticipate what stab will do you can end up pre-cropping and get a frame size that upsets those plugins that want to work with their 8x block sizes and whatnot. Sometimes you might lose a pixel or two somewhere, but it's usually noise anyway.
Looks good. Goes to show you, transfer and cleanup is 5% inspiration and 95% perspiration.
Last edited by sanlyn; 19th Jan 2013 at 13:05.
Good, glad I'm gettin' somewhere for a change. Can you please confirm the framerate Sanlyn? What program do you use for it? I just use info() on avisynth. Virtualdub pretty much gives me the same info. I used Handbrake on the previous one (the one that had the funny frame rate). But my goal is to keep it at 23.97fps. How do I do that on handbrake?
The basic framerate in Avisynth is close enough, but you get more detail with MediaInfo (free app, see http://www.videohelp.com/tools/MediaInfo ). The installer shows ads for several other products (uncheck those, they just get in the way). MediaInfo tells me 23.976 fps, no variable frame rate indicated -- which means it ain't variable.
I tried a couple of versions of Handbrake over a period of time, but lost patience with it. Now using (and learning) StaxRip, TX264, and TMPGenc Mastering Works (I'm sticking with ye olde TMPGenc Plus 2.5 for SD work in plain ole DVD). The TMGenc HD product is easier to use and makes good clean h264, but I think the other two might be a bit easier and a tad cleaner (or just my imagination, maybe?). Any way you look at it, h264 isn't so easy to get a handle on. But I did learn a lot in principle from struggling thru Handbrake's on-web user guide. Tough going, that stuff.
Sorry, I seem to have missed this entire post. Not a new occurrence on this end.
NeatVideo or TemporalDeGrain, you don't just throw it at anything and everything. Seems to work with aliasing and with heavy grain that dances quickly in many directions. Some people on Doom9 seem to have first tried it to clean up dot crawl. It killed some noise, but most of the dot crawl persisted. Can't win 'em all.