mathmax, while I'm obessivelly rnning different version of the Jackson video, I found a recent thread in which I posted smples of using VirtualDub gradation curves to adjust color, gamma, etc.. The example is anime, but the principles would work anywhere.
Before & after frames are posted here: https://forum.videohelp.com/threads/331681-s-video-artifacts?p=2141709&viewfull=1#post2141709 .
A little farther down on that same page, there are captures of settings in various filters are shown here: https://forum.videohelp.com/threads/331681-s-video-artifacts?p=2141853&viewfull=1#post2141853 .
Still working with Jackson, Gibbs, & Co.
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 61 to 90 of 125
Thread
-
Last edited by sanlyn; 21st Mar 2014 at 16:07.
-
I'm still at it.
Making progress.Last edited by sanlyn; 21st Mar 2014 at 16:06.
-
Thank you so much for all your efforts on this video sanlyn. I look forward to see your new results
-
Every time I see damaged video I can't leave it alone.
Last edited by sanlyn; 21st Mar 2014 at 16:07.
-
I have another Frankenstein version of this video running late into the nite on my slowpoke 2.2GHz (it's sooo 2008, poor thing). Will probably get it posted tomorrow so others can point and laugh. Very nice music clip, but really goofy noise and behavior.
Last edited by sanlyn; 21st Mar 2014 at 16:07.
-
Stilll at it . Reading doom9 while this thing runs again.
Will try to post a new version later today.Last edited by sanlyn; 21st Mar 2014 at 16:07.
-
This is very nice, I like how you corrected the skin color. Their faces look way more natural. However, the walls look too pink in my opinion.. I'm eager to see you color settings.
And it seems you reduced the sharpening in MCTemporalDenoise().. which strength did you set?
And then, did you apply other additional filter ? Why do you want to use RemoveDirt()? -
Hi, mathmax. It will take a while this morning to get some documentation/clean up scripts, etc., I have to help the wife with chores (wives do not understand that video should take priority over food). I gave just enough time to give a quick answer.
The walls are the same hue, just darker. Done to prevent big bad circles of color banding near the bright lights. Leave the lights and brighter parts of walls at high RGB and they dominate the picture. Depends on whether you want the viewer to be overcome with lights or to see more of what's in the room. It could be handled according to taste, but glaring light with crushed shadows looks unnatural. I didn't spend a lot of time on color in that scene, as no one knows what it really looks like in that room except the people who were there.
Color was adjusted with ColorMill and gradation curves. I really have to find time to get deeper into SmoothLevels/SmoothTweak. I used ColorYUV to prevent clamping, but SmoothAdjust seems more compliex to handle. But many use the newer version 2, so I'll give it a try.
The idea was to use weaker versions of filters rather than blast problems with one big one. Most of the five or six filters were scaled down. The main targets were bad shadow noise, crushed darks and hot highlights, motion ripples, banding, Trying to keep some of the objects in the picture nailed down won't be easy (that's one unfortunate side effect of high-compression. High compression and low bitrates = no motion control whatsoever and no detail at luma and color extremes).
Step one was RemoveGrain with MCTemporalDenoise at medium, with some values turned down in both. Followed in step 2 by TemporalDegrain and weaker versions of deblockers. RemoveDirt was run at low power, but somehow I left it in the script twice after moving it around in the sequence. Last step was NeatVideo turned just about "off" everywhere except for high temporal settings on low-level noise only (which amounts to running a motion-compensated deblocker and motion stabilizer. It has very little effect on grain). RemoveDirt does have a motion stabilizing effect on hopping and rippling objects, but it often sharpens edges too much. I don't think the sharpening can be adjusted, maybe I'm wrong there. I stayed away from sharpeners.
Look at earlier versions of the video and pay attention to simmering, boiling, blocked-up darks in faces, hair, clothing. Degraining is not that difficult. Motion noise and crushed detail are the pits. Also in the shots of the microphone room with the camera in a side view (both singers in semi-profile), look at objects in the shadows behind the singers.Last edited by sanlyn; 21st Mar 2014 at 16:08.
-
The video has 19 camera shots from 5 scenes. The Avs scripts for the full-video AVI have basic level adjustments to avoid clipping. Some clipping came with the source and can't be fixed. After running NeatVideo in RGB, the 19 shots were levels&color-corrected separately. Some shots that were the same lighting, angle, etc., used common corrections. Each scene was worked in AVI, rendered separately to MPEG2 in TMPGenc Plus, then assembled with TMPGenc MPEG Editor v3.
One problem was the dissolve between the end of scene 3 and start of scene 4. Scene 3 is too bright and green (apparently shot thru layers of glass). Scene 4 needed different corrections. If you correct scene 3, the dissolve-in to scene 4 looks horrible. If you correct scene 4, scene 3 looks horrible. Thus, each scene was corrected as needed in AVI, then dissolved together over the "real" dissolve for 35 frames in Avisynth, then made into a single MPEG.
The images below show the problem if you simply patch in the two scenes, without separate corrections for each scene. the corrections for one will screw up the other:
(left) scene 3 - no correction, before dissolve. (Right) scene 4 - no correction, after dissolve.
[Attachment 11279 - Click to enlarge]
Same frames with scene 3 corrections for both: (left) scene 3, (right) scene 4.
[Attachment 11280 - Click to enlarge]
Same frames with scene 4 corrections for both: (left) scene 3, (right) scene 4.
[Attachment 11281 - Click to enlarge]
from the mpeg, showing separate corrections and an Avisynth dissolve over the "real" dissolve (12 secs, 9-MB).
http://dc473.4shared.com/download/xlEQaMto/Dissolve_V6.mpg
RGB color correction and NeatVideo: these areas are usually subject to personal preference, and they're difficult to describe step by step. Describing the steps usually takes longer than doing it. The attached Version6_settings.zip has the NeatVideo noise patch and noise settings I used after running script #3 and going to RGB.
Version6_settings also has the VirtualDub .vcf settings files for scenes 3 and 4. These two shots are of Gibbs and MJ at the microphone (scene 3 is the head-on shots). They make up most of the video and used common settings, each tweaked now and then. A .vcf lists filters and their settings. It loads these into ViurtualDub using the "File -> load processing settings..." menu. The controls were gradation curves and ColorMill. If you don't have these in your VirtualDub plugins, VDub won't load the controls and gives an error message that you can ignore. There are two instances of gradation curves in each scene.
The three separate .avs scripts are attched below. I'll do more Frankenstein experiments with the video, and start learning more about SmoothAdjust().Last edited by sanlyn; 21st Mar 2014 at 16:10.
-
Thank you very much for your posts. I'm sorry for my late answer, I have been busy this week and away from video stuff in spite of myself..
I want to quote your script and answer point by point:
STEP 1
Code:MCTemporalDenoise(settings="medium",sigma=6,DeBlock=true,useQED=true,quant1=8,quant2=16)
First, I would desactivate the sharpening cause in my opinion that needs to be after the denoising filters.. if each denoisers applies its own sharpening, the video might looks a bit unnatural at the end.
Then, I would use twopass=true... of course the script is slower to process, but I think the difference worth it. You can denoise more keeping more details than using a stronger value in single pass mode.
I would use chroma=false, because the noise is really a luminosity problem.
I would use stabilize=true because it helps with our rippling objects in the background and the noise on camera motion.
Finally, I would keep strong values for thSAD, thSAD2, thSCD1, thSCD2 because it reduce the grain that appears around motion (for example around Michael's hand when he moves it).
Code:new_clip=interleave(eRG,oRG).weave()
STEP 2
Code:TemporalDegrain(sigma=10)
STEP 3
Code:gradfun2dbmod(thr=1.8,str=0.9,strC=0.9,mask=false)
Code:AddGrainC(var=2.5,uvar=4.0)
STEP 3
Code:RD=in1.RemoveDirt(5,0,false) in2=MergeChroma(in1,RD)
I thought this function was used to removed dust on old film.. I might be confusing though...
Concerning your settings on colors, I really love how you fixed each scene individually (thank you for the screens, very helpful to illustrate the problem with these two sequences). I would love to have the settings for each scenes.
I'll play more with the curves but I just wonder why you boost the green and red (i.e yellow) especially on high value... whereas the bights are already saturated with yellow.
And also, the saturation +5 makes the walls too pink in my opinion.. but you might have used different settings for these scenes.
I wonder why do you need to convert to RGB, since these plugins work on YUV too..
I'm impressed by the quality of the final video. It looks calm and more natural than before... You're probably right to use softer settings for MCTemporalDenoise(), the video looks less like a cartoon as it was a big problem before.
Thank you again for all your efforts on this video. I really appreciate your work. -
Those are all good questions, and yes, there are any number of changes one could make in MCTD. I fiddled with many of them. I can see that some of your suggestions could give better results. I wouldn't completely disable sharpening, but in MCTD it occurs after denoising. Sharpening of some kind is used by most of the other functions called by MCTD. As far as I can tell, sharpening appears to be optimized in those functions to the point of outputting the same amount of detail as was input - though obviously some plugins are softening a bit much, such as FFT3D's sigma value. The color noise makes everything look corrupt, so I used mergechroma later. That was strictly a choice of priorities, and it gained something in color depth; skin looked somewhat pasty otherwise.
Removing fields removed too much information, and I got some motion smearing with QTGMC. It did nothing to remove shimmering in shadows, and seems to have made it harder to handle. Otherwise it didn't seem worth the effort. I even tried yadif on this thing -- a dumb idea I admit, but wow, talk about strange results! I questioned the logic of processing both fields and then discarding half of them.
NeatVideo is pretty adept at that kind of noise, but RemoveDirt works on that as well with its restoremotionblocks routine. TemporalDegrain targets grain, with not much effect on motion blocks.
There's a lot of macroblock "movement" in shadow areas, and the plugin does help with the shimmer a bit by addressing banding issues and smoothing ripple somewhat.
I don't discern a lot of difference, but GrainFactory's main component is AddGrainC. AddGrainC works in YV12, YUY2, and RGB, GrainFactory requires YV12 only. Grain was added in stages to prevent an over smoothed look. It also helped with the look of bright lights shining on walls, which showed strong banding.
It is often used as you describe, but it doesn't work well on very strong grain IMHO. It cleaned up some motion and chroma noise and actually restored a lot of smeared detail by working on motion blocks. I cut the strength to prevent over-sharpening on edges. It smoothed those shadow problems quite a bit. Look at earlier versions of the way shadows "simmer" on those music stands and the way background objects in those shots seem to "boil".
The vcf files will load the controls with the settings I used. The vcf might give a problem loading NeatVideo, because the vcf contains the paths to NeatVideo's files. NeatVideo will still load despite the message, but you have to tell NeatVideo where the .dnp and .nfp files are located. VirtualDub's vcf always looks for the basic plugin vdf's themselves in the default plugins folder.
The settings used more than one curves filter. One is a primary correction, the other(s) are secondary tweaks. They are loaded in the proper order; reversing them could have different effects, as each is a tweak for the other.
I don't recall exactly how the curves filters were set up, but remember that each corrects a specific area of the color and levels spectrum. This was necessary because of the mixed lighting used in these shots and the rather aggressive action of the webcam's auto-exposure feature - these tended to brighten shadows unnecessarily and wipe out highlights. If available, I would have used an external spot meter for those scenes and would have set exposure manually. Much of the indoor lighting in these shots appears to be incandescent, which is warmer than other lighting. Trying to make those lights "white" made skin tones look unnatural.
Agreed. After looking at it for a while, I decided my saturation experiment really didn't do what I wanted with the instrument panel in that shot. Oh, well, some ideas look good at the time but on playback they often look silly.
As far as I know, the VirtualDub controls won't work in YUV. They can be loaded in Avisynth with the proper script, but you still have to use them in RGB.
It appears that the FFT3D component in MCTD was solving a few problems, but over correcting for others. As you suggest, many other adjustments could be made and I even ran some of MCTD's components by themselves and observed the effects. Given the time restraints (I have two other projects at hand that must be completed soon, and some of their problems are worse than these!).
I made yet another version. STEP 2 used the same SeparateFields technique as STEP 1, but I eneded up with even more tweaking than before. However, it made the ripples in that zoom shot very nearly disappear (after NeatVideo got its hands on it). I don't know why this works. Really. I maintain that at some point this video was interlaced, possibly when converting from DV to standard DVD and then incorrectly deinterlaced (or) treated as non-interlaced, or whatever. This didn't work on my other videos, and I don't recall ever having used it before. It happened here quite by accident. I simply have no explanation, nor does anyone else who has responded to this thread. Unless the webcam has a built-in dissolve feature, I'd say the original homegrown video has gone through extensive post-processing.
I also noticed this: the first shot thru the zoom shot have much more noise and wiggling than other scenes. I don't know why. More camera movement, perhaps. I might try a small dose of Stab() on those shots and see what happens, but the ripples are pretty annoying. Any stabilizers or activating DePan would involve border cropping I didn't want.
I also used SmoothLevels more extensively and eliminated ColorYUV. Thanks for leading me to re-investigate the newer version of that plugin. I didn't care for the earlier one, but that was quite a while back.
I will try to post a bit of the newer effort, a little later. I'm doing this post on another PC right now.Last edited by sanlyn; 21st Mar 2014 at 16:10.
-
Thank you
I now try to test each functions separately to see their effect..
I have a problem with RemoveDirt() though.. the version you used is quite different than the version on the website (the parameters don't match). Could you upload your avsi?
I also wonder why you need to tweak the curves in two steps.. is it just a personal technique to be more accurate?
And I try to open the original .m4v directly in virtualdub.. both colormills and gradation curves work on it although it's not RGB. So I wonder if the conversion is needed.Last edited by mathmax; 15th Mar 2012 at 11:32.
-
Can you reupload the original sample mathmax, i wanna test a few filters on it
*** DIGITIZING VHS / ANALOG VIDEOS SINCE 2001**** GEAR: JVC HR-S7700MS, TOSHIBA V733EF AND MORE -
I think sanyln reuploaded it in earlier post
http://dc402.4shared.com/download/Osc_fd2F/AllinYourName.m4v
https://forum.videohelp.com/threads/341917-remove-noise-and-block?p=2145097&viewfull=1#post2145097 -
I've seen several versions of RemoveDirt's function, and even some other function names -- including "RemoveSpots(), for one, which is a dandy spot remover mod by jagabo. RemoveSpots uses the same RemoveDirt text seen in the html, which is actually a sample script provided by the author. I've seen several mods of RemoveDirt, but the one I use and which is posted in several threads on this forum and on doom9 is here:
Code:function RemoveDirt(clip input, int limit, int rgrain, bool "_grey") { _grey = default(_grey, false) _dgr1 = 0.35+rgrain*0.3 _dgr2 = 0.45+rgrain*0.4 repmode = 1 clensed = Clense(input, grey=_grey, cache=4) restore = input.FluxSmoothST(3+3*rgrain,rgrain) restore = Repair(restore, input, mode=repmode, modeU= _grey ? -1 : repmode) restore = rgrain==0 ? restore.RemoveGrain(1) : \ restore.VagueDenoiser(threshold=_dgr1, chromaT=_dgr1, nsteps=7, percent=75).RemoveGrain(1) alt = input.VagueDenoiser(threshold=_dgr2, chromaT=_dgr2, nsteps=7, percent=100).RemoveGrain(5) return RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, pthreshold=4+2*rgrain, cthreshold=6+2*rgrain, gmthreshold=40, dist=1, dmode=2, debug=false, noise=limit, noisy=12, grey=_grey) }
Unfortunately, unlike the more professional Adobe versions of curves, it has only one undo step, which is the Cancel button. If you've made 10 changes but only want to keep the last one or two, it's often difficult to remember what you did. Hit Cancel and -- poof -- you're back 10 steps earlier. It's not unusual to use primary and secondary color correction filters anyway, it's standard practice in Premiere Pro and After Effects.
Because opening the clip in VirtualDub and applying any of its plugins and re-saving automatically converts the output to RGB. In this case it works OK, but for many DV's VirtualDub will make assumptions that can screw up the colors. Just viewing a YUV clip in VirtualDUb and closing it (without a re-save) has no effect on colorspace. I always go to RGB in Avisynth, just to be sure. Once I'm in RGB I stay there. If I have to go back to YUV for encoder or whatnot, I use Avisynth again and carefully check the conversion results.
I've just been playing with QTGMC again. Getting some interesting effects. Later.Last edited by sanlyn; 21st Mar 2014 at 16:27.
-
The top link is a direct download to the original m4v. The lower link is to a thread posted earlier for jagabo. As i stated, even a few seconds of lagareith AVI would be a bigger download than the whole m4v.
Have at it, poisondeathray. If anyone can make an improvement with this goofy clip, you can. I'll be tuned in.Last edited by sanlyn; 21st Mar 2014 at 16:27.
-
-
Hmm. It has to be on DVD somewhere. Or should be. But I wouldn't learn anything new, and it wouldn't be any fun!!
Last edited by sanlyn; 21st Mar 2014 at 16:27.
-
I have to give up on QTGMC as a fix for comb effects. It did smooth combing but caused other problems (smearing). This crappy video is in constant motion even when nothing moves, especially in darker areas.
Meanwhile I tried the O.P.'s suggestion of running MCTD with twopass=true. DIdn't solve many new problems, but retained detail looks much better. I added these settings to MCTD's parameters: twopass=true,limit=1,limit1=1. Slowed processing by about 10%, not bad. Will keep on tweaking til it looks better.Last edited by sanlyn; 21st Mar 2014 at 16:27.
-
-
You're right.. we should do that. But that is not an easy thing to obtain..
Once our work on the video will be finish, I'll request for another transfer and show him the result we could get just by cleaning his poor m4v. I think it will be more convincing if we have something to present.
sanlyn, there is no combing artifact in the last sample you sent. I guess the following denoisers and other smoothers could make them disappear... however, I'm afraid we lose details with this technique. Did you try to render the whole thing on progressive frames? -
Making progress, mathmax, QTGMC not withstanding. MCTD with two-pass retained more detail (what there is of it to begin with).
I agree with poisondeathray, this clip must have been interlaced at some point -- which might explain how STEP 1 seems to work when "forced" to look that way, even if it's a renegade method. Apparently there is some slight loss of information doing it that way, but I think most of that loss is noise. I kept trying step 1 as pure progressive, but what a mess. Tried Step 2 as interlaced, just as in step 1, but no gain there and the clip started looking somewhat denuded. I also modified the other filters, but I'm going back to step 1 again to see what to do with those "wiggles". Every attempt to smooth them over just wipes out detail, with little gain otherwise. I even tried reversing field order (!); the results in step 1 looked exactly the same. But things are getting visibly cleaner and more stable, with detail is improved. I'll work more tonight.Last edited by sanlyn; 21st Mar 2014 at 16:28.
-
Hi, mathmax, sorry I had to leave a while, had two PC repair customers and a very elderly father-in-law in his 2nd childhood who will not behave! Pretty much shot the weekend.
Making progress, even if it is a tweak of the earlier routines. Stiil, visible improvement. I'm rebuilding a PC but will hop back and forth between these two projects, so it will take a day or two.Last edited by sanlyn; 21st Mar 2014 at 16:28.
-
Sanlyn,
Trying to read through this thread and figure out what you are doing with this video. Is the source video a downloaded FLV file?
Next question does it have shifting motion blocks in the video?
What are you doing to fix the motion block issue? -
The original is m4v. It's my understanding that the uploaded m4v is a copy of the home-made DV movie being touted as "rare" (whatever that means, but the same copy is all over the internet), and according to the O.P. it is sold ( for real $$) in its present condition. Original explanatio0n in post #5 of Dec23 (https://forum.videohelp.com/threads/341917-remove-noise-and-block?p=2129424&viewfull=1#post2129424) .
Lordsmurf's assessment seems to be a good explanation of the mess, in post #13 on Dec28 (https://forum.videohelp.com/threads/341917-remove-noise-and-block?p=2130585&viewfull=1#post2130585 ). Poiosndeathray agrees, it's a mess, on 16March a few posts back.
My copy of the posted 39MB m4v is here (free download): http://dc402.4shared.com/download/Osc_fd2F/AllinYourName.m4v
Motion blocks? No, the problem is motion, period, and that's not the least of it. You can throw motion-compensated smoothers at it all day long. All you get is more loss, less gain. But we're making progress, however slight. Give it a try.Last edited by sanlyn; 21st Mar 2014 at 16:28.
-
Sanlyn,
Have enough problems dealing with my videos, motion problems can be a nightmare and almost impossible to fix. It can be something simple but more than likely someone messed it up pretty bad. Now you have to deal with that mess. Email the guy who is selling this video and get the source recording and than you can make it perfect.
A $1.99 downloaded Michael Jackson video? -
Not my video, I just "adopted" it as a learning experience. It's that, for sure. The shimmer and wiggles will likely never go away without some very sophisticated motion adaptive work in high-end software. But making progress. Will have something in a day or two for mathamx, but had three PC repair clients keeping me busy (and paying for some hardware I need).
Last edited by sanlyn; 21st Mar 2014 at 16:28.
Similar Threads
-
How to remove audio noise?
By Leo48 in forum Newbie / General discussionsReplies: 4Last Post: 26th Jul 2013, 15:09 -
How to remove noise from my HD video
By ITW in forum RestorationReplies: 18Last Post: 26th Jan 2011, 12:24 -
Multi-colored block noise bug in Adobe Premiere - DV video
By tat2jr1 in forum EditingReplies: 0Last Post: 23rd Jan 2010, 02:20 -
How do I remove/block Flash ads?
By videohalp in forum ComputerReplies: 4Last Post: 26th Sep 2008, 19:45 -
How to remove that type of noise or whatever it is?
By Srivas in forum RestorationReplies: 4Last Post: 4th Dec 2007, 06:50