Doom10 died without notice, so I hope to make videohelp the new home for SMDegrain() and the rest of tools and scripts I made over time.
SMDegrain is simply a wrapper for mdegrain, nothing fancy or magical. Just convenient automations for common tasks to get things done fast and in good quality, without forgetting some flexibility too. I created this based off my needs, and evolved from there. I took Caroliano's SMDegrain() as a base to build it because I also wanted something simple to use and straight to the point, in the same sense as the original script indicates: "To make your scripts shorter and less geeky". Think of it as a front end for mvtools2+mdegrain.
The main features of this function are:
High bitdepth denoising
Higher temporal radius (tr>3)
Prefilter and Motion Filter support
Motion Vectors Globals Input/Output
RefineMotion Option (MRecalculate)
+some more little options and any combination of the above supported
Everything is OFF by default so you just type smdegrain() and it works, but as soon as you start setting more parameters on you will find to need more plugins and scripts. To run it you will need at least mvtools2 and masktools2. Read on dependencies to know more.
A simple SMDegrain() call (for non HD sources) would be equal to:
As you see, nothing too outrageous, just a nice good code block you save from typing and hence human induced errors.Code:super_search = Dither_Luma_Rebuild(S0=1.0,c=0.0625).MSuper(rfilter=4) bv2 = super_search.MAnalyse(isb = true, delta = 2, overlap= 4) bv1 = super_search.MAnalyse(isb = true, delta = 1, overlap= 4) fv1 = super_search.MAnalyse(isb = false, delta = 1, overlap= 4) fv2 = super_search.MAnalyse(isb = false, delta = 2, overlap= 4) MDegrain2(MSuper(levels=1), bv1, fv1, bv2, fv2, thSAD=300, thSADC=150)
I have to specially thank cretindesalpes for bearing my incessant questions related to this function.
SMDegrain() Mod v3.1.2d (21-07-2015)
Code:v3.1.2d: - Fixed regression where external prefilters wouldn't be parsed for luma expansion v3.1.1d: - Workaround to force KNLMeansCL use discrete video card (now required) v3.1d: - New prefilter mode 4 for grainy sources. GPU based spatio-temporal KNLmeans - Added dither output mode "Random Dither". - Added "slices" option from Dither - Added soft=-2 (automatic) to internal lsfmod - Added assert for chroma denoise when luma only motion vectors (mvtools bug) - Improved lsb_in code (now using sub/add method) - Removed vpad/hpad=0 optimization preset for HD - Fixed prefilter=3 code for lsb_in YUY2 - More fixes to Show Panel... - Updated and fixed documentation (notes on YUY2 support) v3.0d: - Adjusted SD<>HD discretion - Added mfilter (Motion Filter) new feature - subpixel=3 (nnedi3); removed qual=2 due to possible blending bug (quote by cretindesalpes), this also boosts speed - Lowered the defaults for less aggressive denoise (still somewhat high) - Fixed chroma not showing in debug (show) mode when lsb=true and prefilter=3 were paired - Fixed long lasting issues in Show Panel for certain not mod height resolutions - Now prefilter=0 is minblur(0), lighter denoising than prefilter=1, default is now -1 (off) - Documentation update (plugin updates, notes, links, examples, etc) v2.2d: - Removed one residual variable in Contrasharpening function - Fixed a regression at v2.0 for Interlaced YUY2 sources by changing wrong variable ifC to if0, and... - ...adding a missing weave() to the last MergeChroma() call - Dropped Dither support. (Will still work/not work as before, but lack of official support renders it to a YMMV condition) v2.1d: - Fixed Refinemotion Globals Input labels - Fixed ContrasharpeningHD Vectors for Interlaced HD sources - Added ability to import Global Vectors even when "tr" is lower than parent instance "tr" (credit to cretindesalpes) - Expanded support for Interlaced Denoising "tr" up to 64 (credit to cretindesalpes) - Cosmetics v2.0d: - Fixed and optimized chroma handling for lsb_in, contrasharpening and YUY2 content v.1.95 - Enhanced Chroma bypass on the sharpening stage - Merged ContraHD() internally for Contrasharpening in HD sources - Dropped QTGMC support on import/export motion vectors (small value for increased confusion) - Fixed Median code for planar inputs in MinBlur() (needs RemoveGrainHD, dropped medianblur requirement) - Defaulted truemotion to false for HD inputs - Code Optimization and Clean up - Others v1.9d: - Added truemotion parameter (default=true). Now you can change it to false (as in MCTemporalDenoise()), so low frequency detail won't be lost (walls...), although it could leave more noise on high frequency details - Added thSADC, and thSCD1/thSCD2 parameters for finer control - Tweaked prefilter=3 dfttest sigmas for better low frequency detail protection - Detached explanation and converted to html. Updated and extended the contents as well - Fine tuned and robustized the Show Panel - Optimized and organized code, more precisely the contrasharpening part - Fixed a few ternary checks for when Globals=3 - Others v1.8d: - Added automatic dark protection for prefilter=3 (dfttest performs badly in darks) v1.71d - Fixed chroma variable when chroma=false, from 2 (copy chroma) to 1 (discard chroma) v1.71d - Explanation and cosmetics v1.71d - Added acknowledgments and updated explanation - Added brackets to all functions (it has been reported to cause problems on occasional circumstances) (http://forum.doom9.org/showthread.php?p=1515886#post1515886) - Reworked and fixed the parameters panel. Now you have a color guide to have an idea on what to tweak depending on selected mode in Show. For strings: "Speed", "Memory" and "Quality" - Optimized lsb_in for over 720p HD resolutions which was causing memory hogs - Fixed contrasharpening for YUY2 sources when lsb=true - Fixed some inconsistencies for YUY2 sources when prefilter=3 and extended protection a bit v1.7d: - Fixed and extended introduction explanation v.1.61d - Fixed some issues for Globals=1 (Read) in relation to lsb_in and luma expansion auto-prefilter v.1.61d - Added mild gauss blur prefilter option as prefilter=1. Consequent modes are displaced v.1.61d - Changed prefilter 1 and 2 to Minblur (Gauss/Median combination) v.1.63d - Updated MinBlur function v.1.62d and v.1.66 - Code tidied up v.1.62d - Updated luma expansion, now you can enhance darks (with 'Str' and 'Amp') in the same step as the TV->PC conversion stage, thus +optimized, +quality. v.1.63d by cretindesalpes (http://forum.doom9.org/showthread.php?p=1548318#post1548318) - Removed pre_custom. Now use prefilter to load a prefiltered clip, as well as modes 0~3 - Added Show parameter. Now you can see the used and default parameters along the prefilter clip as well - Fixed one note related to prefilter clip input for interlaced content v1.6d: - Fixed and extended introduction explanation - Fixed Ditherpost for interlaced sources - Fixed a broken reference for reading MRecalculate globals when tr>3 - Fixed a Globals=1 MSuper automatization - Fixed a plane processing bypass when chroma=false - Added LSFmod as optional contrasharpening (slower, but biased contrasharpening and probably better depending on source) - Added default luma expansion auto-prefilter for motion search (16% more values) - Added YUY2 support - Added the 'mode' parameter of ditherpost() - Added support for interlaced temporal radius up to 6 v1.5d: - Changed "srchSuper" Global variable to "QTGMC_srchSuper", compatible with QTGMC. Anyhow you may probably want to create a new super clip. Test it out! v.1.41d - Introduction spelling and presentation v.1.42d - Added lsb and lsb_in for prefilter=2 (dfttest) v.1.43d & v.1.44d - Some better definitions for the settings help v.1.45d - pelclip is now subpixel = 3 ('sharp' parameter before) v.1.45d - Other minor tweaks v.1.45d - Fixed an important issue when using prefilter with subpixel=3 ('pelclip' parameter before) - Optimizations in ditherpost for chroma - Changed rfilter (hierarchical levels smoothing/scaling) from default 2 to 4. Better results in my judge - Some more little tweaks in code and introduction v1.4d: - Added pelclip option for top quality subpixel interpolation when pel > 1 - Changed QTGMCV parameter to Globals, and added a bunch of new features like vectors output from smdegrain() - Big introduction and code optimization tweaks v1.3d: - Fixed an important issue on the Interlaced parameter and improved overall handling (v.1.21d & v.1.3d) v1.2d: - Added Interlaced parameter, for processing interlaced sources - Some introduction and code optimization tweaks (v.1.12d & v.1.2d) v1.1d: - Implemented MDegrainN for tr > 3, from the MVTools2 mod of Dither - Minor introduction and code optimization tweaks v1.0d: - Added CClip parameter. Reference an earlier stage of your processing chain as your sharp version for the contrasharpening v0.9d: - Added QTGMCV parameter, for reusing vectors from QTGMC bob deinterlacer v0.8d: - Fixed a few things related to the lsb_in option - Minor introduction and code optimization tweaks v0.7d: - Implemented "fake" lsb_in option. Output will have slightly more quality (if any), and will compress better. (Experimental) - Fixed a not passed argument in the hpad, vpad parameters v0.6d: - Minor introduction and code optimization tweaks v0.5d: - Added dfttest option for preblur (now prefilter) option. Slower but better due to its internal deblock+denoise operation, plus sharper results - Added pre_custom option for the prefilter process, define here a denoised clip as your prefiltered version - Updated the introduction help v0.4d: - Added preblur option for slight vector blurring where motionmatch is bad (http://forum.doom9.org/showthread.php?t=161594) - Added RefineMotion parameter for better motion vectors. Based off Killer() (http://forum.doom9.org/showthread.php?p=1266572#post1266572) - Added parameters "lsb" and "lsb_out", for the MVTools2 mod version of cretindesalpes' Dither (http://forum.doom9.org/showthread.php?p=1386559#post1386559) - Other minor aesthetics and performance adjustments v0.3d: - Fixed plane (for MDegrain) and added chroma (for MAnalyse) parameters v0.2 : - Added introductory section - Added more parameters v0.1 : - Basic working version - Not released
+ Reply to Thread
Results 1 to 30 of 210
Last edited by Dogway; 21st Jul 2015 at 12:48.
I got it. Thanks a lot.
Thanks Dogway. I was looking for SMDegrainMod for the past month. Great script. Thanks.
Does error message tell you what column? Make sure there are no spaces or typos. For example, in refinemotion from the quote from your post , it looks like refinemo_tion in my browser (where "_" is a space)
thank you very much for update it. you are doing wonderful job .
Last edited by BÌG•bµdd; 30th Jan 2015 at 16:40. Reason: nuttin
need an idea to improve Smdegrain(lsb=true, mode=6) for 16bit denoise.
Last edited by BÌG•bµdd; 30th Jan 2015 at 16:38. Reason: correcting
It might just be me, but I wouldn't call a function with around 30 parameters simple.
Also is the 'High bitdepth denoising (not officially supported anymore)' still true? (since it was in the doom10 thread, but isn't mentioned here)
-> Ever thought about adding some presets?users currently on my ignore list: deadrats, Stears555
If it's too complicated for you, ignore the greyed out "Advanced" and "Others" columns, I made it so for people like you, if you aren't satisfied enough, you can use the original smdegrain, but then you won't be able to use 16bit depth denoising, work on YUY2 sources, or even denoise interlaced sources. Who told you avisynth was easy? (!)
You can always use bare mvtools calls or the 82 parameter MCTemporalDenoise() function with minimal in-script documentation, you have many choices.
Last edited by Dogway; 26th Jan 2015 at 07:59.
Dogway! I'm really glad that you found a new home . I was wanting to pester you some more, ha ha, but discovered that doom10 no longer exists. A definite bummer that those threads are gone (although maybe it's a good thing to have some of my ignorant idiocy no longer available to the public ). Anyway, as I've poked away at My Seemingly Endless Video Project, using the following basic script (with the DAR info changing with the source)
# Set DAR in encoder to 16 : 9. The following line is for automatic signalling global MeGUI_darx = 16 global MeGUI_dary = 9 LoadPlugin("F:\StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\ffms\ffms2.dll") Threads=8 SetMTMode(5,Threads) FFVideoSource("D:\ReEncTemp\AnyVideoFolder\AnyVideo.mkv", fpsnum=24000, fpsden=1001, threads=1) # SelectRangeEvery(1000,66) SetMTMode(2) #deinterlace #crop #resize #denoise SMDegrain(Lsb=True,Chroma=True,Plane=0)
I've experienced consistently great (to me) results, never having any problems with sources from 480p to 1080p.
Recently, however, I've had some issues running the above script on several 1080p videos that have first been DeLogoed in VirtualDub. Using either UtVideo YUV420BT.709 VCM or single-pass lossless x264vfw for the intermediary file, the resultant videos have all suffered from mild to medium stuttering or jerking (I'm unsure of the proper term) during playback; however, using single-pass crf16 x264vfw for the intermediary file results in a video with no stuttering or jerking during playback. It seems to me that this is an issue of the source file's bit rate, as if my script can't keep up, and was hoping that perhaps you had some ideas. As always, thanks for your time.
Last edited by LouieChuckyMerry; 29th Jan 2015 at 07:17.
You shouldn't judge your results in playback. Go to virtualdub or avspmod and skip frame by frame to see if all the frames are rendered.
My opinion, lossless content is too heavy for smooth playback. Use cheaper encoding entropy codecs-read faster (huffyuv, ffv1) or as I said check on a frame by frame basis.
Thanks for your reply, Dogway, and Happy Friday . I think that I wasn't very clear in describing my issue, so I'll try again.
My objective is to remove a TV logo from an HD source by first using VirtualDub-DeLogo and outputting to a lossless intermediary file, then cleaning and shrinking this intermediary file. My problem:
1) Running any uncorrupted 480p, 720p, or 1080p source through MeGUI-AviSynth-SMDegrain-x264crf16+ with my above standard script--SMDegrain(Lsb=True,Chroma=True,Plane=0)--returns great (to me) results.
2) Running a 1080p source through VirtualDub-DeLogo and outputting to either UtVideo YUV420BT.709 VCM or single-pass lossless x264vfw results in a file that playsback perfectly with multiple players, but if I then run this file through MeGUI-AviSynth-SMDegrain-x264crf16+ the resulting file suffers from occasional to frequent stuttering-jerking during playback.
3) However, if in the VirtualDub-DeLogo phase I output to single-pass crf16 (instead of lossless Ut or x264) and then run this through MeGUI-AviSynth-SMDegrain-x264crf16+ the resulting file playsback perfectly.
These facts make me think (stop laughing) that there's an issue with SMDegrain (or at least my script) and lossless (or at least Ut or x264) video, like the script can't "keep up" with such high bit rate, or some such. Of course, I'm just a noob with a minimal skill set who's just guessing based on his limited knowledge base. Anyway, my problem isn't with the playback of lossless video but with applying SMDegrain to lossless video. Thanks again for your time and sorry for my confusing previous post.
Last edited by LouieChuckyMerry; 29th Jan 2015 at 21:29. Reason: Syntax & Grammar
It makes little sense, I never had avisynth skip frames.
Aside (or not) of this it's possible you are doing too much on one shot, deinterlacing (qtgmc?) and smdegrain in one single pass for 1080p it's probably too much for any given PC setup. Before I recommended mp_pipeline (check doom10 archived pages), it handles memory in a safer way.
If I were you I would try to do the delogo in avisynth along the deinterlacing, and output this to an intermediary file, then use this file for your second pass; resize, denoise, etc.
UtVideo or lossless x264 for the intermediary file, then the result of the second pass (SMDegrain only, the very simple above script) suffers from mild to medium stuttering-jerking (this has happened with 3 different HDTV sources); however, if I use x264crf16 for the intermediary file, then the result of the second pass (SMDegrain) playsback perfectly. Both second passes run in MeGUI without any problems but the outputs for the 2 different lossless types have occasional stuttering-jerking (ghosting?) during playback. At first I thought it was the UtVideo, but the problem occurs with lossless x264 as well. Presently it's not a huge problem because I can't see any difference in quality on my 14" screen, I'm just trying to figure out why this happens to scratch a brain itch and because maybe in the future I will be able to see a difference in quality on a larger screen.
Last edited by LouieChuckyMerry; 1st Feb 2015 at 02:40.
without the actual scripts and source it's hard to tell and I can only guess. Maybe you are mistakingly passing an interlaced flag on encoders? have you tried without smdegrain? raising setmemorymax()?
Maybe it's better you open a dedicated thread so anyone can help you.
Sorry for the delay in replying, Dogway, I was gone for a while. All three sources are HDTV rips with AVC video. My first step for each is to run it through VirtualDub with DeLogo, no other filters, saving the output as an .avi file; the second step for each is to remux it as an .mkv file then run it through MeGUI with the very basic script that was in my first post. Really, that's it. The issue is with the intermediary file: if I use UtVideo or lossless x264vfw the result of the second step suffers from occasional stuttery-jerky playback. As I typed before it's not a huge problem but more my curiosity being piqued; using x264crf16 for the intermediary file eliminates the problem and my eyes can't see a difference on my laptop screen. I just thought that perhaps you knew of a specific issue with SMDegrain and lossless video, so I posted here. If somewhere in the future I decide to rereencode the HDTV rips in an attempt to upgrade the quality I'll open a new dedicated thread. As always thanks for your time and be well, and I'm really glad you're back online .
Hi Dogway, I hope you've been well, and Happy Friday . A simple question for a change: please, what's the difference between versions 2.2d and 2.21d? I've been using 2.2d because my brain was seeing "2 bigger than 1", but now that I think about it I'd reckon that 2.21d is the newer version. Thanks.
what's the difference between versions 2.2d and 2.21d?
Having a look I think I was addressing a chroma bug where prefilter=3 (dfttest) was used. It was a minor fix after someone called out the bug, so I never released it wide open. Use it if you use prefilter=3.
edit: I think I'm going to release an updated revision with this and a few other minor things that stacked up in time.
Last edited by Dogway; 26th Mar 2015 at 23:06.
Thank for that, Dogway . I'm excited to check it out...
Hi Dogway, now to pester you on both your threads simultaneously . I've spent most of my day trying to get v3.0d running with your recommended tp7 mods, and I think that the first option here (masktools2-25-x86.zip) is the correct choice for the "MaskTools2" mod. However, I'm not sure what to use for tp7 mods of "Repair" and "RemoveGrain". Do I simply add the correct RgTools.dll from here to my plugins folder with the original versions of the RepairSSE2.dll and RemoveGrainSSE2.dll there as well? That seems entirely too easy (so it's probably the way to do it...)
Also, what do you mean by "Adjusted SD<>HD discretion", the first line of the ChangeLog?
Ahhh, just so you know, a "Show=True" call shows "tr=2" and "thSAD=300" for defaults, but the .html file still lists "3" and "400" .
Last edited by LouieChuckyMerry; 29th Mar 2015 at 07:33.
What do you mean defaults were the same? A few things change, hpad/vpad, blksize, pel, subpixel...
Remove old Repair and RemoveGrain and drop RgTools.dll in the folder. RgTools has the old function calls embedded in the plugin (ie. Removegrain(), etc). If it fails probably you removed something else, did you keep RemoveGrainHDS.dll?