VideoHelp Forum

Try DVDFab and download streaming video, copy, convert or make Blu-rays,DVDs! Download free trial !
+ Reply to Thread
Page 1 of 5
1 2 3 ... LastLast
Results 1 to 30 of 125
Thread
  1. Hi

    I have a video that suffers two problems. It's badly compressed and it has a lot of noise. I could use Deblock() from Manao but I wonder which denoiser to use... also I wonder in which order should I process: first denoise and then deblock or the contrary?

    Here are two screeshots (the blocking is irregular, very present on some frames and not so much on other..)

    Click image for larger version

Name:	screens0000.jpeg
Views:	988
Size:	202.4 KB
ID:	10218
    Click image for larger version

Name:	screens0001.jpeg
Views:	1072
Size:	59.4 KB
ID:	10219

    You can download a sample of the video here: http://www.sendspace.com/file/glg6qo

    I know the compression doesn't help to denoise.. and I don't expect a miracle. But I hope I can enhance even a bit..

    Thank you in advance for any advices
    Quote Quote  
  2. aBigMeanie aedipuss's Avatar
    Join Date
    Oct 2005
    Location
    666th portal
    Search Comp PM
    get the original source and start over. or suffer from garbage in garbage out.
    --
    "a lot of people are better dead" - prisoner KSC2-303
    Quote Quote  
  3. Originally Posted by aedipuss View Post
    get the original source and start over. or suffer from garbage in garbage out.
    I don't have the original source. This is the best I can have.
    One particularity of this noise is that it's very bright.. any idea where this could come from? Aren't there any filters adapted to reduce it?
    I tried MDegrain2() which already improves.. but I hope something else can do a better job.
    Last edited by mathmax; 22nd Dec 2011 at 22:56.
    Quote Quote  
  4. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    It would help much if we had some history about how this video was made. It's a cinch you can't clean it up in M4v format, it will have to be converted away from lossy m4v compression.
    Last edited by sanlyn; 21st Mar 2014 at 16:53.
    Quote Quote  
  5. Originally Posted by sanlyn View Post
    It would help much if we had some history about how this video was made. It's a cinch you can't clean it up in M4v format, it will have to be converted away from lossy m4v compression.
    It's sold on the official Barry Gibbs website and the only thing they say is that this video has been shot 2002 with Hi8 camera. And the quality of the video is due to the nature of the recording..

    http://www.barrygibb.com/shop/index.php?route=product/product&product_id=79
    Quote Quote  
  6. have you tried msu smart deblock or deblockqed ?
    Quote Quote  
  7. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    Most of the cleanup would be better done in YUV first, before using RGB filters. The noise is less like block noise and more like severe crosshatching, thick grain, and some really bad compression artifacts. Almost all of it is medium- to low-frequency noise. Note that underneath the noise the "detail" seems better than it should be, but overall this looks like a really bad duping/transfer job -- very nearly the kind of low-quality image you often get with Netflicks, but worse.

    I'd start with MCTemporalDenoise at "very high" on a small section of video, mainly to clear some of the noise and see what's left. A real cleanup will require some fiddling and experimentation. Don't expect BluRay from the results, though.
    Last edited by sanlyn; 21st Mar 2014 at 16:53.
    Quote Quote  
  8. Originally Posted by sanlyn View Post
    Almost all of it is medium- to low-frequency noise.
    so.. should I use any special settings for low frequency noise?

    Originally Posted by sanlyn View Post
    I'd start with MCTemporalDenoise at "very high" on a small section of video, mainly to clear some of the noise and see what's left.
    I tried with these parameters:
    Code:
    loadCplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll")
    FFMpegSource2("D:\Bureau\all in your name\AllinYourName.m4v")
    
    MCTemporalDenoise(radius=3, sigma=16, twopass=true, limit=0, limit2=0, bwbh=8, owoh=4, blksize=8, overlap=4 )
    However, there is a problem this filter.. suddenly, some frames becomes dark. Please watch the following clip:

    http://www.mediafire.com/?dyu11p29fxe6ei2


    Click image for larger version

Name:	screens0000.jpeg
Views:	968
Size:	163.9 KB
ID:	10231Click image for larger version

Name:	screens0001.jpeg
Views:	831
Size:	163.2 KB
ID:	10232
    Last edited by mathmax; 23rd Dec 2011 at 11:21.
    Quote Quote  
  9. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    It's an improvement, anyway. In MCTemporalDenoise you might be giving values to parameters that by default are already set according to the general mode settings (low,medium,high,etc.). You might try MCTemporalDenoise's default mode, as they usually run some very sophisticated routines automatically. Try this one:

    MCTemporalDenoise(settings="very high")

    It will slow it down, but don't expect to find "fast" filters to do all the work this video needs.

    The frames don't darken, they flicker. Several experiments using the vDub Deflick filter called into AviSynth have been done in this thread recently with a script by jagabo: https://forum.videohelp.com/threads/341188-Still-have-flicker-after-Denoising-with-AviS...=1#post2128405

    Since this video is already progressive, you won't have to bother with IVTC, etc., but DeFlick and MCTemporalDenoise are called this way:

    To load DeFlick in Avisynth:

    Code:
    LoadVirtualDubPlugin("[path to VirtualDub folder]\plugins\deflick.vdf","DeFlick")
    Thyen, inside the avs script call DeFlick and MCTemporalDenoise in this manner:

    Code:
    ConvertToRGB32()
    DeFlick(8, 0, 0, 10, 0) #window size, softening, interlaced, sc threshold, show sc
    ConvertToYV12()
    McTemporalDenoise(settings="very high")
    The colorspace changes aren't that critical here, as the video9's not in such great shape anyway, but apparently there's little harm going on in the videos we've used these lines on.

    The html documentation that ships with MCTemporalDenoise has a chart showing the default values and procs called for these mode settings.
    Last edited by sanlyn; 21st Mar 2014 at 16:53.
    Quote Quote  
  10. Thank you for your answer. MCTemporalDenoise() was a good advice.. it indeed improves quite well the clip.

    Originally Posted by sanlyn View Post
    It's an improvement, anyway. In MCTemporalDenoise you might be giving values to parameters that by default are already set according to the general mode settings (low,medium,high,etc.). You might try MCTemporalDenoise's default mode, as they usually run some very sophisticated routines automatically. Try this one:

    MCTemporalDenoise(settings="very high")
    I lose more details on the default mode.. and the noise is pretty much the same...
    Is it possible to set a default mode and override some parameters?

    Originally Posted by sanlyn View Post
    The frames don't darken, they flicker.
    Well, there was no flicker on the original video.. the filter added it. With the default settings, the flicker is less regular but way more strong. Please look at this sample: http://www.mediafire.com/?ibql1v0u31lj97c


    It's a problem with MCTemporalDenoise().. I have to fix it rather than compensate with DeFlick(). Don't you think?
    Quote Quote  
  11. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    Sorry, I'm traveling with a little Netbook. Don't have all my Avisynth metrial or your download with me. Tried all morning to download with a local Starbucks crummy wifi connection, but keep getting cut off. WIll just keep trying.

    Meanwhile, I never saw MCTD cause the problem noted. Likely it's the scene-change setting in DeFlick causing one of the problems, but don't know what wrong with MCTD. Will look at it later, but I'm still trying to get your original download clip. (I hate Windows 7 on this damn Netbook).
    Last edited by sanlyn; 21st Mar 2014 at 16:54.
    Quote Quote  
  12. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    Still working with my puny Netbook. Many operations I run at home aren't feasible right now. Anti-flicker wasn't needed, as I saw no sign of luma hopping. The worst problem is high-compression artifacts from XVid, which are pretty nasty. I ran one version with NeatVideo (sorry, I have only the old v1.5 Demo version, as this Netbook won't run the newer v3), and another run with MCTemporalDenoise. The results are similar, witn NeatVideo and MCTD doing better on different types of noise: NeatVideo handled artifacts and motion noise better, MCTD handled grainy clumps better but had poor motion handling and wrinkled edges.

    I used this script to run MCTemporalDenoise:

    Code:
    DirectShowSource("C:\AllinYourName.m4v",fps=23.976) 
    # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
    # - Input is YV12 progressive, bottom field first.
    # - Apparently high base IRE for DV.
    # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
    AssumeBFF()
    ColorYUV(gain_y=-25,off_y=-2,cont_y=-10)
    MCTemporalDenoise(settings="very high")
    ConvertToRGB32(matrix="Rec601")
    To run NeatVideo in VirtualDub on Avisynth's RGB output, use the same script but omit the call to MCTemporalDenoise:

    Code:
    DirectShowSource("C:\AllinYourName.m4v",fps=23.976) 
    # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
    # - Input is YV12 progressive, bottom field first.
    # - Apparently high base IRE for DV.
    # - NeatVideo with low-quality sample run on output.
    # - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
    AssumeBFF()
    ColorYUV(gain_y=-25,off_y=-2,cont_y=-10)
    # # - MCTemporalDenoise(settings="very high") <- Comment to disable MCTD.
    ConvertToRGB32(matrix="Rec601")
    AssumeBFF() isn't really necessary. I use BFF or TFF as a just-in-case paranoia thing.

    I don't have MPG software on this Netbook, so I can't post an MPG until I return home. The two filters produced clean results, but they don't look "alike". NeatVideo cleaned artifacts and motion noise better, but this isn't a fair test for NeatVideo. The MPG download didn't have a suitable frame for noise sampling, I used a small patch from frame 142. The notes on the publisher's website about this video's low quality are no excuse for the bad compression artifacts seen here.

    Here are screen caps comparing the two filters. Ignore the NeatVideo logo on lower right, as I said this is an older demo-only version. The major difference is in motion noise and edge smoothing. NeatVideo looks cleaner in some respects, MCTD in others. Both filters require more tweaking. My past experience with noise like this is that, ideally, a combo of a lower-power MCTD followed by a lower-power NeatVideo (or some other temporal smoother) gets the best from both filters and yields better results.

    MCTemporalDenoise (frame 222). Edges show more remnants of artifacts, but interior detail looks sharper.
    Image
    [Attachment 10279 - Click to enlarge]


    NeatVideo (frame 222). Fewer artifact remnants, cleaner edges, but better noise sample needed to maintain detail.
    Image
    [Attachment 10281 - Click to enlarge]


    MCTemporalDenoise (frame 486). Problem cleaning lines/edges with camera motion, especially on the left.
    Image
    [Attachment 10282 - Click to enlarge]


    NeatVideo (frame 486). Cleaner edges and motion, but poor noise sample softens fine detail.
    Image
    [Attachment 10283 - Click to enlarge]
    Last edited by sanlyn; 21st Mar 2014 at 16:54.
    Quote Quote  
  13. Video Restorer lordsmurf's Avatar
    Join Date
    Jun 2003
    Location
    dFAQ.us/lordsmurf
    Search Comp PM
    Originally Posted by sanlyn View Post
    It would help much if we had some history about how this video was made.
    This most often happens because of poor transcoding settings on a long-GOP format. The I frames are good, while the P and B have degraded significantly into blocky crap. Not encoding, but transcoding. (Think DVD Shrink.)

    Originally Posted by mathmax View Post
    the only thing they say is that this video has been shot 2002 with Hi8 camera. And the quality of the video is due to the nature of the recording..
    That's hogwash on their part. They screwed up the digital conversion because somebody doesn't know what he/she is doing. I've never had a Hi8 video come out looking like crap like that. A Hi8 video should be gorgeous quality. (Having a high end camera or deck with TBC helps, too.)
    Quote Quote  
  14. sanlyn, thank you very much for your post I couldn't answer quickly because I wasn't at home these last days.

    I see very clearly the advantages of each filters.. and to tell the truth, NeatVideo was the first filter I tried before I created this thread, but I droped it because it softens fine details too much. I don't really know how to combine both filters, because NeatVideo requires a noise sample which can't be selected since the noise is already removed by MCTemporalDenoise(). And I don't really know either how to configure NeatVideo.. is the noise sampling only spatial or also temporal?

    Regarding your script, I have 2 questions:
    - why do you correct the luma before denoising and how do you choose the value for ColorYUV()? Why do you decrease the contrast? I also wonder if there is a way to inform MCTemporalDenoise or NeatVideo that the noise is bright, so it should only work on a certain luma range.
    - why do you convert to RGB ?

    lordsmurf, I noticed that the frames suffer more block artifacts every 25 frames.. then the frames are progressively less blocked until the next frame multiple of 25. Does that mean the GOP's size is 25 frames? So the I frames should be number 24, 49, 74, etc.. right? But even these frames have some blocks which is strange for reference frames, isn't it?
    Anyway, I wonder if the filters MCTemporalDenoise(), NeatVideo, and especially Deblock() can take into account the size of the GOP and work differently on the frames according to how much they are blocky.

    So, the poor transcoding explains the blocky crap.. but it doesn't explain the noise. I would be curious to know its origin...

    themaster1, I like your work on the sample you posted.. It looks a bit blurry, but at least the video is calm and pleasant to watch. Could you detail the script you used and the configuration for NeatVideo?

    Thank you all very much for your answers
    Quote Quote  
  15. Here is a rar with the neat profiles: http://www.megaupload.com/?d=K6Q5X9RB

    The script
    - deblock qued (default values (or was it msu deblock v2.2? i don't recall exactly)
    - neat (in vdub)
    > encoding in lossless with lagarith (vdub)

    - dftest (default values)
    - fft3 gpu (luma only with a sigma of 2 i believe)
    - gradfundbmod (for grain)


    You could do it all in avisynth and you could have better results imo with tnlmeans denoiser but i'ts very slow
    Quote Quote  
  16. Thank you

    Why did you decide to use dfttest() and fft3dGPU() on the top of NeatVideo. What is the purpose of each filter?

    Which settings do you use for gradfundbmod()?

    Instead of which filter would you use tnlmeans()? I don't mind so much if it's slow.. because my video is not so long. Why do you think it would be better?
    Quote Quote  
  17. Well the main reason i recall was that the neat profile was only 45%, i couldn't get better (the minimum advised in the official doc is 60%) so some substantial amount of noise if left. I don't like dfftest much though but has proven to be efficient on many occasions, here included. And fft3d (low sigmas) for an even cleaner result.

    gradfun:
    GradFun2DBmod(str=[<"Grain Quantity", 1.0, 10.0, 1.1>])
    I always use 1.1

    Anyways like i said, try tnlmeans + neat
    Quote Quote  
  18. Originally Posted by themaster1 View Post
    Well the main reason i recall was that the neat profile was only 45%, i couldn't get better (the minimum advised in the official doc is 60%) so some substantial amount of noise if left. I don't like dfftest much though but has proven to be efficient on many occasions, here included. And fft3d (low sigmas) for an even cleaner result.

    gradfun:
    GradFun2DBmod(str=[<"Grain Quantity", 1.0, 10.0, 1.1>])
    I always use 1.1

    Anyways like i said, try tnlmeans + neat
    thank you

    Why would you use tnlmeans()? I read the documentation but I don't know which parameters should I use.. and the defaults ones doesn't denoise well.
    Why don't you use MCTemporalDenoise()?
    Quote Quote  
  19. I have never used it actually but read the .pdf presentation i'm totally convinced it's the best algorithm to date. There is a gpu version (not 100% complete), i couldn't make it work so far but you might be luckier.

    Edit:
    link for the pdf:
    http://www.megaupload.com/?d=74E2QVDS
    Last edited by themaster1; 2nd Jan 2012 at 17:49.
    Quote Quote  
  20. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    Originally Posted by mathmax View Post
    sanlyn, thank you very much for your post I couldn't answer quickly because I wasn't at home these last days.

    I see very clearly the advantages of each filters.. and to tell the truth, NeatVideo was the first filter I tried before I created this thread, but I droped it because it softens fine details too much. I don't really know how to combine both filters, because NeatVideo requires a noise sample which can't be selected since the noise is already removed by MCTemporalDenoise(). And I don't really know either how to configure NeatVideo.. is the noise sampling only spatial or also temporal?
    Ideally I would run Avisynth procedures and NeatVideo in a single run, but I couldn't do that with my laptop. The NeatVideo I used was run by itself with no previous filtering. I'm not home yet, so will try these steps again later, but in combination.

    You are correct: if MCTemporalDenoise is run first and NeatVideo second, the noise would be sampled after MCTemporalDenose runs. As for configuring NeatVideo, they have an extensive manual describing how it's done. Most people don't look at the manual, so it's no surprise that NeatVideo doesn't work for them.

    NeatVideo is a spatio-temporal filter with some degree of motion compensation. Both of its filtering modes (temporal or spatial) can be adjusted and/or turned off separately.

    Originally Posted by mathmax View Post
    Regarding your script, I have 2 questions:
    - why do you correct the luma before denoising and how do you choose the value for ColorYUV()? Why do you decrease the contrast? I also wonder if there is a way to inform MCTemporalDenoise or NeatVideo that the noise is bright, so it should only work on a certain luma range.
    The corrections with ColorYUV were rather basic, mostly to recover dark and bright detail which are both outside the 16-235 range for RGB or DVD standards. The best way to "determine" the contrast and luma settings is to use the Avisynth's builtin-in Histogram(), then disable the Histogram for the final run. Decreasing contrast narrows the luma range. Normally I'd further tweak both color and contrast within specific ranges and color bands in RGB. No, you can't direct MCTD or NeatVideo to "adjust" brightness. Many denoisers don't work well with crushed blacks or burned-out brights, which is why those areas should be adjusted before denoising.

    Originally Posted by mathmax View Post
    - why do you convert to RGB ?
    NeatVideo expects RGB input (altough it works in its own YCbCr colorspace). Also, I usually fine-tune color, contrast, and gamma in RGB. There are many who maintain it can also be done with great precision in YUV, but I've never seen any evidence that isolating specific color or contrast ranges in yuv is possible. Try to lower a peak of Green values between 30 and 45 by 6 RGB points in YUV without altering Red and Blue, and you'll see what I mean.
    Last edited by sanlyn; 21st Mar 2014 at 16:55.
    Quote Quote  
  21. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    After several hours I tackled some of the worst problems with this video. What I was looking for is a point somewhere between noise reduction and detail retention. The major disturbance that's not yet under control is that godawful mess every time the camera moves. This video has not quite had the hell filtered out of it, but it's getting there.

    Anyway, the TEST2 download is the result of some basic ColorYUV, MCTemporalDenoise, NeatVideo, and VDub's temporal smoother. Direct download link to the first 2.5 minutes of filtered source (114-MB, MPEG2/AC3, 24fps):
    http://dc433.4shared.com/download/l87D_Lbq/M4vTest2_DVD_24fps.mpg

    Some frame captures show progress, if nothing else. Below, left: Frame 32 has very little camera motion, and Frame 228 (below right) is similar. Not images like "real" DV or DVD, but much cleaner -- as long as the camera doesn't move!
    Image
    [Attachment 10389 - Click to enlarge]



    Below, frames 474 (left) and 1137 (right). Frame 474 is a fast zoom-in with the background objects wiggling like wild salmon, while everything else fares OK. You'll see faces change shape a bit as the camera gets closer. The next few scenes look relatively normal. So, while detail is comparatively stable and grain and stuff are manageable, motion noise is what I'm after now.
    Image
    [Attachment 10390 - Click to enlarge]



    Shots like frame 2866 (left) play tolerably well, considering the source, and even people motion looks OK. But as soon as the camera moves or zooms in (frame 3125, right), Michael and the music stands are OK, but everything in the background turns to simmering mush.
    Image
    [Attachment 10391 - Click to enlarge]


    A .zip file of the six 640x480 frame images is attached below.

    I ran other plugins, but it was soon apparent that camera motion is the big bugaboo, and IMHO it should be fixed before anything goes to RGB or gets further treatment. Using more smoothers is out of the question; they don't fix much of anything past this point, and they make people look as if they're wearing stockings over their heads. After about 4 hours of searching Doom9's endless threads about mvTools (and managing to stay awake most of the time), it seems some form of compensation/frame rebuilding should be possible. I haven't the slightest idea how to do that using mCompensate, and the documentation is somewhat obscure (to me, anyway). As for more denoisers and degrainers, they're a waste of time. If anyone has any idea on how mvTools could help calm some of those wiggles, we'd all appreciate hearing about it. Meanwhile, I'll keep looking.
    Last edited by sanlyn; 21st Mar 2014 at 16:55.
    Quote Quote  
  22. Originally Posted by sanlyn View Post
    After several hours I tackled some of the worst problems with this video. What I was looking for is a point somewhere between noise reduction and detail retention. The major disturbance that's not yet under control is that godawful mess every time the camera moves. This video has not quite had the hell filtered out of it, but it's getting there.

    Anyway, the TEST2 download is the result of some basic ColorYUV, MCTemporalDenoise, NeatVideo, and VDub's temporal smoother. Direct download link to the first 2.5 minutes of filtered source (114-MB, MPEG2/AC3, 24fps, 2min 32secs):
    http://dc433.4shared.com/download/l87D_Lbq/M4vTest2_DVD_24fps.mpg

    Some frame captures show progress, if nothing else. Below, left: Frame 32 has very little camera motion, and Frame 228 (below right) is similar. Not images like "real" DV or DVD, but much cleaner -- as long as the camera doesn't move!
    Image
    [Attachment 10389 - Click to enlarge]



    Below, frames 474 (left) and 1137 (right). Frame 474 is a fast zoom-in with the background objects wiggling like wild salmon, while everything else fares OK. You'll see faces change shape a bit as the camera gets closer. The next few scenes look relatively normal. So, while detail is comparatively stable and grain and stuff are manageable, motion noise is what I'm after now.
    Image
    [Attachment 10390 - Click to enlarge]



    Shots like frame 2866 (left) play tolerably well, considering the source, and even people motion looks OK. But as soon as the camera moves or zooms in (frame 3125, right), Michael and the music stands are OK, but everything in the background turns to simmering mush.
    Image
    [Attachment 10391 - Click to enlarge]


    A .zip file of the six 640x480 frame images is attached below.

    I ran other plugins, but it was soon apparent that camera motion is the big bugaboo. IMHO it should be fixed before any further treatment. Using more smoothers is out of the question; they don't fix much of anything past this point, and they make people look as if they've pulled stockings over their heads. After about 4 hours of searching Doom9's endless thread about mvTools (and managing to stay awake most of the time), it seems some form of compensation/object rebuilding should be possible. I haven't the slightest idea how to do that using mCompensate, and the documentation is somewhat obscure (to me, anyway). As for more denoisers and degrainers, they're a waste of time. If anyone has any idea on how mvTools could help calm some of those wiggles, we'd all appreciate hearing about it. Meanwhile, I'll keep looking.
    I just see now your message as I didn't have access to internet this last month.. I'm so sorry for my late answer.

    First of all, thank you so much for the time you spent on this video.. I watched your last sample and it really looks amazing !! Very calm and well denoised. The only problem is this kind of comb effect all along the video. I wonder where it comes from..

    Could you detail the script you used (if you still have it)? I'm very interested on trying it.

    I figured out that I could quite well get rid of the ugly grain on motion areas with a simple removegrain(3):

    mymotionmask = mt_motion( thT=255, thY1=50, thY2=50, thC1=50, thC2=50).mt_expand().mt_convolution("1 1 1 1 1", "1 1 1 1 1").mt_convolution("1 1 1 1 1", "1 1 1 1 1")

    clip1 = last
    clip2 = removegrain(3)

    overlay(clip1, clip2, mask=mymotionmask)
    Quote Quote  
  23. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    Hi, mathmax. I'll retrieve the external hard drive that has the earlier work and intended to do that anyway soon -- now trying to rebuild one of my PC's to handle HD and BluRay (well, it had to happen sooner or later). I've spent all month doing this, with one interruption after another and a lot of PC repair customers who decided to call all at the same time. Meanwhile, weekends are chore time around this household, when I get very little video work done. I'll dig into the hard drive later today or tomorrow A.M., I'm leaving in a few minutes to finish a PC repair job for a customer.

    Yes, one could work more on the fine grain, but I think removing all grain and every microdot from video makes it look rather plastic and unreal. But everyone has their preference about that. My main concern was looking at what I call "simmering blobs" and motion artifacts every time something moves. There are many ways to smooth grain, but I'd like to find a way to add more stability on movement. I'm certain other members here who know more about motion compensation techniques can help. Meanwhile, I'll be back to this shortly.
    Last edited by sanlyn; 21st Mar 2014 at 16:56.
    Quote Quote  
  24. Originally Posted by sanlyn View Post
    Hi, mathmax. I'll retrieve the external hard drive that has the earlier work and intended to do that anyway soon -- now trying to rebuild one of my PC's to handle HD and BluRay (well, it had to happen sooner or later). I've spent all month doing this, with one interruption after another and a lot of PC repair customers who decided to call all at the same time. Meanwhile, weekends are chore time around this household, when I get very little video work done. I'll dig into the hard drive later today or tomorrow A.M., I'm leaving in a few minutes to finish a PC repair job for a customer.

    Yes, one could work more on the fine grain, but I think removing all grain and every microdot from video makes it look rather plastic and unreal. But everyone has their preference about that. My main concern was looking at what I call "simmering blobs" and motion artifacts every time something moves. There are many ways to smooth grain, but I'd like to find a way to add more stability on movement. I'm certain other members here who know more about motion compensation techniques can help. Meanwhile, I'll be back to this shortly.
    ah ok I look forward to see what you can achieve on motion compensation..

    Meanwhile, I've tried to apply MCTemporalDenoise() and then NeatVideo but it looks too smooth... (not as good as your sample).. I'm trying to find your settings again.. but it's a bit laborious. If you still have the settings you used, it would save me a lot of time..

    Thank you very much for your help
    Quote Quote  
  25. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    Wish I could have gone farther along with the original clip, mathmax, but this has been a frustrating 2 months. Anyway, I change the script from the original posted earlier. There are two additional steps after the script (well, 3 really, but you might not have TMPGenc Plus 2.5. Will explain that later, below).

    Step 1: I ran this script in avisynth. It is really slow (45 minutes to get the first 3647 frames). Not unsual for MCTD at highest setting, but gets even slower if you fiddle with individual settings -- which I did, but with no advantage that I could see. A copy of the avs is attached to this post as well.

    Code:
    DirectShowSource("G:\forum\AllIn\AllinYourName.m4v",fps=23.976)
    ColorYUV(gain_y=-30,off_y=-4,cont_y=-10)
    in1=last
    e=in1.AssumeBFF().SeparateFields().SelectEven()
    o=in1.AssumeBFF().SeparateFields().SelectOdd()
    
    eMC=e.MCTemporalDenoise(settings="very high")
    oMC=o.MCTemporalDenoise(settings="very high")
    new_clip=interleave(eMC, oMC).weave().ConvertToRGB32(matrix="Rec601",interlaced=true)
    return new_clip
    I'm bound to get comments about SeparateFields(). Yes, this is non-interlaced YV12 (allegedly, anyway). I find that separating fields with YV12 sometimes helps some filters in rare cases. My guess is this: at some point in the criminally clumsy pre-processing of this video, field order or field dominance got shifted in YV12 (whether it was interlaced back then or not is anyone's guess), and YV12 chroma got screwed up. That's the least of the problems, but that's my minimally-educated guess. Anyway, if anyone balks at the use of using SeparateFields, let them explain this:

    Same script, without SeparateFields():
    Image
    [Attachment 11146 - Click to enlarge]



    Same script, using SeparateFields():
    Image
    [Attachment 11147 - Click to enlarge]


    Try it yourself for a few frames and see the difference. I'd be very interested in seeing an explanation. Seriously.

    Step2: I didn't spend a lifetime getting NeatVideo settings, as they'll undoubtedly change in future efforts depending on what I learn to do with Avisynth in this clip. I used NV version 3. The settings files should work in earlier versions. A .zip file of the dnp (device noise profile) I used and the nfp (noise filter profile) IS ATTACHED, below. In order to see the nfp settings, you have view NeatVideo in Advanced Mode. These settings are stronger than those I'd use on cleaner video.

    On opening the NeatVideo dialogue, check these initial settings:

    Image
    [Attachment 11148 - Click to enlarge]


    Then click "Configure. . ." and load the dnp and nfp files. Run NeatVideo by itself. Don't use other filters in VirtualDub when running NeatVideo. Other filters make it impossible to judge just what NeatVideo itself is doing.

    Step 3 (Optional): I made a separate run with VirtualDub's temporal smoother set at 3. It's up to you. If you really want to be daring, load temporal smoother together with NeatVideo in Step 2. Chain temp smoother before NeatVideo. Yes, they do affect each other. Temp smoother will make the whole thing softer used in that way. I do that only when I'm really in a desperate hurry, but I recommend against it. In that respect, I'm untrue to the Greenpeace mission.

    Step 4: I made some slight but visible additional level, gamma, and contrast corrections with the excellent color filters in TMPGenc Plus. You could probably accomplish the same thing with something like ColorMill. The trouble with trying that in YUV is that ColorYUV has no effect on chroma gamma, only luma. Individual color gamma is an "RGB thing" AFIK.

    Somewhere in your original video is a frame with a larger, more neutral area that would give NeatVideo a better noise sample. This one came from frame 142. There are many other ways to improve this video, I just didn't have time. But I'll be looking again.
    Last edited by sanlyn; 21st Mar 2014 at 17:24.
    Quote Quote  
  26. Thank you very much for your detailed answer. I'm really impressed by the clarity of your post and very grateful for your efforts on this video.

    Here it's 5:00 AM and I really need to go to bed now.. , but I look forward to follow the steps of your tutorial and to test carefully your settings. I'll be on it tomorrow from the first hour

    Thank you again

    Quote Quote  
  27. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    You're welcome. I'm eager to see if other readers come up with more answers, but I'm off to sleep myself .
    Last edited by sanlyn; 21st Mar 2014 at 16:56.
    Quote Quote  
  28. So.. I finally followed your steps and here is my feedback

    step 1

    - first, I'm a bit surprised the way you corrected the luma range. I already asked about that earlier in the thread, but I still don't get exactly how you chose the values gain_y, off_y and cont_y. I'd rather edit the levels out of fear to distort the scale. Moreover, I feel that you decreased a bit too much the upper limit. But I might be wrong about that... I admit I don't know well the reason why you chose this way. I uploaded below samples for each methods. Please don't hesitate to examine them.

    ColorYUV(gain_y=-30,off_y=-4,cont_y=-10)
    Click image for larger version

Name:	coloryuv0000.jpg
Views:	263
Size:	275.2 KB
ID:	11165

    smoothlevels(0, 1, 255, 0, 235, limiter=2)

    Click image for larger version

Name:	levels0000.jpg
Views:	186
Size:	292.8 KB
ID:	11166

    - I'm also surprised of the difference with and without SeparateFields(). As you advised, I checked several frames carefully and I could notice that the noise is more reduced and the edges are cleaner when working on separate fields. However, there is a comb artifact, which I think might come from frame averaging in MCTemporalDenoise(). So I applied QTGMC() to get rid of this artifact and the result amazes me. Please compare the following frames by switching in separate tabs:

    MCTemporalDenoise() on progressive frames:
    Click image for larger version

Name:	progressive0000.jpg
Views:	872
Size:	191.5 KB
ID:	11169

    MCTemporalDenoise() on separate fields:
    Click image for larger version

Name:	interleave0000.jpg
Views:	712
Size:	191.6 KB
ID:	11167

    MCTemporalDenoise() on separate fields + QTGMC():
    Click image for larger version

Name:	qtgmc0000.jpg
Views:	885
Size:	182.1 KB
ID:	11168

    The last ones really looks clean.. I even wonder if that is needed to apply neatvideo on the top of it..

    Here is the script I used :
    loadCplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll")
    FFMpegSource2("D:\G\MJ\projets videos\all in your name\AllinYourName.m4v")

    smoothlevels(0, 1, 255, 0, 235, limiter=2)

    in1=last.AssumeTFF().SeparateFields()
    e=in1.SelectEven()
    o=in1.SelectOdd()

    eMC=e.denoise()
    oMC=o.denoise()
    interleave(eMC, oMC).weave()
    QTGMC()
    SelectEven()
    ConvertToRGB32(matrix="Rec601")

    function denoise(clip)
    {
    return clip.MCTemporalDenoise(settings="very high")
    }



    step 2

    I imported your settings as you advised. I can see a difference in noise reduction.. but not that much... And also the image looks more sharp and contrasted. I'm not sure that neatvideo contributes so much to calm down the video.. but I didn't compare in motion. I thought there might be other ways to smooth and sharpen but maybe there is a real plus in neatvideo that I don't really get now...


    Step 3

    With the temporal smoother, I guess you're trying to fix the image distortion when the camera moves. For example the fast zoom at frame 474 that you mentioned before. Am I right?


    Step 4


    I don't have TMPGenc Plus. Is it an external tool? Could you detail your settings for fixing colors and contrast?
    I don't really understand what you mean by "ColorYUV has no effect on chroma gamma". Would be nice if I could do a quick test to realize the issue...

    As I final step, I like to add grain with grainfactory()... it sounds a bit stupid to add grain after trying so hard to get rid of it. But there is good and bad grain.. and the last one gives depth to the video..

    Thank you again for your help. I look forward to read your comments and explanations

    max
    Last edited by mathmax; 26th Feb 2012 at 13:47.
    Quote Quote  
  29. Banned
    Join Date
    Oct 2004
    Location
    New York, US
    Search Comp PM
    Nice going. I was just running QTGMC a while ago, but haven't looked at all the results.

    I'll get to some of your questions in such a while. I'm moving 1/2 of a drive full of videos and scripts to another PC. To give a quick answer to one of your questions, TMPGEnc Plus is an MPEG encoder, an old standby for years: http://tmpgenc.pegasys-inc.com/en/product/tp.html
    The publishers tried removing it from their product list a while back and were besieged with complaints from hobbyists and pros alike worldwide. So it's still on the books. Most users feel TMPGenc's current, spiffier encoders aren't as good. But everyone has an opinion. . . . Back in a short while.
    Last edited by sanlyn; 21st Mar 2014 at 16:56.
    Quote Quote  



Similar Threads