Le Mans 24 Hour 1080i HDTV Sample. It's only the first 13 secs that you need to look at. You can see slight color banding in this video source.
F1 Old Race Sample 1080i HDTV (SD upscaled): I need a temporal smoother to remove the flicker or whatever you want to call it. TTempSmoothF works but it blurs too much which is most clear to see if you were to use it on the soap sample below - look at the woman's face for example.
Soap 1080i HDTV Sample . This is 9 Mbps (exactly as broadcast as I copied it from my satellite box) which I thought was good quality for static scenes like what you get in soaps yet the picture quality doesn't look like good HD quality and there's lots of junk noise in the walls. McTemporalDenoise & QTGMC can't remove the junk in the walls. If you look at 43 secs you can get the longest view of the junk on the wall.
What causes the junk on the wall to appear as the clip is 9 Mbps which I thought was plenty bitrate for a static scene like that?
I'm trying to fit the 24 hours of Le Mans (21 hours after commercials removed) on a single Dual Layer Bluray disc. I did a CRF 18 720p encode with TTempSmoothF and it came out at 44 GB (2.57GB to spare) 4.59 Mbps (not including audio) as reported by MediaInfo. The test sample after encoding with the same settings comes out at 3.6 Mbps.
The sample after encoding to 720p with TTempSmooth looks OK (similar to the source) however the color banding is made worse and there's now a lot pixelated junk on the road. TTempSmooth blurs too much. DegrainMedian looks better with less issues.
These are my settings for DegrainMedian and TTempSmoothF (obviously I don't use both filters at the same time!):
Can anyone suggest better settings for TTempSmooth that still removes flicker, etc but which doesn't blur too much. Easiest way to see is in the soap sample in the women's face. I know there's no flicker in these samples but there is in the F1 sample.Code:TTempSmoothF(maxr=3, lthresh=8, cthresh=5, strength=4, interlaced=true) DeGrainMedian(limitY=2, limitUV=3, mode=1, interlaced=true)
I record so much stuff that's it's not possible for me to analyze each video and see what particular script I need for that video. So ideally I would like a simple script that will work with any 1080i HDTV video to prevent or minimize color banding and junk on the road/walls in the Le Mans/soap clip.
PLEASE NOTE: I can NOT use QTGMC as it's not stable on my machine and even if it was, it's too slow for HD footage. I'm using MT Avisynth with an 8 core 4.2 Ghz CPU. I also do NOT want to add grain to the picture even if it does cover up banding because it hurts compression and I don't like grain!
+ Reply to Thread
Results 1 to 30 of 39
Last edited by VideoFanatic; 7th Jul 2014 at 08:01.
If there wasn't banding in the source along gradients - you 're going to introduce banding by denoising , and encoding at lower bitrates . The noise in the original source sort of "covers" it up
You need to apply dithering filters to reduce the banding , which of course acts like noise and requires larger filesizes to encode
So you're going to need to choose among some tradeoffs
handbrake uses hqdn3d from mplayer. It's available everywhere, even ffmpeg, avisynth
No the source doesn't have banding. I've also used Avisynth denoisers before such as DegrainMedian & McTemperalDenoise and I've never got banding before (I aim to get the picture transparent to the source) so I don't think denoising always causes banding - it just depends on the denoiser.
I used Handbrake without any filters and got banding so I thought its' denoiser: HQDN3D wouldn't cause banding. Unfortunately I tried HQDN3D in Avisynth and it DOES causes banding.
Are there any good encoding programs that have strong denoisers that don't cause color banding?
Because I'm trying to fit for example the 24 hours of Le Man on 1 Dual-Layered Bluray disc. hqdn3d does that but it has color banding. ****Edit*** Not a problem for a UFC match for example as I can't see banding but it's a major problem for other stuff such as motorsport where you get gradients - hqdn3d causes banding there.
Last edited by VideoFanatic; 25th Jun 2014 at 14:29.
Your road sample has visible low-bitrate artifacts, including macroblocks. The bitrate is too low for fast action 1920x1080 sports. It's even too low for 1280x720 motion that fast. Degrainers make it worse and cause banding.
So do you have a denoiser suggestion or did you just want to criticize the quality of my source video!?
I've just noticed that the source had slight banding to begin with. I used DegrainMedian and it didn't make the banding worse so it looks like only certain denoisers cause banding or make it worse. 720p looks perfectly acceptable to me at a transparent bitrate.
The video I provided was the quality it was broadcast in. Perfect quality HDTV video doesn't exist. I don't know about the USA but in the UK the technical max bitrate for HDTV is 20 Mbps however I find the bitrate actually used is between 9-14 Mbps for 1080i sports. All programmes broadcast have visible compression artifacts as you would expect when the broadcaster re-encodes a 50Mbps source into 9-14 Mbps. Some shows have it worse than others.
So the answer is no. And my response was hardly violent.
First, 8 bits of red, green, and blue are insufficient to create band free gradients. Here's a noiseless example:
That image changes from RGB=80,80,80 at the top to 80.71,80 at the bottom. Only the green content changes and it changes in 10 steps from top to bottom. You can probably see all 10 bands on your monitor.
8 bits of rec.601 or rec.709 YUV shows even stronger banding because you can't even use the full 0-255 range of YUV values, and the conversion of RGB/YUV conversion is lossy. The above RGB image converted to YV12 and back to RGB in AviSynth with ConvertToYV12() and ConvertToRGB():
If you add noise to the image (I used AddGrain(5) on the RGB image then converted to YV12 and back to RGB) the banding is much less obvious:
So, if you remove all noise from a video you will get visible banding in gradients.
You can use a filter like GradFun2DBMod() to add noise back into the picture to reduce the gradients. But that makes the video less compressible again. It does have an option to add noise that doesn't change from frame to frame. That helps with the compressibility of still shots but not with high action video.
Last edited by jagabo; 26th Jun 2014 at 08:36.
Yep, I request that you let our previous discussion pass as what they call an error in communication.
Jagabo posted a good demo.
I used Avisynth's dither plugin for the following procedure. Cleans about 75% of those hard contours without losing too much detail. I've modded several 1080i vids in a similar way. For more edge softening in flat areas, increase GradFun3's "thr" parameter to maybe 0.8 or 1.0. And add more fine grain. I didn't use a sharpener. If you want one, use it last.
(open the source in Avisynth using whatever you want to get lossless AVI) ColorMatrix(mode="Rec.709->Rec.601",interlaced=true) AssumeTFF().QTGMC(preset="very fast") GradFun3(thr=0.5, mask=0) Dither_convert_8_to_16() Dither_Resize16(1280,720,kernel="Spline64") DitherPost () AddGrainC(1.5,1.5) ColorMatrix(mode="Rec.601->Rec.709")
P.S.: Forgot to mention (well, the question is about encoders, right?)...I ran two. The mkv ran in TMPGenc TVMW5. Made an exact copy with TX264. Same basic settings, similar x264 engines, same results. No banding from either.
The proposed 24-hr BD-DL disc would take, what, a ~4300 kbps bitrate? Expect banding and worse. Would that bitrate be BD-legal ? ?
Last edited by LMotlow; 26th Jun 2014 at 19:15.-ann's brother
Thanks. I've updated my 1st post. Tried the script you gave me but I couldn't get it working. The 720p encoding I need would be about 4.5 Mbps. Your sample looks good but that's because you encoded to 14 Mbps! I need a bitrate of around 4.5 Mbps. I realize I can't fix everything but I'd still like to clean it up a bit if possible.
Yes it's bitrate legal as there's no minimum bitrate required to define what is HD - only the resolution.
The sample source from post #1 has frame indexing and audio sync errors. There's a brief audio dropout in the linked demo, due to those problems
Jagabo thanks for the demo, I see what you're talking about now.
Last edited by VideoFanatic; 6th Jul 2014 at 14:59.
Indexing and sync eventually worked ok after I finally figured out that DirectShowSource with some very specific settings in ffdshow were the only way your sample could be accessed correctly.
Miscommunication about the source.
I accessed the clip fine with DGDecodeNV as it's frame accurate.
Any idea how to get your script working? Did I install it correctly?
I notice you use Spline64 instead of Lancoz4Resize to downscale to 720p. Is there any difference in picture quality between those two?
If you're using a 32 bit Avisynth, then from the downloaded zip file, the version of dither.dll and avstp.dll in the win32 folder both need to be copied to your Aviynth plugins folder (if you want them to auto-load). Nothing needs to be copied to the Windows\System32 folder from the dither zip file (at least not the current one).
I assume dither.dll in the win64 folder is for a 64bit version of Avisynth, or maybe if your running a 64 bit version of Windows (I don't run either) but it'd go in the Avisynth plugins folder too..... I assume.
Do you have the other required plugins also? http://forum.doom9.org/showthread.php?p=1386559
The dfttest zip file contains libfftw3f-3.dll, which needs to go into the Windows/System32 folder, at least when running XP. I'm not sure about other Windows versions.
Others here are no doubt more expert than I when it comes to dither, but for smoothing existing banding, does this at least work (if everything's installed correctly)?
Shouldn't Granfun3 ideally be the last filter in the process (it's not in a script posted earlier), otherwise you risk an 8 bit filter which might follow causing banding again.
I haven't read the thread thoroughly, but here's another denoising suggestion. If you're already using QTGMC for de-interlacing it should be easy to try.
QTGMC(InputType=1, Preset="medium", Ezdenoise=2)
The above is QTGMC in progressive mode (single threaded Avisynth) the way I often use it for HD video denoising followed by gradfun3(). It's slow. Adjust the EzDenoise amount up and down a little if need be (between 1 and 2 is usually ample). For light noise you can try removing QTGMC's denoising completely. It'll probably leave a little behind if you do, but it tends to be more stable, fairly inoffensive, and helps prevent banding. If EzDenoise is not used, you probably won't need gradfun3().
If you're not in a hurry you can remove "Preset=medium", I think the default is "slower". It'll stabilise any remaining noise more, or removes more of it if EzDenoise is used, but sometimes I find a faster speed preset looks more natural because any remaining noise won't be as "static" (it'll behave more like noise does). At least when it comes to HD video.
Just some ideas.....
Not wanting to sidetrack the thread too much, but I'm actually re-encoding a fairly noisy video at the moment using QTGMC for denoising. It's hard to demonstrate how noisy video is or isn't with still pics, but here's an indication.
QTGMC will also help fix "shimmer" and other de-interlacing badness when run on progressive video which has previously been de-interlaced, if that's the sort of thing you were referring to in your original post (I haven't downloaded your samples).
Direct from Avisynth (if you open each in a browser tab and switch between them the denoising should be obvious):
QTGMC(InputType=1, Preset="medium", Ezdenoise=2):
Last edited by hello_hello; 7th Jul 2014 at 07:02.
All my stuff is 1080i HDTV recordings I made myself. They don't have the issues you mentioned. I can't use QTGMC as it's too slow. QTGMC also doesn't fix the issues I mentioned.
DGDecodeNV generates a ".ts" file. I thought it produced a project file, not a fully muxed .ts. Maybe someone familiar with that NV version knows better.
AFAIK the SmoothGrad function is in one of dither's dll's. I've never had to use it. The input to the function is supposed to be converted to 16-bit first. In any case, some of dither's support dll's interfere with some "regular" Avisynth plugins, so I put all dither components in their own plugins folder. I created another plugins folder inside the Avisynth folder and called it "Dither". You can name that folder whatever you want. Then import or load them explicitly in the script:
Import(path to dither plugins\"Dither.avsi") Import(path to dither plugins\"mt_xxpand_multi.avsi") LoadPlugin(path to dither plugins\"avstp.dll") LoadPlugin(path to dither plugins\"dither.dll") LoadPlugin(path to dither plugins\"dfttest.dll") LoadPlugin(path to dither plugins\"mvtools2.dll") LoadPlugin(path to dither plugins\"mt_masktools-26.dll")
You can denoise all you want, but more denoising means more ordered re-noising afterwards. All you'll do is remove stuff and then put it right back again. Those rough edged and some hard edged gradients aren't just in the first 13 seconds as mentioned. They're all over the place.
I guess I have to refer again to poisondeathray's notes in post #2. You have to decide between all that denoise/re-noise with a suitable bitrate for encoding fast motion HD, or denoise the heck out of everything and use an unsuitably low bitrate around 4500 kbps -- which means you're just undoing all your work. A low bitrate is what caused problems in the first place.
I usually avoid Lancoz resizers because of oversharpening artifacts.-ann's brother
Sorry, I didn't see you'd already mentioned QTGMC and I still haven't had a play with your samples. Out of time today.
Did you get Dither working?
Yes thanks for the help, I got dither working on my Le Mans sample but it doesn't seem to have an effect when encoding to lower file sizes such as 4.5 Mbps. The Dither script looks worse than when just using DegrainMedian & Yadif. The dither script gave me pixilation in the banding at around the 7 sec mark which I didn't get when using DegrainMedian & Yadif. I also tried NOT using any denoisers - all I used was Yadif to convert to 720p and I still got the pixellation in the banding which was weird as DegrainMedian and Yadif didn't give pixelation!
I'm not expecting miracles but can anyone give me a script that will clean up the crap on the road or the banding a bit when encoding the Le Mans clip to CRF 18?
Used the same script as before. I cut out a couple of sequences with the encoder. 4500 kbps VBR. A little noise, but nothing really horrible that I can see. Play with GradFun3 "thr" parameter if you want more smoothing. BTW, the maximum GOP size for BD compliance is 2-second GOP's (100 frames PAL).
DirectShowSource(path to ts file\"Qual 2 Day Clip Orig.ts") ColorMatrix(mode="Rec.709->Rec.601",interlaced=true) AssumeTFF().QTGMC(preset="very fast", border=true) GradFun3(thr=0.5, mask=0) Dither_convert_8_to_16() Dither_Resize16(1280,720,kernel="Spline64") DitherPost () AddGrainC(1.5,1.5) ColorMatrix(mode="Rec.601->Rec.709")-ann's brother
It looks like the only thing you changed was "border=true"? Tried the script but it kept crashing. I can't use QTGMC anyway as it's too slow. Do you have an alternative script please?
"Crash" covers a lot of territory.
Runs slow for me, but didn't crash. I do have to reboot now and then to clear RAM for this kind of heavy HD cleanup. I also tried this:
ColorMatrix(mode="Rec.709->Rec.601",interlaced=true) Yadif(order=1,mode=1) GradFun3(thr=0.7, mask=0) Dither_convert_8_to_16() Dither_Resize16(1280,720,kernel="Spline64") DitherPost () AddGrainC(2.0,2.0) ColorMatrix(mode="Rec.601->Rec.709")-ann's brother
Did you test the above script as I had to add this line to make it work?:
Load_Stdcall_plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
The script looked worse again than if I had just used DegrainMedian and Yadif and I can't see any grain yet there was in your previous script.
It's just weird that QTGMC is the only thing that makes the script work when encoding to CRF 18! The QTGMC script did work so surely I can still clean the video up a bit with something else besides QTGMC?
Well....jagabo or poisondeathray or somebody might technically explain this better than I do, but --
Large flat areas and gradients require more data bits to define fine gradations and subtle changes in hue. If you remove the fine data that defines subtle or soft edges, you get hard edges (banding). GradFun3, GradFun2dbMod, AddGrainC, etc., add a specific kind of grain (i.e., noise) that isn't just junk. In effect, they "add data" that helps define gradients. Yes, the added data does come off looking like grain a bit, but it's not just garbage. The reason you do that is because low-bitrate re-encodes don't use enough data bits to define all the pixels needed to avoid banding and choppy edges. Remove that data with strong degrainers and then re-encode with even fewer data bits, the problem gets worse. It might look "better" after degraining -- I don't think it does, really, it just has less visible grain -- but then the lower bitrate encode repeats the original problem. On top of that, very low bitrate will get macroblocks where they didn't exist before.
I seem to be repeating what jagabo said in post #10. If you want the low bitrate you propose, something has to give somewhere.
QTGMC at very fast pre-set does just enough cleanup to prevent obvious banding and aliasing IMO. Slower presets use fft3dfilter, the slowest ones use dfttest. You can use the dither version of dfttest after the resize. It does some dithered smoothing that non-dithered dfttest doesn't. But use a low sigma like 2 or 3, or you'll wipe out that edge data again. Be sure to load the dither version of dfttest.dll.
Loadplugin("path to dither plugins\dfttest.dll"). It's slower than regular dfftest, but not too bad.
then after the resize:
You can speed up QTGMC with this: AssumeTFF().QTGMC(preset="ultra fast") . It uses yadif but has better motion interpolation
You can sharpen lightly if you want with something like LimitedSharpenFaster at low power like 25 or 50 before adding any more grain with AddGrainC. But any sharpening eats up more bitrate. If you use higher bitrates you'd get less time on that BD-DL, but many of your problems would go away. Maybe two DL's instead of one ? ? ?
Last edited by LMotlow; 7th Jul 2014 at 22:53.-ann's brother