How to Remove Dancing Pixels with Avisynth when Denoisers don't work?

[VideoFanatic]

Thanks for your help. So it seems that the filters will give better quality if I convert to progressive however Bluray doesn't support 59.94fps in 720 x 480. Could I just use 1280 x 720 resolution instead but add black borders so the actual picture stays at 720 x 480? So obviously black bars would be added to the side but would black bars need to be added to the top and bottom?

I'm unsure because I just found a video that the WWE re-released on Classics on Demand. The clip was from several years ago and was originally 720 x 480 (4:3). The classics on demand clip is still 4:3 but it looks like they've added side bars to make up the 1280 x 720 resolution. But if the vertial resolution should be 720 pixels and the original clip was 480 pixels then how did they make the picture fill the height of the screen in the new clip? Did they upscale? Here's the 1280 x 720 MP4 Classics on Demand clip. I would say it's still standard definition quality.

I don't really want to upscale the visible part of the picture as I know that it lowers the quality of the picture. Is there any way to do it like the MP4 clip I gave you?

[sanlyn]

Thanks for your help. So it seems that the filters will give better quality if I convert to progressive

The test clip wasn't converted to progressive. It runs at 29fps. It was processed as if it was progresssive, because in fact I think that at some earlier time the video was "de-interlaced" using some form of deinterlace/blend. The reason it looks weird if you deinterlace or use SeparateFields and rejoin is because 50% of the original field information is missing. I've encountered that problem earlier, with other videos. That's another way of saying that some genius found a way to permanently screw up that clip.

Commercially, 4:3 video is usually encoded as 4:3 for playback. Don't confuse the encoded frame storage size with the playback aspect ratio. CinemaScope, too, is encoded in a 720x480 container. So are several other film and broadcast playback sizes. The 720x480 is the container size; the playback sizes are all different. Your players, monitors, and TV know how to handle it.

[poisondeathray]

That latest clip is 1280x720p29.97 (not 59.94)

To upscale to 720p, you would pillar box it to keep the AR

960x720 is 4:3 for square pixels, with 160px borders on each of the left and right

160 + 960 + 160 = 1280



You can upscale with nnedi3 but it's slow. The quality is better than standard upscalers in terms of less aliasing. It not that noticable on live action footage. For something with clean lines like cartoons it is more noticable

Instead of using black bars, they used a logo or some graphic - you can do that if you want

[VideoFanatic]

I made this post before poisondeathray!

OK but any idea how the 1280 x 720 Classics on Demand clip took a 4:3 picture and made it 1280 x 720? It looks like they added some sidebars but I don't know how they made the height of the video still fill the height of your screen without adding black bars. Any ideas what they did?

How could I do the same thing with my 720 x 480 clip? I want it in 1280 x 720 so I can have it in progressive.

[poisondeathray]

e.g.

## starting with 720x480p59.94 source already bobbed & filtered

nnedi3_rpow2(qual=2,nsize=0,rfactor=2,cshift="spline36resize", fwidth=960, fheight=720) 

#sharpening e.g. lsfmod etc....

addborders(160,0,160,0)

You normally sharpen a bit after upscaling, I'll leave it up to your tastes for the settings

If it's too slow, you can substitute the 2nd line with something faster, just a plain spline36resize(960,720)

If you want some graphic or logo instead of black borders, you need to replace the addborders line

[poisondeathray]

I mentioned this in your other thread but I still wouldn't bother. I would use SD blu-ray for this project

Unless your equipment has a very bad deinterlacer and very bad upscaler, then it's just a waste of bitrate and processing time

[VideoFanatic]

It's just that I'm starting to get a bit paranoid now about interlacing artefacts. I think you guys said that you couldn't see any interlacing artefacts on this MPEG2 https://dl.dropbox.com/u/57941257/Raw%201996%20March%2011%20Still.mpg when watching it on your TV. Here's a DVD ISO of it as well https://dl.dropbox.com/u/57941257/DVD%20Raw%201996%20March%2011%20Still.iso

However I took a Bluray of that clip into a professional TV shop and played it on some expensive Bluray players and the interlacing artefacts were very clear to see to myself and the shop assistants. So you guys can't see interlacing but I can which I find weird. However I burned it to a DVD and played it on a small TV and I can't see any interlacing artefacts so I'm beginning to think this is a problem with Bluray players themselves. So I'm not sure if I should convert to progressive or not.

You're saying that upscaling isn't a good idea but I'm just wondering why the WWE did it with their Classics on Demand clip? They kept it in standard definition quality (albeit 1280 x 720) so I don't know why they would upscale the video?

[poisondeathray]

Well, like most things there are pros and cons

If you process with QTGMC, and upscale with NNEDI3 , then you can be fairly sure the aliasing and line twittering will be gone on every display. But the downside is 720p59.94 will take a more bitrate for a certain level of "quality", so you have less room on a disc or shorter time. Also the processing and encoding will be slower and you seem to be always complaining about how slow things are...

Why don't you run some tests and weigh the pros/cons on your equipment? There is a difference, some do very poor upscaling and deinterlacing , some to extra post processing. One of my "cheap" LG LCD's does very poor bob deinterlacing, and you do see deinterlacing artifacts (By the way, they are deinterlacing artifacts, not interlacing artfacts)

They kept it in standard definition quality (albeit 1280 x 720) so I don't know why they would upscale the height of the video?

It's an SD source, so upscaling doesn't make it HD quality. You upscale the height because other wise 640x480 within a 1280x720 sized frame will still be SD sized (there will borders around all the image, not just left and right pillars and the image will look very tiny on the HDTV) . IT's 640x480, because you're using square pixels now. However, a 720x480 SD source is upscaled to whatver your display size is , and the display will put on the pillarbox to keep the AR

[VideoFanatic]

So is "deinterlacing artefacts" the correct term? It's horizontal lines I'm seeing. It's weird, I played the Bluray in that TV shop on very expensive Bluray players and TVs and saw what looked like interlacing artefacts, I played a DVD on my cheap little TV/DVD combo and didn't see any problems which is why I'm starting to think this is a problem related to Bluray players.

Why would the WWE upscale the video then? I know it's still SD but they still upscaled it a bit to fit 1280 x 720. Why wouldn't they just keep it 720 x 480 as upscaling loses quality?

[poisondeathray]

horizontal lines ?????!!! This makes me think something wasn't authored correctly

No , I'm talking about line twitter and aliasing . Looks similar to the 3rd image here:
http://en.wikipedia.org/wiki/Interlaced_video#Interline_twitter

That's a simple bob deinterlace (and that's what the majority of displays do, but the more expensive ones add other post processing)

Of course, it's not that noticable on "normal" footage (that graphic is really worse case illustration). But it looks like "buzzing lines" or "marching ants". Only video enthusiasts/geeks analyze test patterns

Why would the WWE upscale the video then? I know it's still SD but they still upscaled it a bit to fit 1280 x 720. Why wouldn't they just keep it 720 x 480 as upscaling loses quality?

Probably marketing reasons. ohhhh yummmy HD WWF/WWE classics. HD is a selling point. But of course it's a SD source, that's all they used back then.

If the upscaling from your slow offline processing is better than the hardware display (on the fly) upscaling , then you can make a case for upscaling. It's the same argument for deinterlacing. QTGMC is better than 99.9999% of the hardware deinterlacers found on consumer equipment. The downsides are processing time, compatiblity (as you see 480p59.94 isn't a standard format) - hence another reason why I suggested ditching optical media for a media player in yoru other thread

[jmac698]

Hi,
I looked at your clip earlier but I can't remember it now. Anyhow, if TBC is an issue, I've now updated that thread with a new plugin which is much faster. It still needs some quality improvement though.

[VideoFanatic]

OK thanks. Here's my clip: https://dl.dropbox.com/u/57941257/Raw%201996%20March%2011%20Still.mpg

Could you please give me a link to the TBC thread with that plugin. If possible could you please give me a TBC script to use because I tried using the script there and my video looked far worse than if I hadn't used the TBC!

[sanlyn]

A tbc , whether hardware or software, won't do much for that video. You know the old saying: you can't make a silk purse, etc., etc....

[themaster1]

there's nothing wrong with the interlacing in your sample, just a noisy source imo

[jagabo]

A line TBC should been used before the analog video was capture -- long before the DVD was made. A software TBC might help because the blacks outside the active picture area down to near Y=0, whereas the blacks in the body of the picture are around Y=16.

[sanlyn]

And because of that, the "dancing pixels" can be removed? I can understand how that might facilitate repairing some horizontal line slippage. It fixes jumping diagonal jaggies and dot crawl as well? I'll have to try that myself.

[jmac698]

It's not going to fix dot crawl. Let me process this myself.

#RAW sample
#simple tbc

#Requirements:
#findpos 0.1 http://www.sendspace.com/file/fxzxr5
#dejitter 0.2 http://www.sendspace.com/file/upms1f

#Usage:
#set the dir to your clip directory
#set the filename

#open one file
dir="C:\project001a\sampleproblem8-raw\"
fn="Raw 1996 March 11 Still.mpg"
src=directshowsource(dir+fn)
#return src

#tbc
thresh=14
#Mark video edges
findpos_h(src, x1=0, x2=0, thresh=thresh)#search for the first bright pixel, starting 0 from left and 0 from right

#Line up video
rescale(src,last)
bilinearresize(640,last.height)#fix aspect ratio

function findpos_h(clip src, int "searchwidth", int "x1", int "x2", int "thresh"){
    searchwidth=default(searchwidth,32)
    x1=default(x1,0)
    x2=default(x2,0)
    l=findpos(src, x1=x1, x2=x1+searchwidth, thresh=thresh).crop(0,0,-src.width+2,0)
    r=findpos(src.fliphorizontal, x1=x2, x2=x2+searchwidth, thresh=thresh).crop(0,0,-src.width+2,0)
    stackhorizontal(l,r).pointresize(2,src.height)
    greyscale
}

function Rescale(clip src, clip offsets) {
    #Rescales a color video according to offsets
    #This function is necessary to make up for the incomplete nature of the dejitter 0.1 plugin
    #It works by splitting a color video into 3 grey planes, applyiing dejitter to each, then merging them back together
    y=src.Greyscale
    u=utoy(src)
    w1=u.width
    h1=u.height
    u=u.pointresize(y.width,y.height)
    v=vtoy(src).pointresize(y.width,y.height)
    y1=dejitter(y,offsets)
    u1=dejitter(u,offsets).pointresize(w1,h1)
    v1=dejitter(v,offsets).pointresize(w1,h1)
    ytouv(u1,v1,y1)
}

I don't see a lot of jitter, or any dot crawl. It seems to be just noise.

[sanlyn]

The script looks good, jmac698. I'll give it a run. Keep up the good work on your project. Yes, the "noise" and jaggies are awful.

[jmac698]

Yes, just to let you know, there's news since July, I've made my script into a plugin which is much faster.

I want to point out that you can use custom border detection.
The clip offsets is two grey values. For example

stackhorizontal(blankclip(color_yuv=$80000),blankclip(color_yuv=$40000))

where the left/right grey values are the shift to apply on that line.
The above example would take 8 pixels off the left and 4 pixels off the right, for example. You would never do that though. You can use masktools and it's fancy edge-finding algorithms to set the edge positions on each line.

[sanlyn]

Thanks again, jmac698. You're really sending me back to school on some of this! Gotta learn sometime.

[themaster1]

Is it me or Jmac's script is doing more harm than good here? I know Jmac is a script guru and all so maybe that's me

For the noise i would have proposed This

[sanlyn]

Every time I go to Sendspace, I' have to think twice before pursuing a project. Kaspesky and NOD32 love to flash this love note at Sendspace. I know the bad guy is just adware, but it kinda slows me down::


[Attachment 13425 - Click to enlarge]

On the other hand, after I see messages like these for over an hour . . .


[Attachment 13426 - Click to enlarge]

[Attachment 13427 - Click to enlarge]

. . . I tend to put the proposal aside for another day.

[jmac698]

Yes, the ideal result with that source would be to not touch it, however it does make it a bit worse. I know there's quality issues, and this is a good case to examine.

Can you suggest a better filehoster? For me, it's very easy to put up a file, the problem is that it's harder for some people to download the file. There's several download links which can be confusing.

[themaster1]

Mediafire.com is fine for me you can create a free account and manage your files

[sanlyn]

I've used 4shared.com for 3 years, but mine has always been an annual paid account. Not really complaining about the overdone "virus" message, it's just some of their popup ads that set off warning bells with some antivirus software -- even when nothing much is happening. I don't know of any file hosting site that is perfect.

[VideoFanatic]

Sanlyn I have only just now got an 8-core PC so I've been testing this script you gave me:

MPEG2Source(vidpath+"Raw 1996 March 11 Still.d2v") 
Crop(2,0,0,-12) 
AddBorders(0,6,2,6) 
ColorYUV(off_y=10,gamma_y=-15) 
ColorYUV(cont_v=-12) 

MCTemporalDenoise(settings="low",sigma=5,AA=true,useEEDI2=false, enhance=true, DeBlock=true, interlaced=false) 

ConvertToYUY2(interlaced=false) 
TemporalSoften(4,4,8,15,2) 
ConvertToYV12(interlaced=false) 

QTGMC(Preset="medium",InputType=1) 
RemoveSpots() 
LSFMod(strength=50) 
AddGrainC(var=1.0,uvar=1.0)

I've uploaded a longer version (47 secs) of my test video here.

Your original script was for making a progressive video. Since Bluray doesn't support 60 FPS for 720 x 480 I can't do that so I'll have to keep it interlaced. Also I highlighted the parts of your script which I can't use because it slows down the encoding time too much.

I'm able to get 10-11 FPS encoding speed with your script (minus the highlighted parts). I've modified it a bit here:

setmtmode(5,12)
Mpeg2Source("J:\2 = New\z = Encode\Raw 1996 11 March 47 secs.d2v")
setmtmode(2,0)
Crop(2,0,0,-12)
AddBorders(0,6,2,6)

ColorYUV(off_y=10,gamma_y=-15)
ColorYUV(cont_v=-12)

########## Not sure if I need this

ConvertToYUY2(interlaced=true)
TemporalSoften(4,4,8,15,2)
ConvertToYV12(interlaced=true)

########## Remove Dancing Pixels. AddGrain is needed as it covers up the remaining spots that RemoveSpots can't remove

RemoveSpots()
AddGrainC(var=1.0,uvar=1.0)

########### Denoise

McTemporalDenoise(settings="medium", sigma=5, Chroma=true, useQED=false, DeBlock=true, interlaced=true)

########### De-interlace and Re-interlace. Removes de-interlacing artefacts for crap TV media players that may show them

AssumeTFF() # or AssumeBFF() depending on the source
QTGMC(Preset="Super Fast") # Strangely this is faster than Ultra Fast
Vinverse() # Removes any remaining combing artefacts that QTGMC doesn't remove
SeparateFields()
SelectEvery(4,0,3)
Weave()

########### Sharpening 

LSFMod(strength=50)

I'm using the Medium setting for McTemporalDenoise instead of Low like in your script. Is that OK?

I noticed that AddGrainC actually covers up the remaining dancing pixels in the still frames that RemoveSpots didn't remove. Is that correct? Does AddGrainC in this script just add invisible grain to cover up noise? If not, it looks like it's adding grain to the rest of the picture as well such as the man talking at the start. That's why I put McTemporalDenoise after AddGrainC so that if there is visible grain then it will be removed. Note that the dancing pixels are still covered up however in the still frames.

I thought the reason you suggested TemporalSoften was to remove the flicker in the bottom right of the picture? However when using my script, the picture looks the same with or without TemporalSoften and it actually lowers the quality of the whole video by softening the picture. There is still some flashing flicker at the start of the video. Is there any way to remove it?

Is TemporalSoften the latest version of TemporalSoften2?

1. I moved TemporalSoften above McTemporalDenoise otherwise it slows down the encoding time. That won't cause any problems will it?
   
   
2. Can I assume that I don't need CPU=4 or CPU=6 because McTemporalDenoise is a better denoiser and deblocker? If so, which is better, DeBlock or Deblock QED? For whatever one is best what script should I use for it to give maximum deblocking strength? I assume that if there's nothing to deblock then it won't do any harm to the video and if there is something to deblock then it will remove it without harming the video?
   
   
3. What does the sigma setting do?
   
   
4. In McTemporalDenoise I noticed that "useEEDI2" is already false in Medium settings. So should I just remove that part of the code from my script then or is "true" better?
   
   
5. Is LSFMod(strength=50) a safe value to use for any video of this quality? Should I try a higher value?
   
   
6. Can you suggest any other McTemporalDenoise settings that you think will get better results?

Apart from the stuff in my script I also tried these but I can't use them because they slow down my encoding time:

• useTTmpSm
• AA #I noticed then even without this enabled McTemporalDenoise still does Anti-Aliasing on the ring ropes on another video which is what I want.
  
• Enhance
• stabilize=true = 8.80 FPS

[manono]

Vinverse() # Removes any remaining combing artefacts that QTGMC doesn't remove

Almost by definition, there can't be any interlacing left over after the use of a bobber. Unless you do something stupid like resize improperly.

I don't think your crop and addborders did anything screwy, but I still do that kind of thing at the end (or after your SeparateFields line, while it's still progressive). You did change the luma values because of your ColorYUV lines afterwards, so those borders are no longer a proper black.

[VideoFanatic]

See my de-interlacing comment above AssumeTFF.

[VideoFanatic]

I'm guessing Sanlyn put the Crop at the top to lessen the encoding time slightly. Should I just leave the crop at the top and put the add borders at the end of the script?

Any idea how I can preview my changes? I used to use AvsPmod but it's really slow when McTemporalDenoise is used. VirtuaDub is faster but still slow. Is there a faster program or is there any way to speed it up?

[manono]

I'm guessing Sanlyn put the Crop at the top to lessen the encoding time slightly. Should I just leave the crop at the top and put the add borders at the end of the script?

I don't know. You can ask him or he'll see this and reply soon. Yes, sometimes I'll crop at the top (but it's risky as sometimes the script won't open afterwards, or it'll look weird with oddly displaced colors) and do the addborders at the end.

Any idea how I can preview my changes? I used to use AvsPmod but it's really slow when McTemporalDenoise is used. VirtuaDub is faster but still slow. Is there a faster program or is there any way to speed it up?

If you're opening and scrolling around in a script, it might just be slow when you're using slow filters. You can comment out the slow stuff (your MCTemporalNoise) so you can look at the rest of it. Or, you can trim off a small portion and just encode a little bit to see if the result pleases you.