How to Fix Patchy Colour with Avisynth?

[VideoFanatic]

As the title. Here's a few seconds of the video: http://www.mediafire.com/?sn33cb559ppc3b5

Also, what's the correct term for blotchy colour?

[sanlyn]

Those blue blotches are called...well...blotches or patches, AFAIK. Many classify this with rainbowing effects. It doesn't "look like" rainbows, but it's a similar discoloration. Some call it "sunburn", like potato skin discolored blotches. You might notice that the problem is in the background as well, especially in shadows, and the blotches are full of grainy dot crawl. It would be helpful to know something of the history of this poor tape dupe (which is what it looks like to me).

You might want to take a look at a website with helpful explanations of these problems and examples of dealing with them. The examples are from anime, but I think they better illustrate the action of the filters discussed -- almost all of them work with any kind of video. This page has examples and tips for rainbows, dot crawl, and heavy noise areas. Some of the filters discussed are Bifrost for rainbows and blotches, DeDot for dot crawl, and DFTtest for patches with embedded grain.

You'll likely find much more at the AMV website that's helpful with any kind of video, but I think the filtering page referenced above is one of the better Avisynth filter guides around.

[sanlyn]

The sample is straight interlaced video. But it appears that at some point a deintelace, field separation/rejoin, IVTC, or similar operation was performed incorrectly. In other words, it's a mess. I can take another look later.

[VideoFanatic]

All I know is that it was from a VHS to DVD conversion. I got the video from someone else. I tried BiFrost(interlaced=true) but it didn't do anything.

[sanlyn]

Bifrost itself won't remove a great deal, especially if you ran it on interlaced video. It would take several filters to clean screwed-up chroma. You'd have to work with either SeparateFields() or temporarily deinterlace (either way, rejoin the fields later). It will take more than one filter to clean this. There is serious aliasing, jaggies, other problems. It's difficult to believe it's a simple VHS-toDVD transfer, there's too much damage -- although the patchy color alone could result from a bad tape-to-tape dupe. If possible, you're better off working with the source VHS but I suspect the source itself was a tape-to-tape or camera-to-tape dupe.

Chances are, those blotches won't be cleared completely. The best you'll likely get is to smooth and de-emphasize it. It would help if you reduce blue saturation, as well. There's too much blue to begin with.

[themaster1]

@ holygamer:
I think i have improved it a little (you tell me), picture:
http://imageupload.org/?d=89F947FF1

You need colormill and camcorder denoise
link for the Vcf : http://www.sendspace.com/file/yzkrz3

[sanlyn]

Not bad, themaster1. Kinda what I tried in Avisynth + VirtualDub. My experiment ended up pretty red, too, couldn't get black out of it because you have to cut so much Blue. Yours might be as good as it can get (and who knows what the original color was like ???). Those jaggies and other interlace damage, though - - it's inconceivable that the original looked like that. I ran antialising several times, but the artifacts just wouldn't give up without turning the image to soft mush.

[VideoFanatic]

themaster1 can you please give me the script you used. Will it work in Avisynth?

Could you please give me a link to where I can download camcorder denoise.

[themaster1]

Well in avisynth that be like this:

MPEG2Source("C:\Desktop\colour test.d2v", cpu=0,info=3)

converttorgb32(interlaced=true)
LoadVirtualdubPlugin("C:\Program Files (x86)\virtualdub1.9\plugins\ColorMill.vdf","colorm ill",1)
colormill(25700, 31076, 21604, 25700, 25700, 25700, 25734, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 5)

LoadVirtualdubPlugin("C:\Program Files (x86)\virtualdub1.9\plugins\Camcorder_Color_Denois e_sse2.vdf","ccd",1)

ccd(41,1)

[themaster1]

You could even add another instance of colormill after ccd to shift the colors by +13 units so:
LoadVirtualdubPlugin("C:\Program Files (x86)\virtualdub1.9\plugins\ColorMill.vdf","colorm ill",1)
colormill(25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25713, 25700, 25700, 25700, 1124, 5)

It looks better ( more natural tone ) to me

Edit:
Final touch (for me at least), includes deblocking and antialiasing

picture: http://imageupload.org/?d=8C0F44321

Script: http://www.sendspace.com/file/ennrf8

[VideoFanatic]

The file you gave me is called colour.vcf. When I use your script it says there's an error opening the file on this line:

LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMill.vdf","colorm ill",1)

Am I supposed to rename the colour.vcf as something else? I tried called it ColorMill.vdf but it still doesn't work.

[themaster1]

the .vcf that's for virtualdubmod you're supposed to load (file> load processing settings) it but like i made the avisynth script i don't think that's necessary

[VideoFanatic]

Could you please give me a link to where I can download the ColorMill.vdf file to use in Avisynth

Also how did you come up with that script. Is there a page anywhere which explains the settings to use. Thanks

[themaster1]

Link for colormill: http://fdump.narod.ru/rgb.htm

How i made this script ? Hmm i don't know lots of hours reading on doom9 and experimenting with vdub/avisynth. There are far more complicated scripts so don't put on a pedestal yet

[VideoFanatic]

I use this on line 6 & 7 of my script:

LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\Camcorder_Color_Denoise_sse2.vdf","ccd ",1)
ccd(41,1)

I get the following error:

Script error: Invalid arguments to function "ccd"(New File, line 7)

I have the following files. What should they be named as and what should I have for my above script?

ccd.vdf
ccd_sse2.vdf

[themaster1]

http://www.sendspace.com/file/ad3g9p
there you go (ccd filter)

[VideoFanatic]

Thanks. I don't get errors any more and I can see the changes in AvsPmod. However when I click on Encode in HC Encoder I get a message saying that HC Encoder stopped working. If this is of any use - it said the fault module name was: ntdll.dll

This is the script I'm using:

Mpeg2Source("K:\New\z = Encode\9 = colour test.d2v", CPU=6,info=3)
DeGrainMedian(limitY=2, limitUV=3, mode=1, interlaced=true)
converttorgb32(interlaced=true)
LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMill.vdf","colormill",1)
colormill(25700, 31076, 21604, 25700, 25700, 25700, 25734, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 5)

LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\Camcorder_Color_Denoise_sse2.vdf","ccd ",1)

ccd(41,1)

LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMill.vdf","colormill",1)
colormill(25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25713, 25700, 25700, 25700, 1124, 5)

ConvertBackToYUY2()
Crop(0,0,0,-14)
Tweak(Bright=-0, Sat=1.5, Cont=1.0, Hue=0.0, Coring=False)
AddBorders(8, 0, 8, 14)

[themaster1]

i have no idea of what's triggering this error, perhaps DeGrainMedian...

edit

or perhaps this strange path:
Mpeg2Source("K:\New\z = Encode\9 = colour test.d2v"

[VideoFanatic]

It seems to be colormill that's causing HC Encoder to crash. Does anybody else know how to fix this? Are there any alternatives to HC Encoder?

[sanlyn]

Mpeg2Source("K:\New\z = Encode\9 = colour test.d2v", CPU=6,info=3)

I'm following themaster1 on this path statement. What's it doing?

Crop(0,0,0,-14)
Tweak(Bright=-0, Sat=1.5, Cont=1.0, Hue=0.0, Coring=False)
AddBorders(8, 0, 8, 14)

What's the size of your video frame after this crop + borders?

I can imagine HC giving error messages, but just crashing to a halt with no message -- ??

[VideoFanatic]

AvsPmod says it's 720 x 480 after I add my borders.

I use that path all the time without problems. Anyway I tried a different path of:
Mpeg2Source("K:\New\colour test.d2v", CPU=6,info=3)

Same thing happens, HC Encoder just crashes without giving an error message

I've got the following files in the Avisynth plugins directory:

Camcorder_Color_Denoise_sse2.vdf
ColorMill.vdf
ccd_sse2.vdf
ccd.vdf

This is my script. I've noticed that if I remove the 2nd instance of ColorMill then it works fine but if I leave the 2nd instance there then it crashes.

Mpeg2Source("K:\New\colour test.d2v", CPU=6,info=3)
DeGrainMedian(limitY=2, limitUV=3, mode=1, interlaced=true)
converttorgb32(interlaced=true)
LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMill.vdf","colormill",1)
colormill(25700, 31076, 21604, 25700, 25700, 25700, 25734, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 5)
LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\Camcorder_Color_Denoise_sse2.vdf","ccd ",1)
ccd(41,1)
LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMill.vdf","colormill",1)
colormill(25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25713, 25700, 25700, 25700, 1124, 5)
ConvertBackToYUY2()
Crop(0,0,0,-14)
Tweak(Bright=-0, Sat=1.5, Cont=1.0, Hue=0.0, Coring=False)
AddBorders(8, 0, 8, 14)

Why would it crash on the 2nd instance of ColorMill but not the first?

[themaster1]

There is no reason it crash, i use hcenc myself, this never happen

By the way you only have to load Color_Denoise_sse2.vdf not ccd.vdf

What's the original resolution of your video ? You are adding 8pixels to the right and left yet you haven't cropped right/left ,this seems strange to me

[VideoFanatic]

It was 704 x 480. Bluray only allows 720 x 480 so I have to add borders. I do it all the time without problems.

By the way you only have to load Color_Denoise_sse2.vdf not ccd.vdf

So what script should I use instead of this?

LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\Camcorder_Color_Denoise_sse2.vdf","ccd ",1)
ccd(41,1)

[themaster1]

yes

[VideoFanatic]

There's a misunderstanding. I thought you said my script was wrong so I was asking what I should use instead of it. Anyway this is my whole script. Is this correct?

Mpeg2Source("K:\New\colour test.d2v", CPU=6,info=3)
DeGrainMedian(limitY=2, limitUV=3, mode=1, interlaced=true)
converttorgb32(interlaced=true)
LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMill.vdf","colormill",1)
colormill(25700, 31076, 21604, 25700, 25700, 25700, 25734, 25700, 25700, 25700, 25700, 25700, 25700, 1124, 5)
LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\Camcorder_Color_Denoise_sse2.vdf","ccd ",1)
ccd(41,1)
LoadVirtualdubPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMill.vdf","colormill",1)
colormill(25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25700, 25713, 25700, 25700, 25700, 1124, 5)
ConvertBackToYUY2()
Crop(0,0,0,-14)
Tweak(Bright=-0, Sat=1.5, Cont=1.0, Hue=0.0, Coring=False)
AddBorders(8, 0, 8, 14)

I've also got the following files in the Avisynth plugins folder:
I've got the following files in the Avisynth plugins directory:

Camcorder_Color_Denoise_sse2.vdf
ColorMill.vdf
ccd_sse2.vdf
ccd.vdf

Is that all correct? Could you please test that script youself with my test file at http://www.mediafire.com/?sn33cb559ppc3b5 to see if it crashes for you as well.

[Gavino]

I've noticed that if I remove the 2nd instance of ColorMill then it works fine but if I leave the 2nd instance there then it crashes.

I don't know if it will make any difference, but try removing the 2nd call to LoadVirtualDubPlugin("ColorMill.vdf"), while leaving the 2nd call to colormill() itself.

[VideoFanatic]

Still crashes

[VideoFanatic]

I guess I'll just use 1 instance of colormill and see if I can use another plugin to get the natural colour tones back. Any suggestions or could I use Tweak?

[themaster1]

if you want one instance only, including hue shift:
(25700, 31076, 21604, 25700, 25700, 25700, 25734, 25700, 25700, 25713, 25700, 25700, 25700, 1124, 5)

[VideoFanatic]

Thanks it works. But there still seems to be an orange tint. Any way I can remove it?