ColourLike problem

[Gavino]

Can you feather the edges? Other than just using Blur?

Blur() is probably the simplest way, but you can easily make a linear gradient by amending the formula appropriately. Or even a non-linear gradient if you can express it as a formula based on "x" (the original luma).

Time for me too, gavino. I was just in doom9 looking at it. The idea has been been scaring me to death for years.

It looks scary at first, with so many different functions, many of which you probably won't use, but it's worth investing the time to learn the basics. The 'reverse polish' notation used for the mt_lut* family of functions is hard to get used to if you haven't come across it before, but you can avoid it by using the mt_polish function to do the conversion from 'normal' (infix) expression form.

[jagabo]

Can you feather the edges? Other than just using Blur?

Blur() is probably the simplest way, but you can easily make a linear gradient by amending the formula appropriately. Or even a non-linear gradient if you can express it as a formula based on "x" (the original luma).

Thanks. Time for some reading...

[sanlyn]

SmartSmoother is bringing results. Takes plenty of fiddling. But I'm still looking . . .

[sanlyn]

Folks, it seems RemoveDirt is causing more problems than it solves, at least the way we're using it. The plugins that it calls are over-filtering and over-sharpening. Trial runs with the original cap and some light-duty temporal filters show that the shadow problems just get worse with RemoveDirt. I'd drop farther down into that function call and tweak some of the settings (there are plenty to tweak, too!).

Meanwhile I ran some fairly clean videos, will try to post something later.

[sanlyn]

For anyone who didn't get the 1/2-GB AVI, here's a quick look at the first few seconds of the unmodified clip
original, warts and all:
NTSC MPEG2 720x480 (4:3 display), 6.7 MB - NO AUDIO
http://dc266.4shared.com/download/9yIErstx/CherRosie_quicklook.mpg

I converted the downloaded AVI from YUY2 to RGB. This involved two steps, from whatever colorspace the download is in (one media app sez RGB, another sez YUY2, but AviSYnth won't allow me run YUV scripts on the download. I open in AviSYnth using ffdshow's huff, then fast-recompress to huff YUY2, then disable ffdshow, then run a YUY2->RGB script). I don't know what version of huffyuv cherbette uses, but I can only decode his AVI's using ffdshow - which I must then disable, or I can't read any other AVI from anywhere. The only work done in this script is some quick ColorYUV just to keep levels legal for the whole clip, which varies in brighntess/contrast. No colors yet.
Rather Noisy.

AviSource("G:\forum\Biggie\BiggieYUY2.avi")
ColorYUV(off_y=+20.0,gamma_y=-110.0,gain_y=+2.0,cont_y=-30.0)
Tweak(coring=false,Sat=1.2)
AssumeTFF()
ConvertToRGB32(matrix="Rec601",interlaced=true)

NTSC MPEG2 720x480 (4:3 display), 12.3 MB - NO AUDIO
http://dc436.4shared.com/download/M5ZRCNgQ/CherRosieYUY2_toRGB.mpg

The first1/3 of the trial effort with RemoveDirt and a few other noise filters. The script is an extension of the one above; the same ColorYUV prep in YUY2, plus more filters. Some filters are there to counteract overkill from RemoveDirt. It looks too filtered and too sharpened, even without the other softening filters used with it. Might be a good point to see what some of those values in RemoveDirt are doing.
NTSC MPEG2 720x480 (4:3 display), 12.3 MB - NO AUDIO
http://dc227.4shared.com/download/1jlRCzgk/CherRosie_RemoveDirt.mpg

AviSource("G:\forum\Biggie\BiggieYUY2.avi")
ColorYUV(off_y=+20.0,gamma_y=-110.0,gain_y=+2.0,cont_y=-30.0)
Tweak(coring=false,Sat=1.2)
in1=last
e1=in1.AssumeTFF().SeparateFields().SelectEven()
o1=in1.AssumeTFF().SeparateFields().SelectOdd()
#  ----------------------
#   input "In1" is YUY2
#  ----------------------

e1
TemporalSoften(4,4,8,15,2)
FixVHSOversharp(10,14,10)
FixVHSOversharpL(10,14,10)
ConvertToYV12(interlaced=false)
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)      
e2=last

o1
TemporalSoften(4,4,8,15,2)
FixVHSOversharp(10,14,10)
FixVHSOversharpL(10,14,10)
ConvertToYV12(interlaced=false)
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)
o2=last

interleave(e2,o2).weave().ConvertToRGB32(matrix="Rec601",interlaced=true)
return last

function RemoveDirt(clip input, int limit, int rgrain, bool "_grey")
{
  _grey = default(_grey, false)
  _dgr1 = 0.35+rgrain*0.3
  _dgr2 = 0.45+rgrain*0.4
  repmode = 1
  clensed = Clense(input, grey=_grey, cache=4)
  restore = input.FluxSmoothST(3+3*rgrain,rgrain)
  restore = Repair(restore, input, mode=repmode, modeU= _grey ? -1 : repmode)
  restore = rgrain==0 ? restore.RemoveGrain(1) : \
    restore.VagueDenoiser(threshold=_dgr1, chromaT=_dgr1, nsteps=7, percent=75).RemoveGrain(1)
  alt = input.VagueDenoiser(threshold=_dgr2, chromaT=_dgr2, nsteps=7, percent=100).RemoveGrain(5)
  return RestoreMotionBlocks(clensed, restore, neighbour=input, alternative=alt, pthreshold=4+2*rgrain, cthreshold=6+2*rgrain, gmthreshold=40, dist=1, dmode=2, debug=false, noise=limit, noisy=12, grey=_grey)
}

Finally, the same clip using the script from above, but less complicated and without RemoveDirt and some of the steps. On output from AviSynth, I loaded VirtualDub's temporal filter set to 4, which caught most of the slight noise that TemporalSoften left behind. Still work to do, but almost all the bristling noise is gone and less damage in the process.

NTSC MPEG2 720x480 (4:3 display), 12.3 MB - NO AUDIO
http://dc337.4shared.com/download/fDKt7JWX/CherRosie_LightFilters.mpg

AviSource("G:\forum\Biggie\BiggieYUY2.avi")
ColorYUV(off_y=+20.0,gamma_y=-110.0,gain_y=+2.0,cont_y=-30.0)
Tweak(coring=false,Sat=1.2)
TemporalSoften(5,4,8,15,2)
FixVHSOversharp(10,14,10)
FixVHSOversharpL(10,14,10)
ConvertToRGB32(matrix="Rec601",interlaced=true)

[Gavino]

e1=in1.AssumeTFF().SeparateFields().SelectEven()
o1=in1.AssumeTFF().SeparateFields().SelectOdd()

e1
TemporalSoften(4,4,8,15,2)
FixVHSOversharp(10,14,10)
FixVHSOversharpL(10,14,10)
ConvertToYV12(interlaced=false)
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)      
e2=last

o1
[... as above for e1]
o2=last

interleave(e2,o2).weave().ConvertToRGB32(matrix="Rec601",interlaced=true)

Calling ConvertToYV12() on a field-separated clip will result in chroma placement errors (slight shifts), as it is not designed to handle these (regardless of the 'interlaced' parameter). Depending on source, the result may not be noticeable, but in principle it is incorrect. I suggest doing it like this:

in1
TemporalSoften(4,4,8,15,2)
FixVHSOversharp(10,14,10)
FixVHSOversharpL(10,14,10)
ConvertToYV12(interlaced=true)

AssumeTFF().SeparateFields()
e1=SelectEven()
o1=SelectOdd()

e1
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)      
e2=last

o1
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)
o2=last

interleave(e2,o2).weave().ConvertToRGB32(matrix="Rec601",interlaced=true)

[sanlyn]

e1=in1.AssumeTFF().SeparateFields().SelectEven()
o1=in1.AssumeTFF().SeparateFields().SelectOdd()

e1
TemporalSoften(4,4,8,15,2)
FixVHSOversharp(10,14,10)
FixVHSOversharpL(10,14,10)
ConvertToYV12(interlaced=false)
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)      
e2=last

o1
[... as above for e1]
o2=last

interleave(e2,o2).weave().ConvertToRGB32(matrix="Rec601",interlaced=true)

Calling ConvertToYV12() on a field-separated clip will result in chroma placement errors (slight shifts), as it is not designed to handle these (regardless of the 'interlaced' parameter). Depending on source, the result may not be noticeable, but in principle it is incorrect. I suggest doing it like this:

in1
TemporalSoften(4,4,8,15,2)
FixVHSOversharp(10,14,10)
FixVHSOversharpL(10,14,10)
ConvertToYV12(interlaced=true)

AssumeTFF().SeparateFields()
e1=SelectEven()
o1=SelectOdd()

e1
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)      
e2=last

o1
RemoveDirt(10,0,false)
DeGrainMedian(limitY=5,limitUV=7,mode=3)
o2=last

interleave(e2,o2).weave().ConvertToRGB32(matrix="Rec601",interlaced=true)

I appreciate that tip, Gavino. Man, you learn something different every 15 minutes around here! I'll rework those clips this weekend, and will place a note in the original post today. Thank you.

[sanlyn]

The script in post #155 mentioned by Gavino is amended. Running TemporalSoften and the VSHOversharp filters on the interlaced clip seemed to make little or no difference as far as denoising, but RemoveDirt and DegrainMedian were more effective with fields separated. Results of the amended script did look better, even if you'd have to compare old and new frame by frame to see it.

[sanlyn]

I forgot: because different scripts and YUV adjustments were used as above, the color filters and tweaks posted earlier won't work.

[Cherbette]

Just catching up here...it looks like you've made some really strong steps in the right direction here sanlyn. I think your samples above really look so much better than the original...which is what I was hoping for. Not expecting miracles because we all know it wasn't great to start with.

So now I'm going to do some toying around with the scripts you posted above. I appreciate all the help so very much!

[magillagorilla]

I dunno, you guys are way better at this than me.

[sanlyn]

magillagorilla, those rough shadows look pretty smooth, and what detail (?) there is in the hair hasn't been wiped out. Kinda makes you wish that previous owners of the clip had not worked so hard to wipe out highlights.

cherbette, this one's a toughie. Looks like one of those projects you keep going back to over time, piece by piece. It would take some mighty sophisticated skill with several tools to make this one look pretty. You're right, it's kinda wrecked. When I get a doozy loike this, I take a break with something easier for a while, to catch my brteath. Then I come back to the doozy and go crazy again.

[Cherbette]

yeah, I had to take a break from this one for a few days. I'm hoping I'll get my hands on a better transfer of this Rosie show but until then...there have been some great improvements thus far IMO

[Cherbette]

I finally ordered a S-video cord for my Panasonic DMRE55 that I'm using as a pass-thru. I'm hoping that may help some things like perhaps this Rosie clip even if only a slight improvement

[sanlyn]

I finally ordered a S-video cord for my Panasonic DMRE55 that I'm using as a pass-thru. I'm hoping that may help some things like perhaps this Rosie clip even if only a slight improvement

Depends on the cable. Most retail s-video cable has a soft image, some of the silver-plated stuff (BetterCables) has some horrible transmission noise and moire on fine detail, some (Monster) have cyan blotching, odd-looking skin tones, edge artifacts, and noisy reds. But there should be overall a greater feel of clarity compared to composite. Here's a post on the s-video cable my videophile group uses most of the time:
https://forum.videohelp.com/threads/338523-S-Video-Vs-Composite-for-Black-Whit-captures...=1#post2104588

In any case, s-video will generally get better results than composite, and the Panasonic's s-video converter does good work.

[Cherbette]

Depends on the cable. Most retail s-video cable has a soft image, some of the silver-plated stuff (BetterCables) has some horrible transmission noise and moire on fine detail, some (Monster) have cyan blotching, odd-looking skin tones, edge artifacts, and noisy reds. But there should be overall a greater feel of clarity compared to composite. Here's a post on the s-video cable my videophile group uses most of the time:
https://forum.videohelp.com/threads/338523-S-Video-Vs-Composite-for-Black-Whit-captures...=1#post2104588

In any case, s-video will generally get better results than composite, and the Panasonic's s-video converter does good work.

http://www.amazon.com/S-Video-Cable-ft-Gold-Plated/dp/B0002MQGK4/

This is the one I ordered...not sure where this one would fall in those guidelines. I hope it's not crap. lol

[sanlyn]

Hmm. Not the best around, but certainly not the worst, and definitely better than any OEM composite wire.

[Cherbette]

I'm sure if I asked 10 different people I'd get 10 different answers BUT...what would be the preferred wire? Based on the thread you referred, I was a bit confused by all of the discussion as to which gave the "best" result. I'm assuming it's either the RCA VH976 or the Acoustic Research PR-121 Pro Series II?

[jagabo]

Since your original source is noisy, low resolution, multigeneration VHS, the only real difference s-video cables will make is the amount of dot crawl artifacts between the DVD recorder and the capture card. A lot of devices apply different brightness/contrast filters for different inputs. But those can be compensated for with the brightness/contrast controls during the cap or by filtering afterward.

There's little difference between cheap and expensive s-video cables.
https://forum.videohelp.com/threads/221249-Test-Caps-various-composite-and-s-video-cabl...ight=composite

[Cherbette]

Yeah, I figure it won't improve much if anything at all on many of my multigeneration VHS but I have plenty that I've recorded myself on decent quality VHS tape that should fair better with the S-video I'm hoping. I'm at least hoping it will get the "best" of what the multigeneration tapes have to offer in cases where I don't have another source for a better copy.

[jagabo]

Even on first generation VHS tapes the bandwidth and signal to noise ratio of VHS is far below that of composite. As long as you aren't using long (>>12 feet) cables. The major difference will be the dot crawl artifacts. The question here will be how good the comb filter in the DVD recorder is, vs how good the comb filter in the capture device is. And since we're only talking about the last link (between DVD recorder and capture device) how good the composite and s-video circuits in the two devices is.

[Cherbette]

I should also be able to capture from cable broadcasts better this way as well I would think? I know I had recorded several things from the RCA out on my Comcast box to the RCA in on the VC500 and the picture wasn't very crisp at all. I'll post some results once it arrives...

[jagabo]

I should also be able to capture from cable broadcasts better this way as well I would think? I know I had recorded several things from the RCA out on my Comcast box to the RCA in on the VC500 and the picture wasn't very crisp at all. I'll post some results once it arrives...

You should see more difference from the cable box. Although most digital cable systems do no transmit full D1 video. They typically use something like a 540x480 frame size. So the picture still won't be razor sharp.

Although, on our cable box the s-video output appears to be made from the composite signal via a cheap comb filter -- it's full of dot crawl artifacts. Capturing the composite output with an ATI 650 USB2 is better because the 650 has a good comb filter.

[sanlyn]

Since your original source is noisy, low resolution, multigeneration VHS, the only real difference s-video cables will make is the amount of dot crawl artifacts between the DVD recorder and the capture card. A lot of devices apply different brightness/contrast filters for different inputs. But those can be compensated for with the brightness/contrast controls during the cap or by filtering afterward.

True. The big difference will be more a matter of less noise and fewer artifacts with s-video, not so much a matter of what people call "sharpness" (which involves more than resolution, but is affected by noise). Bad tape just looks bad, period, but less noise is always better. Yep, different devices have better or worse circuitry. Other things being equal, an s-video pic outr of Cherbette's Panasonic will look "different" in subtle ways from the same VCR/tape/wire/etc thru my Toshiba. But both will look pretty good.

There's little difference between cheap and expensive s-video cables.
https://forum.videohelp.com/threads/221249-Test-Caps-various-composite-and-s-video-cabl...ight=composite

My obsessive/compulsive videophile group (a really pissy-picky bunch, these folks) don't use test patterns to check cables. The problem with test patterns is that they don't move. They don't pose the same transmission conditions as "real" video. I've been looking at the referenced link for years and find it very handy. The difference between composite and s-video patterns is obvious, but differences between (for example) the "GE" and "Belkin" s-video patterns is less obvious but still visible. One problem with close comparisons is how the test image was processed; just as playback components have different circuitry, so all graphics apps don't produce compressed pics the same way. The silver-plated cable mentioned will have more transmission and motion noise than the others, and so will Monster's with its soldered joints and twisted-pair wire. You can't make final judgments in these tsets about color transmission (the gray patches are too green).

No, there's not much difference between cheap and expensive s-video cables as far as overall quality (which isn't very high), but there are differences - which we find obvious on first viewing. Most of them tend to look worse the longer you watch, regardless of price. But we do recall the specific s-video product Cherbette referenced. We found it slightly less contrasty than the best-contrast samples we tried, so there was a certain lack of "punch" (in comparison). But it's free of transmission noise, echo effects (moire), and so forth; its impedance is likely smooth across the desired impedance spectrum because color and tone gradations looked pretty clean. We didn't see edge noise or sparkles with high-contrast objects. Skin colors look "unprocessed", shadows are free of the infamous Monster shadow "blotching", and deep reds (low-frequency color) look clean. We'd say it's OK stuff by our criteria, which is basically that good wire inflicts less damage than crummy wire.

Here's a very old link of pics I posted while auditioning various cables: an example of how difficult it is to judge a cable thru still images, although you still see differences (the pics on the right have better shadow detail). https://forum.videohelp.com/threads/205330-VCR-Playback-too-hot?p=1373826&viewfull=1#post1373826 In a following post, EdDV rightly notes I did no levels correction but the point was to observe what each cable was doing when chained along the components, unmodified. Proc amp controls were disabled. EdDV judged the generic RCA I use today as "best", or at least better, and AR-Pro second. The color correction EdDV made later was not the way I'd do it -- I'd correct levels during capture first, not later, but EdDV had only the captures to work with.

Of course, if you see none of these effects with any s-video cable then it doesn't matter, test patterns or not. Still captures aren't that conclusive either, especially with tape.

[Cherbette]

Is the RCA you mention the "VH976" as seen below?
http://www.amazon.com/RCA-VH976-S-Video-Cable/dp/B00005T3F9/ref=sr_1_1?ie=UTF8&qid=1324250444&sr=8-1

[sanlyn]

Is the RCA you mention the "VH976" as seen below?
http://www.amazon.com/RCA-VH976-S-Video-Cable/dp/B00005T3F9/ref=sr_1_1?ie=UTF8&qid=1324250444&sr=8-1

Yep. Believe it or not, that's the original package I first spotted at CircuitCity in 2004! It looked kinda generic, but I tried it out of desperation; I had tried many more expensive big-name wire that was even worse than composite, except for the absence of composite dot crawl. No idea who makes it. The package was distributed by Recoton at CircuitCity and another chain (P.C.Richards). It also appeared under the "G-E" label; formerly sold by NewEgg, CablesForLess and an industrial supplier called MMC in a no-name plastic bag - and even bought a couple on eBay shipped in ZipLock bags! I've seen it listed for as little as 99 cents in various packages. I suspect the same wire is (was) sold in the UK by Profigold. The original price shown in the link (about $16) is what I paid in 2004. Since then I've collected more than a dozen copies from 6 to 25 feet.

At the price it might be worth a try. But most users wouldn't see much difference from the product you say you purchased. You'd have to spend a few hours watching exactly the same material played thru exactly the same equipment to spot any difference.

[sanlyn]

I should also be able to capture from cable broadcasts better this way as well I would think? I know I had recorded several things from the RCA out on my Comcast box to the RCA in on the VC500 and the picture wasn't very crisp at all. I'll post some results once it arrives...

You should see more difference from the cable box. Although most digital cable systems do no transmit full D1 video. They typically use something like a 540x480 frame size. So the picture still won't be razor sharp.

Although, on our cable box the s-video output appears to be made from the composite signal via a cheap comb filter -- it's full of dot crawl artifacts. Capturing the composite output with an ATI 650 USB2 is better because the 650 has a good comb filter.

Agreed, jagabo, there are differences in the SD outputs of cable boxes. Besides other lousy effects, our older digital SD box has temporal filter smear and other artifacts. The analog outputs of the newer HD box are cleaner and sharper, but copy protection is active on that newer box (shucks) and the HD image is letterboxed and pillared thru all but the component outputs. So I guess that makes me about ready for that Hauppauge HD PVR.

[Cherbette]

Yeah I suspect I would be better off connecting the cable wire from the Wall>DVD-R>VC500 would give better picture quality. The comcast HD box we have gives all kinds of funky effects as well. Unfortunately, only some channels come through when connected directly from the cable in the wall to DVD-R. It would have to go Wall>Cable Box>DVD-R>VC500.

[jagabo]

You don't need time base correction for cable TV.

[Cherbette]

That is correct but my cable box doesn't have S-video output...wouldn't I fair better doing it the way I mentioned for the benefit of S-video or would standard composite RCA cables do the same?