This is probably simple in Avisynth..., but..., (Help a poor girl out?)

[dedlykme]

Okay, so..., long time lurker -- first time poster.
I couldn't find a dedicated "avisynth" sub-forum, so I picked this one (hope that's okay).

I'm primarily an "audio chick", who has somehow managed to become fairly functional working
in Windows with digital video (Avisynth, VirtualDub, FFMPEG, Pinnacle Studio..., blah, blah....).
As a result, at the recording studio where I work I have sort of been dubbed the goto gal for video stuff.
(Their mistake.)
Well, on Monday I was asked to turn some existing raw footage into a 4 minute+/- low-budget music
video for one of the studio's clients, and for which I am being paid a small nominal fee.
(Also their mistake.)
I have now, seemingly, hit a brick wall trying to accomplish the goal which is outlined below.

The source of the problem (not a new one I'm sure):
Approximately 60% of the footage was shot with the intention of being chroma/color keyed over
various static & motion backgrounds. Fine..., not a problem so far.
The thing is that, while the foreground subject (the lead guitarist/frontman) looks good
(well lit, good white balance, color, etc.), the background to be keyed-out is beyond abysmal.
First, as is often the case with 'amateurish' footage, the lighting of the backscreen is so uneven
that it ends up being about 40 different hues of color.
Second, as is also often the case, the subject was too close to the backscreen so there tonzzz of shadows.
Lastly, (and this is a real piece of work), the backscreen is not green, blue, or even red.
It's Orange!!
(I know, right?!?! Who does that????)
Because of the uneven back-lighting, a lot of it ends up being almost flesh-toned.
The footage cannot be re-shot, because the band is on tour for the next 6 months.

So, I said to myself: "Rina, (that's me), you're just going to have to mask the heck out of this video."
Well, the first stumbling block was that, being a 'rockstar', the subject is pretty much animated all over the framespace.
This effectively ruled out "garbage masking" using a static overlay.
Thus began my journey this past week.

What I've done so far:
1. I loaded the video clip into VirtualDub, went into "Video-Filters", added "Brightness/Contrast" with the contrast cranked
up like crazy, added "Threshold", & adjusted it to then output 2 "binary" black & white clips (adding the "Invert" filter for
the 2nd one). This took all of 90 seconds, and the results look like this:
http://dedlykme.strangled.net/px/HighContrast_BW_Versions_SBS.png
2. Wrote a quick Avisynth script which can output one, or more, color permutations of the black & white binary clip(s),
(which, incidentally, has since evolved into a "function" script in case I ever have to do this again). For example:
http://dedlykme.strangled.net/px/PossibleOverlays.png

Now, the desired goal, (**and where I am stuck**), is to overlay/blend/merge (whatever...) two, or more, of the above
resulting clips into one(1) binary clip that looks something like:
http://dedlykme.strangled.net/px/UltimateGoal_SBS.png

All manner of changing Overlay "modes" (blend, add, chroma, luma, etc.), "opacity" levels, color tweaks, and so on, have failed miserably.

Late on Wednesday (thinking it might help), I crossed my fingers, bit my lower lip, and waded into the 'loveless swamp' that is the
documentation for Avisynth's "MaskTools"..., (What the heck is a "LUT"?!?!?).
More than 48 hours later I have now sunken up to my eyebrows into that MaskTools swamp and..., after many hours spent at
VideoHelp, Doom9, lots of trial & error, and learning new ways to 'Google' that I never wanted to know..., my eyes have glazed-over
with cataracts, my brain is numb, and I am no closer to a solution.

I know that there is probably some really obvious method to achieve my above stated goal. However, to paraphrase an old saying:
'But for the forest, I just cannot see a single tree.'.

Thanks in advance for any & all assistance.

Ciao,
Rina
<3

******************************************

[poisondeathray]

A LUT is a "look up table". It won't help you here. Unlikely that these types of manipulations will help you very much

If you want to do a good job with a clean matte, likely you will need to pull multiple keys + composite with some roto work

[dedlykme]

A LUT is a "look up table". It won't help you here. Unlikely that these types of manipulations will help you very much

If you want to do a good job with a clean matte, likely you will need to pull multiple keys + composite with some roto work

Yeah...., but....,
Awwww, gee..., shucks!!!

[poisondeathray]

The reason why you won't find the "possibilties" or your combo screenshot useful, is all those iterations are really representing the same thing. An inverted version of the matte (regardless of what color you want to use) represents the same data since you've binarized everything. Combining them in any weird or wonderful way won't help, because the separation (inclusion vs. exclusion) is the same for all of them.

And the screenshot for your "goal" actually has problems, and isn't what you "should" want - For example, the area between the right arm and body is obliterated. Ideally you would keep that separation, along with hair detail, etc....

If you post a sample clip, there might be some other manipulations you could try, but it's 99% certain that if what you described earlier is correct, you would need to pull multiple mattes, roto + composite

[dedlykme]

And the screenshot for your "goal" actually has problems, and isn't what you "should" want - For example, the area between the right arm and body is obliterated. Ideally you would keep that separation, along with hair detail, etc....

I just threw that supposed "goal" example together in GIMP in a few seconds, in order to provide a **very rough illustration** of what I'm
trying to accomplish.

I know that much more attention to detail/tweaking than that would actually be involved.

That said...., shucks!!!

Thanks,

Rina
<3

[aedipuss]

if it were an absolutely required shot and had to be done you could import it into photoshop and do frame by frame masking. are they paying you a million+? didn't think so. you're going to have to come up with something off beat like create fake animation where you take stills and create a gif like guitar strumming arm movement. or just pop him into and out of the video once in a while when the masking works ok. or go psychedelic? hard to come up with ideas without a sample.

[jagabo]

So you have videos that looks like:



And you want to merge them to make:



Something like this will work:

Overlay(v2,v1, mask=v1.mt_binarize())

If you still have the B/W video in the script you can use that as a mask instead of using mt_binarize(). Keep in mind that when using YUV video black is usually at 16 and white at 235. So you may need to perform a contrast stretch to get a mask with blacks at 0 and whites at 255.

[dedlykme]

Something like this will work:

Overlay(v2,v1, mask=v1.mt_binarize())

Thank you soooo much for your reply, jag!!!
I've learned through my long period of lurking here that your reputation precedes you as one of the true "gurus" around here.

So..., I was already able to achieve that same result using the good old trusty ColorKeyMask() method by doing something like this:

bg = m3 
fg = m6 
ckm = ColorKeyMask(fg, $000000, 5) 
Overlay(bg,fg, x=0, y=0, mask=ShowAlpha(ckm), mode="blend", opacity=1.0)

It gets me into the ballpark..., but..., the problem that I'm having is figuring out an effective way to overlay that type of result onto
the original source footage in such a way that it masks the old (lousy!) background, but allows the **entire** foreground subject
(the guy with the guitar) to come through. With what I have now, for example:
If I pull out the blue ($0000ff) it gets me close..., but I am left with a guitar player who has a nice bright green forehead & guitar.
If I pull out the green ($00ff00)..., well..., I have the same old, unusable original background and a really pretty blue smurf-guy!!

What I'm missing, I think, is some sort of interim step that creates a clip that (I guess) is "tri-nary" (Did I just make that word up?),
so that I can use that clip to create one that is two-color (binary) in that the background is one color and the foreground subject is another.
Perhaps some other overlay/layering "mode" or method that I am missing which, when two (or more) of the mask clips are combined,
it blends/merges them into a usable result. Even if it is an "interim" usable result.

So...., thank you again..., really!!

Ciao,

Rina
xo

P.S.

"if it were an absolutely required shot and had to be done you could import it into photoshop and do frame by frame masking..."

Yeah..., I'll get right on that. When do you think I'll finish frame number 4800? LOL.
Not a chance for the paltry $300 I'm getting!!
Sadly, though..., the footage from this shot comprises a good 60% of the video.

[poisondeathray]

You're looking to fill in holes in the matte, but only the "wanted" ones, for example you wouldn't want to fill in the right arm / body hole mentioned earlier as the foreground matte, because that is supposed to be background.

You wouldn't do it frame by frame in photoshop - you would do it in a compositor like After Effects which has dedicated tools for this - it interpolates between frames, so a roto mask might only take a few hours instead of days . Other tools like mask manipulation tools, motion tracking make it much much more effective for these sorts of tasks. Trust me - anybody who has done this sort of thing would use AE or NukeX as their 1st choice

But in general terms - what you would do (in any software, even avisynth) is use multiple masks, either using different settings, different keys, or isolate using different characteristics. A "color key" is really isolating by hue. But you have other characteristics you might use to cause separation and isolation. For example, set the parameters so only brighter areas are keyed out, then combine it with your other mask(s). A luma mask. I'm guessing his right side of his face was "brighter" or different lighting, thus the "holes" in the matte. You combine that with a different matte to your existing one to fill in the holes . It's easier to do in more specialized programs, but in avisynth you can still use functions to combine different masks. mt_hysteresis is a good one that will conbine parts of smaller masks into larger ones. MaskHS is a decent avisynth approach, at least for 1 or more of the masks, because you can isolate by hue range, saturation, set a min / max saturation level

But to be completely honest - color keying is terrible in avisynth. Anything of this sort that can be done in avisynth half decently woudl be done 10x better and faster in a program like AE . Avisynth is awesome for many things. This isn't one of them, and I'm a huge avisynth fan

Or maybe you have done several approaches, but the only one you have presented is 1 key in different colors. That's really like 1 mask or 1 data point.

If you post a sample, I can tell you what is "doable" with avisynth, and what will take a real compositing / keying program

[jagabo]

If you can come up with multiple mask that cover the areas you want you can add them together with Overlay().

newmask = Overlay(mask1, mask2, mode="add")

[dedlykme]

So...., I know it's been quite a while since I originally started this thread, but I thought I would return (somewhat after the fact)
to let everyone know how I finally resolved this issue just in case it is helpful to anyone else.

Not surprisingly, I was looking for a cost-free ($$$) way to achieve the goal.
Ultimately, following the wisdom of @poisondeathray, I started looking around for a decent, **and free** application
to do the required rotoscoping/masking/compositing in order to "re-key" the original footage.

Now..., I have alway thought of "Blender" as primarily being an application for...., you know....,
creating characters, etc. & animation for video games and, of course, the ubiquitous "Sintel" & "Big Buck Bunny" videos.
However, after continuing to stumble upon various YouTube video tutorials for Blender I decided to give it a try.

It appeared to be a little bit daunting/overwhelming at first but I was pleasantly surprised to find that, after viewing a couple more of
those YouTube video tutorials, the learning curve was really *not* all that steep.

Long story short....,
Using a few of its capabilities on the original **lousy** footage..., specifically rotoscope/masking combined with placing motion trackers
on various key locations of the primary subject (person), I was able to output re-keyed footage that was not only usable, but was (IMO)
quite **good**. And I was able to achieve the result very quickly...., within a couple hours of downloading the program.
I ended up liking "Blender" so much that I used it to do all of the required chromakeying
and compositing for the entire project.
I even used the included (and not much talked about) NLE in "Blender" to put together, and output,
the whole rest of the video.

So...., **thank you** @poisondeathray and @jagabo for all of your assistance.
And, I must say to anyone who reads this, "Blender" is definitely worth checking out for many of
your video needs.
Just dip your big toe into the "Blender" water...., you won't be disappointed.

Ciao all,

Rina
<3

[poisondeathray]

Yes blender is very good. If you are using it as a NLE with the video sequence editor, check out the plugins called "Blender Velvets" which makes the VSE easier and friendlier to use
(ignore the "glamorous" , sensationalized webpage, they are just simple functional plugins)
http://blendervelvets.org/


Alternative freeware , that is easier to use than blender for keying and masking and matte creation is cinegobs keyer. It does a lot more than just keying - you can do garbage mattes, bezier masking with the pen tool etc.. It has "tweening" or interpolation of keyframes, so the rotoscoping is half decent for freeware. It has avisynth import support too, and can export AVI or image formats directly (eg. png sequence). Blender is way more powerful, but at the same time, more difficult to learn and use

[dedlykme]

Thank you very much @poisondeathray for the tip on "Blender Velvets"...., I'll check it out right away.
(I am quickly becoming a huge Blender "Fan-Girl".)

I had previously played with Cinegobs Keyer, but I just couldn't seem to get it to do what I wanted.
(It was also during the "brain-numbed/cataracts" period described in my original post, so I should probably give it another go sometime.)

Thanks again,

Rina

[pandy]

This may help also:
http://www.avisynth.nl/users/vcmohan/WaterShed/Watershed.html

Not sure if there is any plugin with morphological filters (i mean erosion, dilation, opening, closing) - crude approximation can be downsample mask copy and later enlarge to previous size + stack with original one - should fill gaps... (weird idea but may work)

-edit--
seem VCMohan is the answer:
http://www.avisynth.nl/users/vcmohan/ColorIt/Morpher.html