Removing logos with content-aware fill

[Mephesto]

PDR, you said Adobe didn't support this for video and that I would have to do a batch job on photoshop.

I'm dealing with a different video right now where I would be willing to try it out.

I looked around Photoshop CS6's interface but the batch and scripting features didn't support automating content-aware fill.

How do you pull this off?

[poisondeathray]

I said AE doesn't have content aware fill, so you would have to use photoshop

eg.
http://blog.motionboutique.com/content-aware-fill-script-210

IMO, PS's content aware fill only works well on certain scenarios and pattern types. Certain backgrounds don't "fill" very well

But you can use other tools and approaches as well , like batching the heal brush
e.g
http://vimeo.com/49017666


Often dedicated logo removal tools (e.g. avisynth rm_logo) might work better . It really depends on what you're trying to remove/cover up/replace

[budwzr]

I can't believe that in the motionboutique link above, they needed to use motion tracking, when a simple cookie cutter mask offset a few pixels. against a second copy on the track below would do the same job. The ship's name is tiny, and the ship's motion is close to zero.

People get hung up on specific exotic tools and techniques, and forget about their own brain. What if there's no "turd" remover for that dog park video?

The secondary color corrector, running in a travelling matte, would work well too.

Hahaha.

[poisondeathray]

I can't believe that in the motionboutique link above, they needed to use motion tracking, when a simple cookie cutter mask offset a few pixels. against a second copy on the track below would do the same job. The ship's name is tiny, and the ship's motion is close to zero.

People get hung up on specific exotic tools and techniques, and forget about their own brain. What if there's no "turd" remover for that dog park video?

The secondary color corrector, running in a travelling matte, would work well too.

Hahaha.

Haha easy tiger... sorry to burst your "ballons", but they didn't need to do it that way ... I agree that one wasn't a good example , and you wouldn't need to use a motion tracker for removing a static logo or something stationary like a dust spot anyway. It's when you have an animated logo where linking to motion data might be more useful

The point was to use content aware fill or similar methods in a PS batch action. It's the process / technique that he was asking about - it's described more in the 2nd link


Now maybe you can help me with removing this logo:


(I couldn't do much the original avatar it was too tiny, but popping your balloons is payback for killing off hulky)

[budwzr]

Hey, that's cool. Maya? What, no fireworks? > "Haha easy tiger..." < hahaha.

I can't come back on that one. My tools are stone age. I'm debating on a Wacom Intuos 5 tablet for Chrizm. OR, perhaps a nice fluid head from Manfrotto, hehehe!

Are you snowed in yet?

I remember the heady old days when somebody would come in with a project, like that 2001 Odyssey one, remember? That was fun, and we got to sharpen our fangs.

And those Appleheads disappeared too. What gives?

================================================== ========
We need gradient fills for text. For holiday cheer.

[poisondeathray]

Hey, that's cool. Maya? What, no fireworks? > "Haha easy tiger..." < hahaha.

I can't come back on that one. My tools are stone age. I'm debating on a Wacom Intuos 5 tablet for Chrizm. OR, perhaps a nice fluid head from Manfrotto, hehehe!

Are you snowed in yet?

I remember the heady old days when somebody would come in with a project, like that 2001 Odyssey one, remember? That was fun, and we got to sharpen our fangs.

And those Appleheads disappeared too. What gives?

================================================== ========
We need gradient fills for text. For holiday cheer.

Not Maya, C4D

Yes snow

Either the Appleheads are too busy waiting in line for the next ithingy , or you scared them off





But on more on topic:

What I would like to see is a temporal content aware fill that looks at other frames .

PS's only looks at spatial data (ie. the current frame.) So when you have certain patterns, or data on edges it can make mistakes

Other tools like Mocha Pro remove objects by tracking foward/backward frames and "pasting" in data from adjacent frames, so it looks at temporal data, but it's not content aware fill (better in some cases, worse in others)

[budwzr]

Those audio people talk about out-of-phase channels cancel each other out, or something like that. I wonder if there's any phenomenon like that in video?

[Mephesto]

Damn I typed out a message but it got lost. Short story: this failed horribly like I knew it would. The area filled was jittery.

The logo is not opaque though. Here's a screenshot. What logo removing tool should I use?

[poisondeathray]

not sure

have you tried any of the "usual suspects" ? rm_logo, inpaintfunc, delogo, x-logo, msu logo remover etc...

I usually don't remove logos - they don't bother me - but if you post a sample clip you might get better / specific suggestions. Some people remove logos all the time

[Cornucopia]

Those audio people talk about out-of-phase channels cancel each other out, or something like that. I wonder if there's any phenomenon like that in video?

Not the same. Sound has both positive and negative excursions away from ZERO (compression & rarefaction of the air). Light has just a positive luminance amplitude (well, not counting polarization, which our eyes are insensitive to anyway).

You could combine pictures with compositing functions to do a similar thing, but the result would usually be Medium Gray, not Black. Multiple layers of compositing could probably get you what you are looking for, but it's much more complicated to set up correctly (IOW, not worth the effort).

Scott

[budwzr]

Scott, what I was pondering, since yesterday, is if dodge and burn tools, combined with extremely tight masks, like the logo would have to be traced in a vector program, and black separated from white, and also accompanied with the appropriate composite levels.

What you're saying really encourages me, because I already came to the conclusion that the result would be grey, unless certain composite operators were used. i.e. add, subtract, lighten, difference, etc.

In this case, the logo is black and white with about 50% transparency. I can't get a good trace on the sample, but I'm gonna fool around with the idea.





What if a mask could neutralize the logo, and dodge/burn could bring out the underlying 50%?

[budwzr]

I think I have a workflow now.

Start with the offensive logo. Note that it's the typical 50% white transparency type.


Select the logo using color tool in additive/contiguous mode, and fill it with white


Duplicate the selection onto a new layer


Composite with these dodge/burn settings, and voila!


This method can be translated into video tools, and should act as a "content-aware" solution. The matte must fit the logo exactly to work, so bring it in as a PNG with transalpha.

[poisondeathray]

Looks like a good start

But as Scott suggested above you have to do at least 2 composites ; 1 for "white" and 1 for "black" . Most likely you'll need a different operator for the "black" scenario

Look at his logo, it is a black and white logo with adjusted opacity (ie. it's not a white-only logo)

[budwzr]

Yeah, right Scotty, we need to reverse polarity of the dylithium crystals, right?

I was thinking TWO mattes, yes. But this thing might be huge, so I need to file a patent first. Hey, wait a minute, by posting it here I'm hereby stating that Patent Is Pending.

[budwzr]

Here's the black parts stroked with a dodge/burn brush with different settings from the white example.

Obviously, additive/subtractive compositing does indeed work. The tedious part is getting a good matte. The sample was too pixelated even after upsampling, but it could be done.

Notice the differences in the brush settings, and make your mattes accordingly.



I'm assuming that the brightness/contrast settings will reveal the correct colorations thru temporal space, but that's the last unknown. hahaha, what if it falls apart when the scene changes?

[poisondeathray]

I'm assuming that the brightness/contrast settings will reveal the correct colorations thru temporal space, but that's the last unknown. hahaha, what if it falls apart?

That's why you need to test a video sample, because areas where an individual channel clips won't look so good- it will "fall apart"

[budwzr]

Well, the OP was looking for a logo removal tool, and this is quite a process just to clean up something recorded from a teevee. Like a $10 haircut on a 50 cent head.

At least I proved the concept. This could be handy for a lot of other things too. A while back I made the statement that a flattened alpha channel logo couldn't be "reversed", but it seems it can be neutralized, yeah?

[Mephesto]

Pixelated? This is a 720p source. It doesn't get better than this.

It's been sooo damn long since I did logo removing. Last time was around 2008 and the most impressive results I've achieved was alpha-blended logos with this filter albeit with a lot of (tolerable) artifacts.

I've given up logo removing because it was obvious it didn't work and MSU told me they weren't gonna work on a better, motion-compensated one for a long time.

I've noticed they finally did release a new version recently but I couldn't get it to work. I'm getting a black output on the second (removal) pass. The manual even said for some reason that the first 1800 frames will be blank but only if you're removing animated logos. I even tried to work on 2500 frames to see if the last 700 would not be blank. They still were.

Here's a video sample of the logo.

[Mephesto]

So I tried to isolate the logo, which I did, and then subtract but this isn't working. The chroma shifts depending on the scene. No idea why. I tried shifting the brightness of the logo but this seems to have more to do with contrast. Ugh.

Ridiculous.

[jagabo]

The alpha blended logo is added to the base image (except when the logo fades in) with a simple algorithm like:

 output = (original+logo) / 2

You can generate the logo by subtracting a clear frame from a logo'd frame.

DirectShowSource("s14e6.mp4").ConvertToRGB()
Merge(SelectEven(), SelectOdd()) # temporal filter for noise reduction
Merge(SelectEven(), SelectOdd()) # temporal filter for noise reduction
Subtract(Trim(last,10,0), last)



Cleaned up and contrast stretched:



Hand edited to make an alpha mask:



Sanity check, frame 0:

DirectShowSource("s14e6.mp4").ConvertToRGB()
ovr=ImageSource("logo.png")
mask=ImageSource("alpha.png")
Overlay(last, ovr, 0, 0, mask, 0.5)

very close to frame 50 of the original video:



Once you have the logo you can reverse the process.

output * 2 - logo  = original

I'll work on that later...

[Mephesto]

Yeah, that's what I did. I subtracted frame 50 from 0 since it was the same scene with and without the logo, then I saved the subtracted image, editted accordingly and then subtracted the whole video with the subtraction containing the isolated logo. It worked perfectly only on that specific scene. It screwed up on the next. The chroma seems to be non-linear. If I subtracted from a scene with an orange background, the isolated logo would have an orange tint (blue actually since it's the opposite color).

I just ran grayscale() on the logo I isolated from a scene with a uniform orange background by photoshopping the logo out and subtracting and now I have a more neutral logo but the contrast is still off. The result is what you see in the screenshot of the guy in blue pants.

[budwzr]

The second logo in jagabo's post is the holy grail. The "hand edited" one. Very nice!

[Mephesto]

To me it looks like a demented Disney yuppy/cop blowing out smoke in the shape of the letters "HD".
Is this supposed to say something about my subconcious? Rorschach test FTW.

[budwzr]

This is what I was thinking.

[poisondeathray]

Once you have the logo you can reverse the process.

output * 2 - logo  = original

I'll work on that later...

In theory this should work. I think the problem is 8bit avisynth is clipping values when you do the output*2 step (or you can use "add" as an operator to itself) . values > 255 for R,B,G will clip to 255 . Unless you know away around this ?

The "black" overlaid areas are returned to almost normal , but I think the "white" overlaid areas were clipped in the calculation. I think the "almost normal" discrepancies in the "black" overlaid areas might have to do with avisynth using YUY2 internally for Overlay() - I remember gavino saying something about that . Using pc_range=true doesn't help . Or it might just be rounding errors. I think Layer() might be able to work in RGB natively, but I can't find operators like add, subtract . Or maybe this can all be done in masktools in YUV ? They have all these polish notation and operators, I'm not too good with the programming side...

e.g.
If the original "red " patch was RGB 180,17,17
The White overlay areas on the red patch is RGB 216,134,134
The Black overlay areas on the red patch is RGB 90,8,8

A 134*2 step will give 268
A 216*2 step will give 432

bars=colorbars(480,480,"RGB32")
logo=imagesource("logo.png",pixel_type="RGB32").bicubicresize(480,480)

Overlay(bars, logo, mask=logo.ShowAlpha(), opacity=0.5, pc_range=true)
#logo
output=last

overlay(output,output, mode="add", pc_range=true)
output2=last

overlay(output2,logo, mode="subtract", mask=logo.ShowAlpha(), pc_range=true)

(I didn't limit the "fix" with a mask, so ignore the outside areas)

[jagabo]

Once you have the logo you can reverse the process.

output * 2 - logo  = original

I'll work on that later...

In theory this should work. I think the problem is 8bit avisynth is clipping values when you do the output*2 step

You would do it like this:

 (output - logo/2) * 2 = original

I think you can also use masktools which, if I remember correctly, works with more than 8 bit internally.

By the way, VirtualDub's Delogo tool has a deblend mode for just this type of logo.

[poisondeathray]

Once you have the logo you can reverse the process.

output * 2 - logo  = original

I'll work on that later...

In theory this should work. I think the problem is 8bit avisynth is clipping values when you do the output*2 step

You would do it like this:

 (output - logo/2) * 2 = original

I think you can also use masktools which, if I remember correctly, works with more than 8 bit internally.

I cheated "*2" by adding to itself . I don't know how to do a pixel mulitplier, and my masktools knowledge is rudimentary - do you want to see if you can figure it out ?

By the way, VirtualDub's Delogo tool has a deblend mode for just this type of logo.

Yes, I suggested that above in post #9 . Unobtrusive logo's like his I usually don't mind . If it was like the one I made....that's a different story