Video editing: doubling part of the video

[yayo]

Hem... Hello!
I have a question related to video editing.

I'm used to record movies from TV every now and then.
Sadly one of the channels I get movies from, started past week to put annoying and totally useless scrolling text over the video (which is the title of the movie itself repeated 2 times after every commercials interruption, something that doesn't make sense at all: I'm supposed to know what's the movie I'm watching, and anyway I just need to press a button of the remote control to get the title! -_-; ).

I usually remove easily the commercials with Avidemux and the channel logo with a Virtualdub filter, but cannot find a way to get rid of that ugly scrolling text. Blurring it with a filter is still ugly to see.

I was wondering if there is a way (maybe with an avisynth script via Virtualdubmod?) to double (repeat) a part of the video taken from above or below and past it over the text.
(Maybe 2 stripes half size of the text, one taken from right above and another from below?)
Is it possible to do this with an avisynth script or something?
Perhaps it's not as good as getting rid of it completely, but maybe it's better than blurring it or covering it with a black box.

Hope you get what I mean. Sorry for my poor english.

(P.S. I didn't get time now to read all the rules of the forums, hope I didn't commit mistakes. )

[techiejustin]

Are you trying to remove scrolling text from your video?
Can you post a picture of what you are trying to remove?

[jagabo]

I was wondering if there is a way (maybe with an avisynth script via Virtualdubmod?) to double (repeat) a part of the video taken from above or below and past it over the text.
(Maybe 2 stripes half size of the text, one taken from right above and another from below?)

It's easy to do in AviSynth. Something like:

box = Crop(x, y1, boxwidth, halfheight) # box just above the scrolling text
Overlay(last, box, x,  y1+halfheight) # overlaid onto the top half of the scrolling text

box = Crop(x, y2, boxwidth, halfheight) # box just below the scrolling text
Overlay(last, box, x, y2-halfheight) # overlaid onto the bottom half of the scrolling text

It won't look great though.

[yayo]

Are you trying to remove scrolling text from your video?
Can you post a picture of what you are trying to remove?

Sure. I saved a short clip today, and then this screenshot from it. Here it is.



The channel logo was on the right side and it's already fixed as you can see.

It's easy to do in AviSynth. Something like:

box = Crop(x, y1, boxwidth, halfheight) # box just above the scrolling text
Overlay(last, box, x,  y1+halfheight) # overlaid onto the top half of the scrolling text

box = Crop(x, y2, boxwidth, halfheight) # box just below the scrolling text
Overlay(last, box, x, y2-halfheight) # overlaid onto the bottom half of the scrolling text

It won't look great though.

Yeah, I know it. But I also understood that thare's no way to get rid of such text as it is for the channel logo. Maybe some sort of very scientific ubercomplex filter which replicates the video from inbetween and inside the letters would make a better result (still supposing it'd be possible), but I never found anything like that. And by the way it'd surely be very expensive. :P
I just hope this way will give me something slightly better than other ways I tried. :/

Duh! It's a such short piece of code! And I got mad yesterday trying to understand something of avisynth language!

Well, I'll try it. Thanks a lot for the reply to both of you.
If someone has other ideas they'll be appreciated.

[jagabo]

Keep in mind that it's pseudo code -- you have to fill in values for the x, y, boxwidth, halfheight, etc.

[jagabo]

An example:

ImageSource("attachment.jpg") 
ConvertToYV12()
BilinearResize(720,576) # assuming source is 720x576

box = Crop(0, 476, 720, 20).FlipVertical()
Overlay(last, box, 0,  496)

box = Crop(0, 536, 720, 20).FlipVertical()
Overlay(last, box, 0, 516)

Note, I didn't pay any attention to the fact that your video is interlaced. The exact numbers and the FlipVertical() may not work properly for interlaced video. You may have to fine tune it.

[yayo]

Hehe. Yeah, I know it's pseudocode of course.

I now just have to understand how to get Virtualdubmod to work. I read it should give me an "open file via avisynth" option which virtualdub has not, but it doesn't. ?? :/
Perhaps without Virtualdubmod (even if the embedded editor would be handy), and just making instead a chain with Avisynth and normal Virtualdub or something. I only need to figure out how it works.

And also how to process only the parts of the video with scrolling text, to avoid damaging it all.

But I'll try later, or maybe tomorrow. Don't feel very good today... Gh.
Thanks again in the mainwhile.

[jagabo]

In VirtualDub or VirtualDubMod just use File -> Open Video File to open an AviSynth script. Or drag/drop the script onto the program.

In AviSynth you can use Trim() to break a video into pieces then just add them together at the end:

WhateverSource()
part1 = Trim(0,999) # frames 0 to 999
part2 = Trim(1000,0) # frames 1000 to "the end"

part1 = part1.filter() # do some kind of filtering on part1

return(part1+part2)

Another way to approach this is to use ReplaceFramesSimple to replace frames in one video with those from another video:

WhateverSource()

filtered = filter() # a filtered version of the video

ReplaceFramesSimple(last, filtered, Mappings="[0 999]")

The second script does the same thing as the first. ReplaceFramesSimple() is part of the RemapFrames package:
http://www.avisynth.nl/users/stickboy/

One thing that often confuses people new to AviSynth: if you don't specify a name for a clip the name "last" is assumed. So the first script is the same as:

last = WhateverSource()
part1 = Trim(last, 0,999) # frames 0 to 999
part2 = Trim(last, 1000,0) # frames 1000 to "the end"

part1 = part1.filter() # do some kind of filtering on part1

return(part1+part2)

The second script is the same as:

last = WhateverSource()

filtered = filter(last) # a filtered version of the video

last = ReplaceFramesSimple(last, filtered, Mappings="[0 999]")
return(last)

[yayo]

Ok, I finally managed to get it working.
(By the way, that clip was not interlaced at all, only the scrolling text was, don't ask me why. :P )

Now I'll play a bit with the code searching for the best result. The avisynth documentation is someway poor in examples, but I guess I'll find some more on the web.

Thanks jagabo.

P.S.
Since you helped me, maybe I can contribute to the community too... (?)

Sometime ago I wrote a little set of AutoIt scripts to automate the process of audio-resync of DVDs.

Provided the amount of delay to apply for each movie, it sequentially calls pgcdemux, delaycut if needed, muxman and then vobblanker.
It's based on my personal standard of DVD ( :P ), which is 1 or 2 audio tracks (italian + english), for a maximum of 3 movies.
Also it uses my software paths. But it shouldn't be so hard to fix according to each one purposes.

Perhaps someone can find it useful? :/

[yayo]

Now that I finally understood something of the avisynth scripting language I tried making a filter to remove the text, but the result is still a bit messy...

[jagabo]

Script?

[yayo]

Wops, you're right, I forgot the code. sorry. :P

Here it is.

It basically tries to make an alpha mask out of the text, to cut it out and then recover as much as possible from the background. It uses 5 subsequent frames.

Sadly in this case the letters are interlaced and with grey/black pixels on the border, which doesn't help at all.

Also this works for a single frame. (I'd need a for/next construct or something to process a full sequence. -_- )

Anyway, if you have ideas for improvement you're welcome. :d

DirectShowSource("f:\tomahawkscurediguerra_2.avi")

frame1=Trim(52750,52750)
frame1=Crop(frame1, 8, 496, 400, 40)
frame2=Trim(52751,52751)
frame2=Crop(frame2, 8, 496, 400, 40)
frame3=Trim(52752,52752)
frame3=Crop(frame3, 8, 496, 400, 40)
frame4=Trim(52753,52753)
frame4=Crop(frame4, 8, 496, 400, 40)
frame5=Trim(52754,52754)
frame5=Crop(frame5, 8, 496, 400, 40)
frame=Trim(52752,52752)

# attempt to create an alpha mask with the scrolling text
Function mask( clip c) {
  A=greyscale(c).tweak(bright=-128).tweak(cont=10).Levels(254, 1, 255, 0, 255)
  B=A
  B=overlay(B,A,-1,-1,mask=A)
  B=overlay(B,A,0,-1,mask=A)
  B=overlay(B,A,1,-1,mask=A)
  B=overlay(B,A,-1,0,mask=A)
  B=overlay(B,A,1,0,mask=A)
  B=overlay(B,A,-1,1,mask=A)
  B=overlay(B,A,0,1,mask=A)
  B=overlay(B,A,1,1,mask=A)
  B=tweak(B,bright=-125).tweak(cont=10).Levels(254, 1, 255, 0, 255)
  return(B.invert())
}

# make alpha masks from 5 frames, takes the part of the images without the text, and paste them all together
Function fix(clip a, clip b, clip c,clip d,clip e){
return(c.overlay(a,0,0,mask=mask(a)).overlay(b,0,0,mask=mask(b)).overlay(d,0,0,mask=mask(d)).overlay(e,0,0,mask=mask(e)))
}


frame3=crop(fix(frame1,frame2,frame3,frame4,frame5),0,0,400,40)
return(overlay(frame,frame3,8,496))

[poisondeathray]

Sadly in this case the letters are interlaced and with grey/black pixels on the border, which doesn't help at all.

You should be able to "grow" or expand the mask using masktools to extend over the black borders of the text

Also this works for a single frame. (I'd need a for/next construct or something to process a full sequence. -_- )

You should be able to isolate by high luminance values (ie. a luma mask) for a motion sequence , since the text is primarily "white" . Then use various logo removal methods /plugins with that mask . The problem is in your 1st screenshot, there are some "bright" grass areas, so separation might not be complete with a luma mask alone



Can you post a sample of the clip

[poisondeathray]

Maybe some sort of very scientific ubercomplex filter which replicates the video from inbetween and inside the letters would make a better result (still supposing it'd be possible), but I never found anything like that. And by the way it'd surely be very expensive.

The generic method is called "inpainting" .
http://en.wikipedia.org/wiki/Inpainting

Algorithms attempt to "predict" the patterns in the region of interest . If you've heard of photoshop's "context aware fill ", it's basically the same thing . It's not perfect, there are situations where the prediction fails, but others where it looks almost perfect

There are various logo removal filters/plugins for avisynth that use inpainting . But some of the logo removers will only accept a single bmp (no good for an animated sequence), but others will accept a clip e.g. exinpaint will. Because the mask clip is generated on a sequence (mask changes over time) , and exinpaint works with the mask sequence, theoretically it should work ok. But you might still encounter the "usual" problems with inpainting as the logo removing/covering up method (they usually work better on predictable repeating patterns like grass , but can fail miserably on certain backgrounds)

e.g. using lumamask and exinpaint

[yayo]

Wops.. sorry for late reply. :P

You should be able to isolate by high luminance values (ie. a luma mask) for a motion sequence , since the text is primarily "white" . Then use various logo removal methods /plugins with that mask . The problem is in your 1st screenshot, there are some "bright" grass areas, so separation might not be complete with a luma mask alone

Can you post a sample of the clip

Sure. A short sample should be attached to this post.

And yeah, making a mask out of white would work only where background is at least slightly darker. But it would probably fail anytime a whatever kind of white object moves under the text. Sadly implementing more algorythms to get a more effective way to isolate the text would be too much complex for me. :P

The generic method is called "inpainting" .
http://en.wikipedia.org/wiki/Inpainting

Algorithms attempt to "predict" the patterns in the region of interest . If you've heard of photoshop's "context aware fill ", it's basically the same thing . It's not perfect, there are situations where the prediction fails, but others where it looks almost perfect

There are various logo removal filters/plugins for avisynth that use inpainting . But some of the logo removers will only accept a single bmp (no good for an animated sequence), but others will accept a clip e.g. exinpaint will. Because the mask clip is generated on a sequence (mask changes over time) , and exinpaint works with the mask sequence, theoretically it should work ok. But you might still encounter the "usual" problems with inpainting as the logo removing/covering up method (they usually work better on predictable repeating patterns like grass , but can fail miserably on certain backgrounds)

Uh. Thanks for the link. I was pretty sure there already was something like this, but didn't know it was named inpainting.
Didn't know even about this "exinpaint". I'll try it out. Maybe it'll result useful to fix some movies I really want to save.

Anyway, about my little code, I was just playing out a bit, for learning/experimenting purposes. I was not planning to use it on my video. It's all but functional. And it's too complex for me to write something better than this. Also I have no time nor patience to do it. :/
I guess from now on I'll just avoid to record stuff from channels which uses this boring scolling text. Or maybe only movies which I really want and that I'm sure I wouldn't find somewhere else (...movies that I'll probably just keep as they are, to avoid making a mess out of them.) Duh. :/

Thanks again. ^^

[jagabo]

If you use inpainting on moving letters you'll still be able to read the text.

[poisondeathray]

If you use inpainting on moving letters you'll still be able to read the text.

Well you shouldn't be able to "read" it - at least not legibly ; but I get what you're trying to say - because inpainting is a spatial process, often there are aritfact variances between frames in addition to the errors with inpainting in the first place

[jagabo]

Here's an example using something like the OP's original idea:

 # grad.pnt is a 560x40 image with a gradient from black at the top to white at the bottom
grad = ImageSource("grad.png").ConvertToYV12(matrix="PC.601")

src = ffVideoSource("sample.avi") # the source video

topovr = Crop(src,0,456,560,40).FlipVertical() # just above the text area
botovr = Crop(src, 0,536,560,40).FlipVertical() # just below the text area
ovr = mt_merge(topovr, botovr, grad, false) # blend them together via the gradient

filtered = Overlay(src, ovr, 0, 496) # overlay the filtered block onto the original video

ReplaceFramesSimple(src, filtered, Mappings="[30 330]") # apply to frames 30 to 330 only

[poisondeathray]

You can see in mask_compare.mp4 , that using the luma mask method only, it does an ok job of isolating the text but misses a couple spots, and mistakes other spots that shouldn't be included (e.g .white parts on the horse). You can play with the settings to get a better mask. For a more precise mask you would have to combine methods to refine the mask, or use another technique altogether

In exinpaint.mp4, using a similar luma mask, but slightly expanded (you need more area for inpainting to derive textures from) , it looks ok in some sections but fails miserably in others (especially near the end).

If you were so inclined, you could combine several methods and masks to improve the results , maybe using certain methods only on certain frames


Luma Mask Compare

AVISource()
crop(0,492,0,-40,true)
orig=last

lumamask(orig.converttoyv12, b=220,w=255, show=true)
tweak(cont=8, coring=false)
mt_expand(u=-128,v=-128).mt_expand(u=-128,v=-128)
mymask=last

stackvertical(orig, mymask)




function LumaMask(clip filtered, clip raw, int "b", int "w", bool "upper", bool "show"){

raw      = default(raw, filtered)

LO       = string(default(b, 24))
HI       = string(default(w, 48))
upper    = default(upper, false)
show     = default(show,  false)


code     = upper ? "x "+LO+" < 255 x "+HI+" > 0 255 x "+LO+" - 255 "+HI+" "+LO+" - / * - ? ?" : \
                   "x "+LO+" < 0   x "+HI+" > 255 0 x "+LO+" - 255 "+LO+" "+HI+" - / * - ? ?"

msk      = raw.mt_lut(code,u=-128,v=-128)

show  ?  msk : mt_merge(raw, filtered, msk, luma=true, U=3,V=3)}

Exinpaint

AVISource()
crop(16,0,-20,0,true)
orig=last

b=blankclip(orig)

lumamask(orig.converttoyv12, b=225,w=255, show=true)
tweak(cont=8, coring=false)
mt_expand(u=-128,v=-128).mt_expand(u=-128,v=-128).mt_expand(u=-128,v=-128).mt_expand(u=-128,v=-128).mt_expand(u=-128,v=-128).mt_expand(u=-128,v=-128)
converttorgb24()
mymask=last

mymaskfull=overlay(b, mymask.crop(0,490,0,-38,true), y=490)

ex=ExInpaint(orig.converttorgb24(),mask=mymaskfull.converttorgb24().blur(1,1), xsize=4, ysize=4)

ex
converttoyv12()
deblock_qed
blur(1,1)
ex2=last

overlay(ex, ex2, mask=mymaskfull.blur(1,1))

addborders(16,0,20,0)