Batch method to find images that contains only text !

[zyck]

Hello !

I'm using Videosubfinder to OCR subtitles from different vids and it creates images with text and trash images without any text.
And it's hard to filter manually images that contains text.

Is there any software that finds images that contains only text or a certain colour ?

[jagabo]

AviSynth can usually do that. But it has a steep learning curve. Upload a short sample of what you're working with.

[zyck]

I have uploaded an exemple of images from a show extracted with VideoSubFinder.
There many images that doenst contains any sort of text. I would like to separate or direct delete them.

[jagabo]

Sorry, I lost track of this thread and just ran across it when I was deleting old stuff on my drive. I don't know if it will be useful to you but AviSynth can pretty easily remove most of the non-text image from the video. Here's a sample of one of your images with most of the background removed:


[Attachment 61318 - Click to enlarge]

On the top is the original image, the bottom with the background removed. I picked this shot because it has those bright white lights in the background that interfere a bit with the subtitle text. Most shots won't have that problem. They'll be totally black other than the sub text. This should eliminate most of the false positives from SubFinder.

The idea would be to process the source video to produce a new video with only the subtitles. You could then use that new video as a basis for SubFinder, presumably eliminated all the false positives.

The script I used here only works for bright white subs with black outlines. It doesn't see the subs in 0_02_08_240__0_02_11_519.jpeg.

If you think this would be useful I can upload the script and describe how it works.

[zyck]

I think it would be useful and I will provide myself some methods that I have learned this days, and maybe it will get a 100% result of only text images, and less manually work.

[jagabo]

Here's the script I used to make the earlier images:

ImageSource("0_05_20_320__0_05_23_319.jpeg", start=0, end=23, fps=23.976) 
Crop(0,0,-0,-1)
ConvertToYV12()

edges = mt_edge(mode="roberts", thy1=40, thy2=40).mt_expand().mt_expand().mt_expand().mt_inpand().mt_inpand()
Overlay(BlankClip(last), last, mask=edges)
ColorYUV(off_y=-200).ColorYUV(gain_y=5000)

ImageSource() was used to import one of your JPEG images. You will use a different source filter to load your videos, probably LWlibavVideoSource(). And that will probably give you a YV12 video so it won't need the Crop() or ConvertToYV12().

The video needs to be in YV12 format but that requires the frame dimensions be even values. So I used Crop() to remove the bottom line, reducing the image from 1060x143 to 1060x142. Then converted to YV12.

I used mt_edge() to detect edges in the video:


[Attachment 61339 - Click to enlarge]

We need to fill in the outlines of the text to create a suitable mask. I used mt_expand() to do that. mt_expand() takes any white pixel and spreads the white to the adjacent 8 pixels around it:


[Attachment 61340 - Click to enlarge]

You can see that the single pixel of white above the y in Friday has become a 3x3 white dot. The letters aren't full filled yet so two more calls to mt_expand() gave:


[Attachment 61341 - Click to enlarge]

The insides are filled but the outsides have expanded more than necessary. So mt_inpand() (the opposite of mt_expand() was used twice to reduce the outer edges:


[Attachment 61342 - Click to enlarge]

That was then used as an alpha mask to overlay the original video over a black video (BlankClip()) resulting in:


[Attachment 61343 - Click to enlarge]

That leaves a little background junk around some of the text. Most of it was darker than the text so I turned everything darker than 200 into 0 (full black). That leaves the text very dark so I lighted what was left with a gain_y of 5000:


[Attachment 61344 - Click to enlarge]

When running this with the full video you would want to crop the frame down to just the subtitle area.

If your subfinder can open AviSynth scripts directly you can do that. Otherwise, you'll need to save the result of the script to create a new video to use with the subfinder. You can use very fast settings in the video encoder since all you have is a little white text on a black background.

[jagabo]

If you wanted to use your subfinder on the original video you could probably use AviSynth and ffmpeg in a batch file to delete most of the images that don't contain text...

[jagabo]

Here's a batch file that will delete all the jpeg images without text:

@echo off
setlocal enableextensions enabledelayedexpansion

for %%F in ("*.jpeg") do (
echo ImageSource^("%%~nxF", start=0, end=0^) > _test.avs
echo ConvertToYV24^(^) >>_test.avs
echo edges = mt_edge^(mode="roberts", thy1=40, thy2=40^).mt_expand^(^).mt_expand^(^).mt_expand^(^).mt_inpand^(^).mt_inpand^(^) >>_test.avs
echo brights = mt_binarize^(200^) >>_test.avs
echo Overlay^(edges, brights, mode="multiply"^).GreyScale^(^) >>_test.avs
echo WriteFileIf^(last, "_test.txt", "^(AverageLuma^(^)>1^)", "AverageLuma", append = false^) >>_test.avs
ffmpeg -loglevel fatal -i _test.avs -c copy -f null -
set /p AVERAGELUMA=Enter ID: <_test.txt >nul
set /a VAL=AVERAGELUMA
if !VAL! LSS 1 del "%%~nxF"
if exist _test.avs del _test.avs
if exist _test.txt del _test.txt
)

It builds an AviSynth script, runs it with ffmpeg, then deletes all the the images with an average brightness (after processing) less than or equal to 1.

Of the 840 images in your archive it deleted 419. The remaining 421 all had subtitles. If I change the code so it deletes all the files with subtitles it deletes 421, leaving 419. The only remaining files of the 419 with subtitles were the two with dimmer subs, 0_02_08_240__0_02_11_519.jpeg and 0_02_11_520__0_02_11_919.jpeg. It's not especially fast. It took about 2 minutes on my computer. I think the loading of AviSynth 840 times is the major time eater.

You'll need AviSynth+, the masktools2 filter for AviSynth, and ffmpeg.