How to find the tags on a video.

[bosyk]

Hello Dear forum users. I have a question. Is it possible to find marks on the video. I know that there are marks in several places on the video with the length of one frame of the video. I know how the mark looks like, it is a set of a few numbers and I know them. Is it possible to find them, any program or script.

[poisondeathray]

Do they ever change location ?

Do they ever change appearance /color ? Opaque or semi-transparent ?

How large ?

[jagabo]

It's best if you provide some sample videos. Or at least images.

[bosyk]

The label can be in any place, the color can also be any color. The video marks 0:02:36.05

[jagabo]

AviSynth's RemoveDirt() or RemoveDirtMC() can remove that particular mark. Here's the same frame after RemoveDirt(5,false)


[Attachment 68684 - Click to enlarge]

If you're only looking at finding them (for example printing a list of frames that have a mark) one can do that too. It may not be 100 percent accurate though (some false positives and some false negatives). And it may not work if the mark appears right over a moving part of the picture (the man's hands, for example).

[bosyk]

I need a way to find these tags . Here is the tags I was looking for looking at each frame to show which tags I need to find.

[bosyk]

If you're only looking at finding them (for example printing a list of frames that have a mark) one can do that too.

Can you tell me in more detail how to do this.

[jagabo]

Here's an AviSynth+ script that finds the frame in the sample video:

source = LSMASHVideoSource("tags.mp4") # mark on frame 4685
source = source.Blur(1.0)

RemoveDirtMC(source, 10, false)

mt_lutxy(source, last, mt_polish("abs(x-y)"), chroma="-128")
ColorYUV(gain_y=1000, off_y=-100)

WriteFileIf(last, "match.txt", "AverageLuma>0.010", "current_frame", """ " : " """, "AverageLuma", append=false)
ScriptClip(last, "Subtitle(String(AverageLuma))")

prefetch(4)

It gets the source video and blurs it to reduce noise. It then uses uses RemoveDirtMC to remove "dirt" that appears for only one frame, including the mark you are trying to detect. The difference between the cleaned frame and the original blurred frame is computed with mt_lutxy. That difference is amplified (and small differences discarded) with ColorYUV. If the average luma of the resulting difference stream exceeds a threshold (0.010) the frame number (and the average difference) is written to a file called match.txt. Finally the difference value is written to the video for visual reference. the output file looks like:

102 : 0.014081
4685 : 0.037033
17445 : 0.010202

Here it found three frames where something was removed by RemoveDirtMC: frames 102, 4685, and 17445. If I had used a higher value, say 0.030, it would have found only the one frame with the mark we are looking for. But sometimes the cleaned portion of the frame may not stand out as much (depending on the brightness and color of the mark and background. It will also vary depending on the size of the mark (if it ever varies). So I used a lower threshold, assuming a few false positives were preferable to not finding anything.

You can run the script by opening it in VirtualDub (File -> Open Video File) and selecting File -> Run Video Analysis Pass. Or drag/drop it onto a batch file that uses ffmpeg to read every frame:

time /T
ffmpeg -loglevel fatal -i %1 -c copy -f null -
time /T
pause

The script took about 15 minutes to complete on my computer. You could make it faster by reducing the frame size at the start of the script. But you will also need to change the RemoveDirt strength and the difference threshold. A quick test shows that it took about 5 minutes and half width and height.

[bosyk]

Thank you very much. It's a bit complicated for me, but we'll study and figure it out.

[bosyk]

@jagabo
I couldn't figure it out. Can you show in more detail, step by step, the steps to find and remove tags.
Thank you in advance.

[jagabo]

How far did you get?

[bosyk]

Installed all programs. The problem was writing a script to process AviSynth correctly.

[jagabo]

You can use Notepad or any other plain text editor to write the scripts. Copy and paste the script from post #8. Save in the same folder as the video with .avs as the extension (not .txt). Use VirtualDub2 to open the avs file.

[bosyk]

As I understand it, you used this script knowing the location of one mark:

source = LSMASHVideoSource("tags.mp4") # mark on frame 4685
source = source.Blur(1.0)

What if the location is not known?
When I tried to paste this code I got an error:

Avisynth open failure:
Script error: There is no fundion named
'LSMASHVideoSource'.
(C:\Users\bosyk\0esktop\dir.avs, line 1)

[jagabo]

As I understand it, you used this script knowing the location of one mark:

source = LSMASHVideoSource("tags.mp4") # mark on frame 4685
source = source.Blur(1.0)

What if the location is not known?

That "# mark on frame 4685" is just a note for our benefit. It's not telling AviSynth where the mark is. I put it there so I would remember where the mark is to verify the script is working properly.

When I tried to paste this code I got an error:

Avisynth open failure:
Script error: There is no fundion named
'LSMASHVideoSource'.
(C:\Users\bosyk\0esktop\dir.avs, line 1)

You need to install the LSMASH source package:

http://avisynth.nl/index.php/LSMASHSource
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases/download/1129.0....r1129.0.1.0.7z

Put LSMASHSource.dll from the x86 folder in AviSynth+'s plugins+ folder. Put LSMASHSource.dll from the x64 folder in AviSynth+'s plugins64+ folder.

You will also need the MaskTools2 package:

http://avisynth.nl/index.php/Masktools2
https://github.com/pinterf/masktools/releases/download/2.2.30/masktools2_v2.2.30.7z

Like with LSMASH, put the dll files in their respective folders.

And you will need the RemoveDirtMC() function. There are many variations of this filter. The one I used is attached. Unzip the .avsi file and put a copy in each of the two plugins folders. It was derived from this post:

https://forum.videohelp.com/threads/382527-Need-help-setting-up-avisynth-pls%21?p=2477...=1#post2477423

In turn, RemoveDirtMC requires several other third party filters. You will get similar error messages. Find, download, and install them as above. Yes, getting all this set up is a pain.

[bosyk]

Sorry to bother you, but now a new error is popping up. I insert any script
AVI Import Filter errorUnknown)(80040154)
I forgot to write, I have Win7 64bit.

[jagabo]

Sorry to bother you, but now a new error is popping up. I insert any script
AVI Import Filter errorUnknown)(80040154)
I forgot to write, I have Win7 64bit.

I don't know exactly what's wrong there. If you search for that error code in these forums you'll find it mentioned several times. Some of the posters said uninstalling Avisynth then re-installing it fixed the issue. Did you install 64 bit AviSynth+? Or 32 bit AviSynth+? Or an older version of AviSynth (not plus). I recommend using 64 bit AviSynth+.

[jagabo]

Also, try running Avs Info Tool to check your installation. Note any warnings or errors that it gives.