How to search one's hard drive for a video ?

[Sebastian42]

I have a snapshot of a frame in a video, and want to use it to locate the video source on my hard drive.

[sophisticles]

Not going to happen.

[Sebastian42]

I can only hope that someone else thinks differently. There are tantalising hints of the possibility in the cybersphere; in fact one thread claimed that commercial programs had been found, but 'he' wanted a free one. Sadly, 'he' did not mention which paid-for items filled the bill.

[lingyi]

Give Video Comparer a try. There's a limited function free version. Make the snapshot into a repeating short video and it may find the video in thorough mode. I don't know how accurate it is, but I use the Pro version and it finds dupe scenes of a few seconds.

[pm-s]

Same problem happened to me few years ago. I had to watch every single video I saved until u found the right one

[Sebastian42]

Lingyi - I have tried the Video Comparer, and am at a loss as to how to select the snippet to find its 'long' video source. I seem to be able to only select folders. There does not appear to be a useful HELP file to get me started. Can you help ?

[lingyi]

AFAIK, it's not possible to select individual videos to compare. Put your snippet on the drive you're searching and let it run. It will search all the folders for a match. Be patient as it can take a long time perform the search, especially in thorough mode.

[Sebastian42]

The procedure you describe is obvious. The snippet IS already on the drive - but how do I identify the snippet to COMPARE, and then how do I tell Compare which drive/folder to search in ? That is a TWO step process and I have NO idea how to start it.

[lingyi]

I'm not sure what the confusion is. As shown on this page https://www.video-comparer.com/product-screenshots.php, you choose the directory/folders on the left that has the videos you want to compare. You don't choose the individual videos, only the entire directory or folder.

Anyway, I just checked and did a test and it probably won't work for you. The minimum duplicate length is 10 seconds. Settings>Advanced. I thought it was shorter because it will tag multiple scenes as dupes. I ran a test with shows that have preview snippets of the next episode and it didn't tag them. I also tried comparing an alternative version of a scene from a TV show and it didn't tag that either.

[Sebastian42]

Three things have happened.
Once you mentioned comparing videos rather than searching for a video from a snapshot, I realised that in the particular need I have RIGHT NOW, I have a long enough segment of the complete video to look for the complete one.
I decided to look for a user guide and found one - WITH AN EMAIL address for technical support. I also saw that the basic function is to look for ANY duplicates on a drive, as distinct from finding a particular one that matches another.
I have sent off an email asking the 'publishers' if their product can be made to do what I want - else if they know of a product that might.

[lingyi]

Good luck and keep us updated on your results.

[DB83]

If this video is a commercial film, you might be better off uploading a still or two. You never know someone might recognise it or at least provide some pointers.

[Sebastian42]

I've had a reply to my inquiry. It said that the feature I want is not available in the free version.
Google had an entry '21 alternatives to Video Comparer' - I am about to launch into testing those alternatives.

[sophisticles]

What the OP is trying to do is not possible, at least not in any practical way.

He wants a piece of software that is capable of "looking" at a single still screenshot/snapshot and match it to a movie he has on his hard drive.

If he had an actual frame, in theory, he could put together a script that used ffmpeg to compare said reference frame to each individual frame of each video and then include logic in the script that when a PSNR ratio higher than a certain value was returned and/or an SSIM value closer to 1 than a given percentage was calculated, said movie was flagged as the likely match.

But he states he has a "snapshot" of a single frame, maybe he could try facial recognition software but the stuff law enforcement uses is not available to the public and the open source variants, as far as I know, exist only as libraries that he would need to incorporate into an application he created.

As I said, for all intents and purposes, it's not going to happen.

[Sebastian42]

I revised my original statement - I do have a film clip (more than just a snapshot) of the complete video I want. Does that make it more feasible ?

[johns0]

Just do a search for .mkv,.mp4 etc...,it will show up.

[jagabo]

I've used an AviSynth script to locate an sample image in a video file. It could be used in a batch file to search all videos in a folder or on a drive. But the sample frame must not be cropped. It also must not be too "distorted" by filtering or compression artifacts.

[nicholasserra]

If I were tasked with this, i'd be writing a script in python or another language to input all videos, break the video into individual frames, then analyze each frame image with one of those image comparison libraries against the sample image.

I think you're going to have to build something yourself. Cool utility though. Maybe if I get free time i'll build it.

[nicholasserra]

@jagabo do you still have this Avisynth script?

I just ran into an issue where I also need to do this, take an individual frame and search a video frame by frame for a match.

Before I go writing this in python I want to find out if this exists anywhere.

[jagabo]

Here's the script, a sample video, and a sample image. The script searches through the video for the image. If it finds a match it outputs the frame number in a text file called match.txt.

v = LSmashVideoSource("video.mp4")
i = ImageSource("small145.jpg", fps=v.framerate).ConvertToYV12(matrix="rec709").Trim(0, length=1)
v = v.BicubicResize(i.width, i.height)
diff = AbsDiff(v, i)
WriteFileIf(diff, "match.txt", "AverageLuma<5.0", "current_frame", append = false)



function AbsDiff(clip v1, clip v2)  # Absolute value of clip1 - clip2
{
    Subtract(v1, v2)
    Overlay(last.ColorYUV(off_y=-126), last.Invert().ColorYUV(off_y=-130), mode="add")
}

You need to run through the entire video to find the matching frame. You can do this by opening the script in VirtualDub and selecting File -> Run Video Analysis Pass. VirtualDub will then scan through the video as fast as it can. Or you can use the CLI version of AvsMeter to run through the video (suitable for use in a batch file). You can also play the AVS script in a media player like MPCHC but that will be much slower as the media player will play the script in real time (ie, an hour long video will take an hour to play).

In this example the image is half the size of the video so the video is downscaled to match the image dimensions. Also note that the ConvertToYV12() will use a limited range rec.601 matrix by default. You may need to change that if your image was produced differently (rec.709, full range...). And this will not work if the image doesn't contain the full frame. An image that's a crop of the full frame will not be found. You may also need to change the threshold value (5.0 in the script) to another, depending on the quality of the image. Lower values will be more selective (require a closer match), higher values less selective (a looser match).

[nicholasserra]

Beautiful thank you!

[nicholasserra]

I think I need a bit more instruction on how to run this.

I can't open the avs script in vdub because it gives me the error "does not have a video stream" and menu option "run video analysis pass" is greyed out until you have something open.

I must be missing something simple here.

[nicholasserra]

Nevermind, 32 bit AVIsynth vs 64 bit vdub issue.

Got it running with a full frame image, no video cropping, against a UTvideo clip. But not seeing matches. Does this only handle black and white? That's what i'm seeing when I run the analysis.

[nicholasserra]

And now have it all working. Sorry for the wall of messages. Here's my modifications for running against a UTVideo uly2 AVI file, and comparing against a jpeg of identical size. I had to bump the score up to 8.0 to get the matches I was looking for, as i'm trying to find spots in Video8 captures where the screen goes blue, when the analog converter can't process the image. Very useful, thank you!

Attached is the frame I was searching for.

v = AVISource("clip.avi").ConvertToYV12(interlaced=Tru e)
i = ImageSource("frame.jpg", fps=v.framerate).ConvertToYV12(matrix="rec601")
diff = AbsDiff(v, i)

WriteFileIf(diff, "match.txt", "AverageLuma<8.0", "current_frame", append = false)

function AbsDiff(clip v1, clip v2)
{
Subtract(v1, v2)
Overlay(last.ColorYUV(off_y=-126), last.Invert().ColorYUV(off_y=-130), mode="add")
}

[jagabo]

For a nearly all blue image like that you could probably use the average U and V values rather than an image. Cropping away the black borders shows the average U value is 215, average V values is 114.

ImageSource("frame.jpg") 
ConvertToYV12() # blue screen will have U=215, V=114
Crop(32,32,-32,-32)
ColorYUV(analyze=true)

You can probably get away with just looking for an average U greater than 213 or thereabouts. Some other colors near blue may give false positives but real video usually won't have frames that are a solid, highly saturated, color like that.

AVISource("clip.avi").ConvertToYV12(interlaced=True)
Crop(16,16,-16,-16) # crop away discolored edges
WriteFileIf(last, "match.txt", "AverageChromaU>213.0", "current_frame", append = false)

[nicholasserra]

Awesome, yup that's way nicer and quicker. Takes about 25% of the time of the other method. With some rounding of the frame numbers i'm given a nice easy list of approximate sections to check for dropouts. This is going into my capture process right after trimming down the capture file to check if I need to do another pass or use a different deck/camcorder.

Thanks!

[jagabo]

Back on the subject of how to find an image in a bunch of videos, here's a batch file that will search all the MP4 file in a folder:

@echo off

for %%F  in (*.mp4) do (
    echo v = LWlibavVideoSource^("%%~dpnxF", cache=false^) >script.avs
    echo i = ImageSource^("image.jpg", fps=v.framerate^).ConvertToYV12^(^).Trim^(0, length=1^) >>script.avs
    echo v = v.BicubicResize^(i.width, i.height^) >>script.avs
    echo diff = AbsDiff^(v, i^) >>script.avs
    echo WriteFileIf^(diff, "match.txt", "AverageLuma<5.0", "current_frame", append = false^) >>script.avs
    echo function AbsDiff^(clip v1, clip v2^) >>script.avs
    echo { >>script.avs
    echo     Subtract^(v1, v2^) >>script.avs
    echo     Overlay^(last.ColorYUV^(off_y=-126^), last.Invert^(^).ColorYUV^(off_y=-130^), mode="add"^) >>script.avs
    echo } >>script.avs

    echo %%~dpnxF
    "g:\program files\avsmeter190\avsmeter64" script.avs 1>nul 2>nul
    type match.txt

    del match.txt
    del script.avs
)
pause

For each file in the folder it builds a script then uses afsmeter64 to run the script. It will show the full path of each file as it is being searched. If a matching frame is found the frame number is displayed. If you don't see a number the frame was not found in that video:

E:\VCR\AVS\Find Image in Video\ScanAll\1min clip.mp4
E:\VCR\AVS\Find Image in Video\ScanAll\avideo.mp4
145
E:\VCR\AVS\Find Image in Video\ScanAll\PhaseShifted.mp4
Press any key to continue . . .

You can also search an entire folder tree by changing the "for" line from

for %%F  in (*.mp4) do (

to

for /R %%F  in (*.mp4) do (

Running the batch file from the root of a drive will search through all mp4 files on that drive.

[codehound]

Wow. You lads have some time on your hands.........fairplay for your commitment.

[_Al_]

I'd like to bring attention to vapoursynth, it all can be done within a single script, because of the nature of being it a python script.
-script could generate list of paths for particular files (by extensions, dimensions etc.)
-check for differences
-actually request all frames (no need to run it in other app)
-print finds to python console.

import vapoursynth as vs
from vapoursynth import core
import os

DIRECTORY = r'D:\movies'
IMAGE = r'D:\pictures\snapshot.jpg'
TOLERANCE = 8


tolerance_float = TOLERANCE/255
files = os.listdir(DIRECTORY)
paths = [os.path.join(DIRECTORY, file) for file in files if file.endswith(".mkv")]

image_clip = core.ffms2.Source(IMAGE)[0]

for path in paths:
    clip = core.ffms2.Source(path)
    print(f'searching in: {path}')
    if clip.width > image_clip.width:
        image_clip_adjust = image_clip.resize.Bicubic(format = clip.format.id, matrix_s='709')
        clip = clip.resize.Bicubic(width = image_clip.width, height = image_clip.height)
    else:
        image_clip_adjust = image_clip.resize.Bicubic(width = clip.width, height = clip.height, format = clip.format.id, matrix_s='709')

    y1 = clip.std.ShufflePlanes(0, vs.GRAY)
    y2 = image_clip_adjust.std.ShufflePlanes(0, vs.GRAY)
    diff = core.std.Expr([y1, y2], 'x y - abs').std.PlaneStats()

    for n, frame in enumerate(diff.frames()):
        if frame.props.PlaneStatsAverage < tolerance_float:
            print(f'match: frame {n}')
            continue
print('done')

I just use simple luminescence average comparison for pixels, method could be perfected, just bringing attention to avs/vs differences