How do I do an image-search within a video file? I want to find the exact moment in the video where a specific frame appears, which I have as a jpeg.
Ideally would be a way to do this within Adobe Premiere but I couldn't find any mention of this ability online.
Try StreamFab Downloader and download from Netflix, Amazon, Youtube! Or Try DVDFab and copy Blu-rays! or rip iTunes movies!
+ Reply to Thread
Results 1 to 11 of 11
Thread
-
-
Not possible.
Bit for bit the jpeg is different to a video. And both will also vary according to compression. -
Hi DB83, thanks for the reply.
This is unfortunate. I understand that the frame in the video and the image in the jpeg are different in their encoding, but they look practically identical.
I would imagine that some sort of image-search could work if they're visually similar enough, but I guess no one created such a tool yet.
I would think that it would work somewhat like Premiere's Synchronize option, which can synchronize similar audios even if they were recorded with different mics or use different types of audio formats. -
Its a numbers game. That's why audio has always preceded video when it comes to technical tricks.
To match a piece of audio you are comparing a pattern of 48,000 bits of information per second. In typical 1080 video it's over 63,000,000 per second, almost 4,000,000,000 per minute or nearly 224,000,000,000 per hour.
Someone will figure out how to do it, but it's not there yet. -
Also note that in audio matching you're searching for certain patterns of peaks and valleys over time. Matching a still to a video is like trying to pick out a specific cymbal crash from a recording of an entire symphony. Yep. Manually is still the best option for now.
-
You can do it with an AviSynth script. Load the video, the image, then compare frames with a runtime function and print out the frame number that matches (within a threshold). It's pretty simple if the jpeg image is the full frame (so you can compare the full video frame with the full image frame). It's much harder (and can't reasonable be done with AviSynth) if the jpeg image is just a small unknown crop of the video frame.
Example script:
Code:function AbsDiff(clip v1, clip v2) { # compute abs(v1-v2) Subtract(v1, v2) Overlay(last.ColorYUV(off_y=-126), last.Invert().ColorYUV(off_y=-130), mode="add") } v = LSmashVideoSource("video.mp4") i = ImageSource("image.jpg", fps=v.framerate).ConvertToYV12() diff = AbsDiff(v, i) WriteFileIf(diff, "match.txt", "AverageLuma<3.0", "current_frame", append = false)
The frame I was searching for:
[Attachment 48908 - Click to enlarge]
A diff example of a non-matching frame:
[Attachment 48909 - Click to enlarge]
Diff of the matching frame is all black. I'm sure you can imagine what it looks like.
The contents of match.txt
Code:367
As noted earlier, this won't work if the image is only a portion of the frame. For example, this small image was cropped from the same frame:
[Attachment 48910 - Click to enlarge]
There is no way to know beforehand what portion of the frame to compare too (subtract from). It gets even more difficult if the cropped image is further distorted (brightness, contrast, rotated, scaled, etc.):
[Attachment 48911 - Click to enlarge]
It would be very difficult to match that image to the video frame. It would probably require some A.I. algorithm.Last edited by jagabo; 29th Apr 2019 at 10:00.
-
Thanks jagabo!
Your solution is interesting.
Unfortunately, since I don't know anything about Avisynth, let alone scripting in it, and since I'll need to calculate the timecode from the frame number one by one, I doubt it'll be faster for me than looking up the frame manually.
Hopefully, your solution will be helpful to someone looking into a similar problem in the future.
It might also help me if I run into a frame which I have no idea where in the movie it was taken from.
Thanks! -
You will find some answers here (and in similar places):
https://www.learnopencv.com/image-recognition-and-object-detection-part1/ -
Similar Threads
-
Merge audio file and image to create a video. Error: "track 1: muxing mp3..
By DennisO in forum Newbie / General discussionsReplies: 3Last Post: 28th Nov 2018, 07:02 -
Video image won't load only sound with this file type
By Klamp in forum Newbie / General discussionsReplies: 1Last Post: 29th Jun 2016, 14:57 -
Help Please. Video Converter search 3d to m2t ?
By JohnBrayn in forum Video ConversionReplies: 1Last Post: 4th Jun 2015, 19:50 -
Partial video image in mov file
By Urban20 in forum EditingReplies: 5Last Post: 31st Jan 2015, 17:28 -
Pulling an image sequence from a video file
By greycobalt in forum Newbie / General discussionsReplies: 1Last Post: 31st Aug 2014, 08:30