Automate Frame Grabbing based on screen events

[Nick73]

Hi all, I am looking for a software solution to the following issue.

I video files that have an voltage input line running along the bottom of the screen, and video taking up the majority of the screen.

What i need is either to grab frames when the voltage spikes / or to drop the frames immediately following the spike.

There are frame grabber cards that will do this, but we have some (a lot) of older captured video that needs to be processed. Now I could go through frame by frame and click "grab frame" in VLC for example, but each file is ~16 thousand frames, so I am hoping there is a way to automate the process.

I have considered using the motion detection software that you can get for webcams and making the area where the voltage spikes as a 'hot zone' but I can't work out how to make the software 'see' the video as though it were live footage from a camera.

I have access to both PC and MAC so software for any platform has potential, and the video is what I think is called raw capture (~2gb = 10 mins) so rendering to any format is also not a problem.

Thanks in advance

Nick

[jagabo]

Something like this: https://forum.videohelp.com/threads/344888-Automatically-Delete-Select-Frames

A short video sample would be useful if you want more details.

[Nick73]

Hi Thanks for the reply, that sounds possible as my video files are monochrome so dropping frames when a certain area is white may be possible.

Ive got a short video loop that might help, but not sure how to upload but I have a screen shot .

What I need is to remove the frames (approx 15 frames) immediately following each pulse so that the pressure twitch of the vessel at the top is removed. Or to put it simply, I need to edit the file so I only get the frames between each spike!

[jagabo]

You can upload a video sample using the "Upload Files/Manage Attachments" button below the Quick Reply edit box.

Let me see if I understand the picture you uploaded. The lower third (not the very bottom) shows four major spikes (and some minor spikes). The spikes enter from the right of the frame and scroll to the left. At the top of the frame is an image (ultrasound?) that corresponds to the spikes current entering at the right. You want to keep frames when a spike appears at the right edge. You want to discard frames where there is no spike at the right edge. Is that right?

[Nick73]

Hi Thanks for the response, you almost correct. The spikes sweep across the screen and I want to delete the frames where there are spikes (and shortly after each spike) and only keep the frames leasing up to each spike. We measure the width of the vessel at the top, and the spikes make the vessel pulse and appear wider than usual, which is what Im trying to edit out.

The lower spikes are an ECG and the upper bigger spikes are doppler flow signals (yep its an ultrasound study)

I have attached a video file (hopefully) but this is a quick screen cap so ignore frame rates / file types etc.

I looked at avisynth, and it looks promising, unfortunately I code like brick floats....

[jagabo]

That clip has many duplicate frames (and not in a perfectly regular pattern) that makes detecting the differences between frames more difficult. It's also got some other oddities -- like it's interlaced video encoded progressively, variable frame rate, etc. Do you have something closer to the original?

In any case, here's the first part of the process, detecting when the peaks occur. At the bottom of the video are just the major peaks of the ECG (blue background) that would be used to determine what frames to eliminate.

I can see that there's a little swelling during the small peaks too. Do those need to be filtered out? It would just be a matter or increasing the height of the window used to detect the spikes (I intentionally used a window that only includes the taller spikes).

[Nick73]

Hi, sorry for the delay, but have been camping and out of contact for the last fortnight so please don't think the effort is wasted or unappreciated!!

I can see the idea from your file, I think is that if white spikes show up in the blue box, then those frames are dropped?

I havent got a sample to upload yet as they are huge raw digital video captures and I dont have any software to cut a small segment with. VLC codec report is attached. And yes if the box could be bigger to catch the secondary waves that would be ideal.

[jagabo]

I can see the idea from your file, I think is that if white spikes show up in the blue box, then those frames are dropped?

Yes, you would test the box and when the brightness exceeds some small threshold you would drop that frame.

And yes if the box could be bigger to catch the secondary waves that would be ideal.

That's a simple change.

I havent got a sample to upload yet as they are huge raw digital video captures and I dont have any software to cut a small segment with. VLC codec report is attached.

VirtualDub will let you trim out a small section for upload. Just put it in Video -> Direct Stream Copy mode, open a source video, use the Mark-in and Mark-out tools to mark the section you want to export, select Edit -> Crop To Selection, then File -> Save as AVI.

[Nick73]

H, thanks I had forgotten about VirtualDub. So attached is a small sample of the original capture file. Although as I said earlier if its easier to do this processing in a different file format that is no problem either as we have to covert the raw video anyway.

[jagabo]

That file is much cleaner. I'll work on it some more over the next few days.

[Nick73]

Brilliant. thanks very much!

[jagabo]

Here's what I've come up with:

GScript("""
  function DeleteFrames(clip c, string condition) {
  # DeleteFrames (Gavino, 16 Nov 2011)
  # A generic function that will delete all frames satisfying an arbitrary user-defined condition.
  # The condition can use any of the Avisynth run-time functions such as AverageLuma(), etc, 
  # and/or the variable 'current_frame'.
  # (Note - Since the condition is evaluated inside the function,
  # any variables used (except current_frame) must be global.)
  # Example use (delete duplicate frames):
  #   DeleteFrames("YDifferenceFromPrevious < 0.1 && current_frame > 0")

    c    
    fc = FrameCount()
    res = BlankClip(c, length=0)
    current_frame = 0
    
    while (current_frame < fc) {
      while (Eval(condition) && current_frame < fc) {
        current_frame = current_frame+1
      }
      if (current_frame < fc) { # not at end
        start = current_frame # start of wanted section
        while (!Eval(condition) && current_frame < fc) {
          current_frame = current_frame+1
        }
        res = res + Trim(start, start-current_frame)
      }
    }

    return res
  }
""")

# First get the video
prev = AviSource("Base-1.avi")

# We're going to subtract adjacent frames so get the next frame
this = Trim(prev,1,0)

# subtract the previous frame from the current frame
# you can't have negative values with signed integers so Subtract() adds 128 to the results
# we are only interested in positive changes (when new data is being drawn) so subtract
# 128 with ColorYUV(off_y=-128)
# values now range from 0 to 127, so double them with ColorYUV(gain_y=256)
# finally, crop the window down to just the graph area, call it "spikes"
global spikes = Subtract(this, prev).ColorYUV(off_y=-128).ColorYUV(gain_y=256).Crop(0,362,-0,78)

# add a copy of the spikes to the bottom of the frame for visual confirmation
# replace this line with "last = this" if you don't want to see "spikes"
StackVertical(this, spikes.ColorYUV(off_u=30))

# delete frames when the average luma of spikes is over a threshold, 0.2 used here
DeleteFrames("AverageLuma(spikes) > 0.2")

The output of this script is attached as a DV AVI, like the source.

[Nick73]

HI, many thanks for this. Your output file looks perfect, but I am having difficutly running the script . Virtuadub gives the following error

Avisynth open failure:
Color YUV: requires YUV input
(C:\\\framedrop.avs line 46)

Originally VirtualDub failed to work as this PC didn't have the right codec installed. Im using a pinnacle DV25 codec (which makes sense as the capture box is a pinacle one) - but am wondering if that is not seeing the file 'corectly' What DVSD codec have you been using?

Once again - thank you!

[jagabo]

I'm using Cedocida as my VFW DV decoder. It's set to output YUY2. I suspect your DV decoder is putting out RGB. Add ".ConvertToYUY2(interlaced=true)" right after the AviSource() command:

 prev = AviSource("Base-1.avi").ConvertToYUY2(interlaced=true)

[Nick73]

Hi, thanks for that, Ive added the line you suggested, and that fixed that error...but now I have a new one.

Avisynth open failure
Average plane:Only planar images (as YV12) are supported!
([Gscript] line20)
([Gscript] line 28)
(C:users\\\framedrop.avs, line 53)

Ive googled to try and fix this but the main problem is I really don't understand the terminology so Im sort of worried that my attempts to fix anything will make it worse!

Im using the same file as I uploaded here (Base1) in case that matters.

Thanks

Nick

[jagabo]

Oh, sorry, the video should be YV12, not YUY2. So use ConvertToYV12 instead:

 prev = AviSource("Base-1.avi").ConvertToYV12(interlaced=true)