Removing Chunks of a Video Where Nothing Happen (Video and Sound)

[Mordan]

Hello,

I have been trying many video programs and I have not been able to do a "simple" thing: I record a card game online. many times nothing happens on the screen for 3 minutes.

I want to remove and merge all identical frames into a single one, so that viewers of my video only see frames when there is actually something happening. There is no point for the viewers to watch 3 minutes of nothing!

Card game last 2 hours with lots of small uninteresting pauses. By removing all identical frames, video would be about 30 minutes.

Also the tricky part, sometimes some audio comes in. Therefore, a frame is identical only if there is no human audible audio. I guess you would have to give a threshold at which audio is deemed audible.

Any tips on how to achieve this? I have VirtualDub.

Cheers

[hech54]

I want to remove and merge all identical frames into a single one

That makes no sense.....that's never going to happen.
You will need to watch(at least scroll through) the entire video and manually remove(edit out) the boring parts.

[jagabo]

There are AviSynth filters that can remove identical frames but they are not sensitive to audio. There's another filter which draws the audio waveform onto the frame. Putting those together you may be able to come up with something. Here's a start:

AviSource("video.avi") 
ConvertToYUY2()

AddBorders(0,0,0,height*2) # triple the frame height so AudioGraph() doesn't write into the picture
AudioGraph(last, 1) # draw audio waveform in the middle of the frame
Crop(0,0,-0,-height/3) # remove bottom 1/3 since it's all black

# Figure out how to remove duplicate frames here....

Crop(0,0,-0,-height/2) # remove the drawn audio waveform, leaving just the original picture

A sample frame before the final crop:



Silent frames will have just a straight line across the frame.

[poisondeathray]

Similar to jagabo's idea, this plugin waveform() might help
http://forum.doom9.org/showthread.php?t=165703

If you use a NLE (eg. sony vegas, premiere pro etc....) you can do similar method.

But you're probably looking for easy "automatic" way to do this - I don't know of any

[Mordan]

I want to remove and merge all identical frames into a single one

That makes no sense.....that's never going to happen.
You will need to watch(at least scroll through) the entire video and manually remove(edit out) the boring parts.

Not going to happen? Maybe I didn't explain it correctly. Your option is not practical. I can have literally thousands of small cuts during a game.

The software BB FlashBack Pro 3 Recorder is the closest to be able to do that. It has a function tool to remove frames of "inactivity" but I can't define inactivity as identical consecutive frames, let alone without sound activity.
I emailed them I would be willing to buy if that feature is implemented. They responded saying the feature will not be implemented. It is a simple inactivity definition! The way their recording format works, it seems bloody straightforward.

Anyways thx for the starting tips. Any forum post on how to get started with AviSynth. The website is in Russian...

[Mordan]

Silent frames will have just a straight line across the frame.

So the idea is to tag a frame Sound On or Sound Off. Any frames with Sound On is never deleted...

[hech54]

I want to remove and merge all identical frames into a single one

That makes no sense.....that's never going to happen.
You will need to watch(at least scroll through) the entire video and manually remove(edit out) the boring parts.

Not going to happen? Maybe I didn't explain it correctly. Your option is not practical. I can have literally thousands of small cuts during a game.

.....and after getting your patience worn down to an indistinguishable fleshy nub trying to learn AviSynth and scripting....you'll be scrolling through the video as I suggested.....then you'll have to deal with which output format you'll want for the final, edited video since you most likely will not have cut on the video's Key Frames.

[sanlyn]

Any forum post on how to get started with AviSynth. The website is in Russian...

This one is in English: http://avisynth.org/mediawiki/Main_Page
version 2.5.8 download (stick with 32-bit version): http://sourceforge.net/projects/avisynth2/files/AviSynth%202.5/

You might want to check the forums Tools section for mp4 editors.

[hech54]

Any bets on how long it will take before the OP comes back to inform everyone of his complete success.....without giving any details as to how he did it. About a week maybe?

[Mordan]

Any bets on how long it will take before the OP comes back to inform everyone of his complete success.....without giving any details as to how he did it. About a week maybe?

No... I have been looking (on and off) for a long term solution for 6 months. I will never go through the videos manually. Imagine doing this work for countless videos.

I have just installed AviSynth 2.5.8 and AvsPmod.

[smrpix]

So are you basically recording all these hours of video to never watch them?

Any good NLE with a waveform display should allow you to go through and manually do a 2 hour video in less than a half hour. Not a very long time if you don't let it get backlogged.

[Cornucopia]

Ok, this is a really corny idea, but how about: run these videos through security camera systems where they are motion-activated (threshold adjustable w/ preroll buffer settings). with no motion (or sound) it cuts off, then creates a new clip the next time there is motion/sound. Then just append all the clips together.
Doubtful that you could apply it to already recorded footage, but in a pinch you could "shoot the screen".

Scott

[smrpix]

but in a pinch you could "shoot the screen".

Scott

Yes, that would be much faster than loading it into an NLE

[poisondeathray]

I haven't tested this but, a semi automated way might be premiere/audition .

In audition you can use Diagnostics > Delete Silence and Mark Audio . There are settings and adjustments for threshold. I think those markers can be transferred to premiere through dynamic link . In premiere you would ripple delete the marked sections

[Mordan]

There are AviSynth filters that can remove identical frames but they are not sensitive to audio.

AddBorders(0,0,0,height*2) # triple the frame height so AudioGraph() doesn't write into the picture
AudioGraph(last, 1) # draw audio waveform in the middle of the frame
Crop(0,0,-0,-height/3) # remove bottom 1/3 since it's all black

# Figure out how to remove duplicate frames here....

A sample frame before the final crop:

Audiograph plug in installed. I'm up and running on a test video.

What is the plug-in you are referring to for deleting identical frames?

I will sort out the audio sync later.

PS: i have looked around and read about Decimate but it does not sound like it is what I need here. I mean, I'm not looking for specific frame rates.

[jagabo]

What is the plug-in you are referring to for deleting identical frames?

Here's a discussion of a similar problem: https://forum.videohelp.com/threads/344888-Automatically-Delete-Select-Frames

[manono]

I never answered earlier because you mentioned the audio requirement too.

AviSynth's Multidecimate filter can remove all duplicate frames without regard to framerate. But audio synch will fly out the window.

http://neuron2.net/multidecimate/multidecimate.html

Dedup can do the same:

http://akuvian.org/src/avisynth/dedup/

[Mordan]

I never answered earlier because you mentioned the audio requirement too.

AviSynth's Multidecimate filter can remove all duplicate frames without regard to framerate. But audio synch will fly out the window.

http://neuron2.net/multidecimate/multidecimate.html

Dedup can do the same:

http://akuvian.org/src/avisynth/dedup/

ok. i got them thx!

by the way, which AVI codec do you advice to use in order to record the 1200*800 screen on the game table?

Right now I'm using the free BB Flashback Express recorder. This program records 2 hours with 1 gigaoctet used on disk without any loss of quality. It is a proprietary format from which I can export to AVI or Flash video. I tried to export to AVI YUV intel and it crashed after writing 10 Giga!!

[manono]

ok. i got them thx!

You can set an arbitrary framerate later on. It might be interesting to see the play speeded up and set to something like a silent film soundtrack.

by the way, which AVI codec do you advice to use in order to record the 1200*800 screen on the game table?

I don't record card games. Sorry.

[Mordan]

*duplicate post*

[Mordan]

Hello,

BB Flashback Express allows me to export with any AVI codec. So now I'm using DivX 6.9.2 codec which makes my 2hours video only 400mb.

Anyways, I managed to use the Dedup function. I'm happy with the result.

However I have a few questions.

During the first pass, do I have to run the 2 hours video for the time codes log to be written?
Also where is written the log file? I'm curious to see the content.

I'm trying to solve the audio sync issue since Dedup does not cut audio. Would VFR video format help?
Couldn't Dedup use the time codes in the first pass to cut the audio correspondingly?

Cheers.

[manono]

I've never used Dedup. I had quite a lot of experience with MultiDecimate a few years back. With it you run a fast first pass for it to gather the information about the relative differences between frames, with no other filters being used. It's not really a pass, as in a first encoding pass. You just play the video in VDub. Maybe Dedup is the same, I don't know. And the information gathered is generated in the same folder as the rest of the video stuff you're using.

It also has a way to keep the audio synch but, again, I have no idea how Dedup works.

[Mordan]

OK.

I tried Multidecimate.

AVISource("C:\Users\Mordan\Documents\BB FlashBack Movies\test_dedup.avi")
ConvertToYUY2()
AudioGraph(last, 1)

MultiDecimate(pass=1)

The audio graph is used to tag frames with audio so they are not removed.

So after running the video.

It creates the mfile.txt

Load it in the standalone to create dfile and cfile with the setting "Remove duplicates: Global: Naive"

Now run second pass.

AVISource("C:\Users\Mordan\Documents\BB FlashBack Movies\test_dedup.avi")
ConvertToYUY2()

MultiDecimate(pass=2)

It works just as good.

But like Dedub, the audio is not synchronized with the multidecimate and the script shown above.

Anyone knows how to decimate while keeping the audio synchronized?

Hopefully I can soon show you the end product of this thread !
Cheers

[poisondeathray]

dedup isn't supposed to cut audio. It outputs timecodes for VFR video (this means you drop the duplicates but audio is the same, it's for purposes of encoding fewer frames)

there is a script for after effects that might do what you want
http://aescripts.com/awkward-pause/

[manono]

But like Dedub, the audio is not synchronized with the multidecimate and the script shown above.

Anyone knows how to decimate while keeping the audio synchronized

AviSynth's Multidecimate filter can remove all duplicate frames without regard to framerate. But audio synch will fly out the window.

I warned you. The only way to preserve audio synch using Multidecimate the way you are is to use a Cycle-based mode (not Naive). The downside is that way too many duplicate frames will be retained and the scenes with movement will play very jerky. The only alternative I can think of is to start removing silences in a WAV editor.

[jagabo]

I just rediscovered an AviSynth function called CutFrames.avsi. It removes the audio as well as the video.

http://forum.doom9.org/showthread.php?t=135423

Unfortunately, you have to work backwards because the forward frame numbers change after your each cut. Ie, after cutting out frames 100 to 200, what was frame 201 is now frame 100.

[vhelp]

it sound like you have a camera pointing on a persons hand with cards.

Mordan, can you post a few scenes, motion and motion leading to no motion/activity ?

[vhelp]

this reminds me of when i wrote a twitter app to document/storyboard things as they happen during my capturing of the 2012 London Olympic games. as i captured the games, i wrote down what was going on. as boring as it was, it worked out well for that project because i needed an actual example situtation to develop and test the twitter tool. my point is, assuming you are shooting the video, then you could do something similar, in real time, on your computer. you could rig something crude like, using windows notepad and F5 key.

your twitter example could follow something like this:
1. open notepad
2. while filming live or playing back on your computer,
3. press F5 key, this will post the time/date, you could add a sentance or special code for cut/keep, etc.
4. convert {3} into timecodes.txt
5. import the timecode file into your NLE timeline
6. create macro to cut or keep sections according to your timecodes.

[Mordan]

it sound like you have a camera pointing on a persons hand with cards.

Mordan, can you post a few scenes, motion and motion leading to no motion/activity ?

Hello, thx for the interest.

Here is an example of the kind of scenes I'm dealing with.

https://www.youtube.com/watch?v=8Nv3iXMMsEc

We are currently recording the scene and compress by accelerating it. You can't actually see much because of that.

I haven't found a way to do what I wanted... maybe in the future. I will keep watching this thread.