VideoHelp Forum
+ Reply to Thread
Results 1 to 18 of 18
Thread
  1. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    I am not a English speaker, so please don't mind my bad grammar

    I got amounts of videos that are made from different types highlight videos, each video has so many different elements,I want to cut
    the video by scene and classify each video into different tag, at last combine each group of videos.

    here are my problems:
    1.some videos have a lot of flash or shift which may cause the cut be too short and discontinuous; How can I get over those noise or how
    can I easily find the wrong cut and delete them before combine;
    2.some videos have different transition frames, some have black frames, some have TV snowflakes, some might change directly....
    Is there an automatic way to find those video types, so I can cut the video by different method.
    3.I am thinking about cut the video by audio beat change, is there any good advice to do that way?

    I am a newbie of video processing, some questions might be ridiculers, please let me know if you have any advice.

    I code In python and know some C and C++,Its ok if you list the code or algorithm in other code languages.

    sincerely

    Jackie Yan, China, Beijing
    Quote Quote  
  2. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    emmmmm...
    Quote Quote  
  3. Member Budman1's Avatar
    Join Date
    Jul 2012
    Location
    NORTHWEST ILLINOIS, USA
    Search Comp PM
    Quick question... when you say different types do you mean different content or file type some avi, some mp4, some mpg, etc? What are some of the file types?
    Quote Quote  
  4. Member Cornucopia's Avatar
    Join Date
    Oct 2001
    Location
    Deep in the Heart of Texas
    Search PM
    Describe your idea of "beat change", "scene" as you would if to a computer?...
    (that will help you realize the uphill battle with coding such a task)

    Scott
    Quote Quote  
  5. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    Originally Posted by Budman1 View Post
    Quick question... when you say different types do you mean different content or file type some avi, some mp4, some mpg, etc? What are some of the file types?
    content that I mean. Like cat, basketball, cartoon, etc. those parts are in the same video, I want to split different content.
    Quote Quote  
  6. Member Budman1's Avatar
    Join Date
    Jul 2012
    Location
    NORTHWEST ILLINOIS, USA
    Search Comp PM
    I use ffmpeg for my editing although it takes a bit to understand how to use since it is a command line program. It has a scene detect function but it fails on some scene changes like fade in fade out. But most editors only cut on key frame or I-frame which may not be at the cut point you wish. For this you need a smart cutter that only reencodes what is necessary to reduce quality loss.

    I have written a program to do this but it is only for mp4 h264 at the moment. It uses avisynth to trim and encode the portion before the key frame and join it to the next frame accurate cut.

    It works for me but is still very beta and buggy.
    Quote Quote  
  7. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    Originally Posted by Budman1 View Post
    I use ffmpeg for my editing although it takes a bit to understand how to use since it is a command line program. It has a scene detect function but it fails on some scene changes like fade in fade out. But most editors only cut on key frame or I-frame which may not be at the cut point you wish. For this you need a smart cutter that only reencodes what is necessary to reduce quality loss.

    I have written a program to do this but it is only for mp4 h264 at the moment. It uses avisynth to trim and encode the portion before the key frame and join it to the next frame accurate cut.

    It works for me but is still very beta and buggy.
    Thanks for your reply, I use moviepy and pyscenedetect which are based on ffmpeg and easy to understand.

    What I want to know is how you deal with the complicated scene, like quick camera shifting, explosions which may make me cut in wrong place.
    or how can I easily found a cut is wrong so I can trash this cut. I really don't want to confirm them one by one
    Quote Quote  
  8. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    Originally Posted by Cornucopia View Post
    Describe your idea of "beat change", "scene" as you would if to a computer?...
    (that will help you realize the uphill battle with coding such a task)

    Scott
    difficult task can be done in some dirty and direct ways, and optimize the result by specific strategy to solve problem in specific situation.
    I don't want a general method, a method can solve some problem is ok.
    like beat change, I can do It by recognizing the sound wave form, also can do it by clustering sound size on time.

    Its might be ridiculers, but I want to try every method to solve the problem
    Quote Quote  
  9. Member budwzr's Avatar
    Join Date
    Apr 2007
    Location
    City Of Angels
    Search Comp PM
    I looked over all your posts, and I think it's a non-starter inside a conundrum.
    Quote Quote  
  10. Member Cornucopia's Avatar
    Join Date
    Oct 2001
    Location
    Deep in the Heart of Texas
    Search PM
    Originally Posted by JackieYan View Post
    Originally Posted by Cornucopia View Post
    Describe your idea of "beat change", "scene" as you would if to a computer?...
    (that will help you realize the uphill battle with coding such a task)

    Scott
    difficult task can be done in some dirty and direct ways, and optimize the result by specific strategy to solve problem in specific situation.
    I don't want a general method, a method can solve some problem is ok.
    like beat change, I can do It by recognizing the sound wave form, also can do it by clustering sound size on time.

    Its might be ridiculers, but I want to try every method to solve the problem
    I'm an audio engineer, and what you're say makes no sense from an audio perspective.

    To recognize "sound wave form" of not just a "beat" but a "beat change", you would have to have an (AI?) engine that could understand the difference between high & low frequencies, and between low transients vs low steady tones, and between repetitive low transients which form the perception of a beat and those which are just part of the tone/overtone structure. And once it understood a "beat" it would still have to build up an understanding of a steady tempo in order to find a "beat change".

    Also, "clustering sound size over time"? What is your definition of "Sound Size"? That is not a thing.

    Scott
    Quote Quote  
  11. Member Budman1's Avatar
    Join Date
    Jul 2012
    Location
    NORTHWEST ILLINOIS, USA
    Search Comp PM
    What I want to know is how you deal with the complicated scene, like quick camera shifting, explosions which may make me cut in wrong place.
    It is possible to display the movie in a player as well as its frame number and time code. This would allow 'jumping' over bad frames say 240-242 and make the cut 0-240 and 243-264, where another 'blast/shadow/fade' occurs.

    Additional information:
    To show you what you are up against, I have included a short sample video that has a key frame every 60 frames (GOP 60). Notice the key frame is at 240 in this example but the scene change occurs at frame 241. This is the same problem with splitting at audio changes, even if you could, it doesn't mean you would get the correct spot to cut. The only way I know that works correctly is to find the approximate cut point, find what key frame is before and after and decide if you want to lose some by cutting at Key frame After your scene change or cut at the scene change and re-encode up to the next Key Frame.

    Image
    [Attachment 49112 - Click to enlarge]

    Image
    [Attachment 49113 - Click to enlarge]

    Image
    [Attachment 49114 - Click to enlarge]
    Last edited by Budman1; 23rd May 2019 at 00:56.
    Quote Quote  
  12. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    Originally Posted by Cornucopia View Post
    Originally Posted by JackieYan View Post
    Originally Posted by Cornucopia View Post
    Describe your idea of "beat change", "scene" as you would if to a computer?...
    (that will help you realize the uphill battle with coding such a task)

    Scott
    difficult task can be done in some dirty and direct ways, and optimize the result by specific strategy to solve problem in specific situation.
    I don't want a general method, a method can solve some problem is ok.
    like beat change, I can do It by recognizing the sound wave form, also can do it by clustering sound size on time.

    Its might be ridiculers, but I want to try every method to solve the problem
    I'm an audio engineer, and what you're say makes no sense from an audio perspective.

    To recognize "sound wave form" of not just a "beat" but a "beat change", you would have to have an (AI?) engine that could understand the difference between high & low frequencies, and between low transients vs low steady tones, and between repetitive low transients which form the perception of a beat and those which are just part of the tone/overtone structure. And once it understood a "beat" it would still have to build up an understanding of a steady tempo in order to find a "beat change".

    Also, "clustering sound size over time"? What is your definition of "Sound Size"? That is not a thing.

    Scott
    well, I don't know much about sound encoding, but I have worked a while for electrocardiogram recognition, I thought they are same from image perspective,
    maybe I am wrong, but thanks for your reply
    Quote Quote  
  13. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    Originally Posted by Budman1 View Post
    What I want to know is how you deal with the complicated scene, like quick camera shifting, explosions which may make me cut in wrong place.
    It is possible to display the movie in a player as well as its frame number and time code. This would allow 'jumping' over bad frames say 240-242 and make the cut 0-240 and 243-264, where another 'blast/shadow/fade' occurs.

    Additional information:
    To show you what you are up against, I have included a short sample video that has a key frame every 60 frames (GOP 60). Notice the key frame is at 240 in this example but the scene change occurs at frame 241. This is the same problem with splitting at audio changes, even if you could, it doesn't mean you would get the correct spot to cut. The only way I know that works correctly is to find the approximate cut point, find what key frame is before and after and decide if you want to lose some by cutting at Key frame After your scene change or cut at the scene change and re-encode up to the next Key Frame.

    Image
    [Attachment 49112 - Click to enlarge]

    Image
    [Attachment 49113 - Click to enlarge]

    Image
    [Attachment 49114 - Click to enlarge]
    maybe I can just skip the short cut and edit each cut, delay Start Key Frame and advance the End Key Frame
    Quote Quote  
  14. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    Originally Posted by budwzr View Post
    I looked over all your posts, and I think it's a non-starter inside a conundrum.
    hahaha, but this work is interesting
    Quote Quote  
  15. Member Cornucopia's Avatar
    Join Date
    Oct 2001
    Location
    Deep in the Heart of Texas
    Search PM
    Originally Posted by JackieYan View Post
    Originally Posted by Cornucopia View Post
    Originally Posted by JackieYan View Post
    Originally Posted by Cornucopia View Post
    Describe your idea of "beat change", "scene" as you would if to a computer?...
    (that will help you realize the uphill battle with coding such a task)

    Scott
    difficult task can be done in some dirty and direct ways, and optimize the result by specific strategy to solve problem in specific situation.
    I don't want a general method, a method can solve some problem is ok.
    like beat change, I can do It by recognizing the sound wave form, also can do it by clustering sound size on time.

    Its might be ridiculers, but I want to try every method to solve the problem
    I'm an audio engineer, and what you're say makes no sense from an audio perspective.

    To recognize "sound wave form" of not just a "beat" but a "beat change", you would have to have an (AI?) engine that could understand the difference between high & low frequencies, and between low transients vs low steady tones, and between repetitive low transients which form the perception of a beat and those which are just part of the tone/overtone structure. And once it understood a "beat" it would still have to build up an understanding of a steady tempo in order to find a "beat change".

    Also, "clustering sound size over time"? What is your definition of "Sound Size"? That is not a thing.

    Scott
    well, I don't know much about sound encoding, but I have worked a while for electrocardiogram recognition, I thought they are same from image perspective,
    maybe I am wrong, but thanks for your reply
    That explains it. EKG would show the equivalent of a single non-harmonic tone (sine wave if you will), with large gaps of silence in between, thereby making obvious a beat and beat changes (which you could do with difference thresholds).
    Normal audio of movies etc would be like trying to determine the beat and beat changes of an entire zoo (of animals, birds, fish...) all combined at once.

    Scott
    Quote Quote  
  16. Member
    Join Date
    May 2019
    Location
    China BeiJing
    Search PM
    Originally Posted by Cornucopia View Post
    Originally Posted by JackieYan View Post
    Originally Posted by Cornucopia View Post
    Originally Posted by JackieYan View Post
    Originally Posted by Cornucopia View Post
    Describe your idea of "beat change", "scene" as you would if to a computer?...
    (that will help you realize the uphill battle with coding such a task)

    Scott
    difficult task can be done in some dirty and direct ways, and optimize the result by specific strategy to solve problem in specific situation.
    I don't want a general method, a method can solve some problem is ok.
    like beat change, I can do It by recognizing the sound wave form, also can do it by clustering sound size on time.

    Its might be ridiculers, but I want to try every method to solve the problem
    I'm an audio engineer, and what you're say makes no sense from an audio perspective.

    To recognize "sound wave form" of not just a "beat" but a "beat change", you would have to have an (AI?) engine that could understand the difference between high & low frequencies, and between low transients vs low steady tones, and between repetitive low transients which form the perception of a beat and those which are just part of the tone/overtone structure. And once it understood a "beat" it would still have to build up an understanding of a steady tempo in order to find a "beat change".

    Also, "clustering sound size over time"? What is your definition of "Sound Size"? That is not a thing.

    Scott
    well, I don't know much about sound encoding, but I have worked a while for electrocardiogram recognition, I thought they are same from image perspective,
    maybe I am wrong, but thanks for your reply
    That explains it. EKG would show the equivalent of a single non-harmonic tone (sine wave if you will), with large gaps of silence in between, thereby making obvious a beat and beat changes (which you could do with difference thresholds).
    Normal audio of movies etc would be like trying to determine the beat and beat changes of an entire zoo (of animals, birds, fish...) all combined at once.

    Scott
    I get it, many thanks to you!
    Quote Quote  
  17. Based on my personal experience. Here are my perspectives.
    1. As for the question 1 and question 2, the automatic way might rely on AI ( i guess). Most video editing tools at the current market (as far as I concerned) still lack such intelligent power.
    Therefore, my advice is that you'd better find the video clips you wanna remove, remember or write down the exact start/end time, and then manually do the cutting stuff.
    2. The black frames and the snowflakes, are you sure they're born with your video? Or is it appearing only when you playback on your device?
    3. Do you have to try to set up the fixed GOP for your video or change your GOP?
    Quote Quote  
  18. Member Budman1's Avatar
    Join Date
    Jul 2012
    Location
    NORTHWEST ILLINOIS, USA
    Search Comp PM
    maybe I can just skip the short cut and edit each cut, delay Start Key Frame and advance the End Key Frame
    1. Yes you will have to edit each cut, preferably with a viewer that allows forward and backward movement by frame with an indicator of which type of frame each is. Scene changes do not always occur with a key frame where you want it and scene change scripts do not always catch all the scene changes as in fade to black where each successive frame is not much different than its preceding or following frame.

    2. You can't just 'Delay Key Frame' without Re-encoding. If you are re-encoding you can just put a Key Frame anywhere you wish. Most any editor will work if you encode, after you find the cut point you wish. If you wish to have minimum quality loss by copying, you will still have to encode any areas starting without a I-Frame (key frame) up to the next I-Frame with approximately the same attribute, size, FPS, etc. for joining afterward.

    3. You don't have to advance the key frame at the end unless you are working with Open GOP and the end occurs before an I or P frame.

    In the diagram below, I have made a pictorial of the video I posted in my earlier thread. If I want to cut at frame 241 (scene change start frame) through the end of the video, Frames 241 through 299 would have to be re-encoded since it does not start with an I-Frame. Then frames 300 through the end of the video could be copied and the 2 segments joined. together

    K = Key Frame
    | = Cut Points

    K-------------K--------------------K----------------|
    0 240. 300

    |---------------|-------------------|-----------------|
    0 241. 300

    COPY-------->|RECODE----------|COPY-----------|
    0 241. 300

    K. KK. K

    CONCAT 241-300, 300-END

    As a Pictured explanation, The program below copies (Keep checkbox) frame 0 through 240, then re-encodes 241 through 299, then copies frame 300 through 451

    Image
    [Attachment 49222 - Click to enlarge]
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!