VideoHelp Forum

+ Reply to Thread
Results 1 to 19 of 19
Thread
  1. I am currently editing vob files of old sci-fi dvds. However there are lots of dirt, scratch etc. WB simply dumped all the episodes without doing any basic editing, such as cleaning the films or cleaning the whatever mm reels free of dust etc.

    So far i am making progress and restoring dvd to better quality output but cleaning each episodes is rather time consuming. The amount of flickering is staggering. Currently using virtualdub2, avidemux, photoshop with topaz plugins, avisynth (a pain to learn), topaz gigapixel and video enhance (to output lossless tiff files), staxrip, handbrake.

    Film Dirt Cleaner (FDC) plugin for virtualdub2 is doing the most of the cleaning. However when there is a scene change, the last frame before scene change and the first frame after scene change doesn't get cleaned. FDC requires comparison of previous and next frame for the present frame to be cleaned or not. Therefore last frame before scene change and the first frame after scene change makes FDC unworkable.

    Using virtualdub2 I can go from one scene change to another but then I to type out the frame numbers in notepad++ of frames that require cleaning in photoshop. This is very time consuming.

    Is there a way to output all the frame numbers of all scene changes automatically to a txt file. I have been searching for a week and do not think anyone ever thought of creating this potentially time saving software/plugin whatever. Virtualdub2 is somewhat 3/4 of the way there.
    Quote Quote  
  2. Originally Posted by coolgit View Post
    Is there a way to output all the frame numbers of all scene changes automatically to a txt file. I have been searching for a week and do not think anyone ever thought of creating this potentially time saving software/plugin whatever. Virtualdub2 is somewhat 3/4 of the way there.
    There are several avisynth variations on writefileif (txt file) and scenechange detection in these threads

    https://forum.doom9.org/showthread.php?p=1678941
    https://forum.doom9.org/showthread.php?t=168697
    Quote Quote  
  3. Banned
    Join Date
    Jan 2021
    Location
    PAL
    Search Comp PM
    Originally Posted by coolgit View Post
    I am currently editing vob files of old sci-fi dvds. However there are lots of dirt, scratch etc. WB simply dumped all the episodes without doing any basic editing, such as cleaning the films or cleaning the whatever mm reels free of dust etc.

    So far i am making progress and restoring dvd to better quality output but cleaning each episodes is rather time consuming. The amount of flickering is staggering. Currently using virtualdub2, avidemux, photoshop with topaz plugins, avisynth (a pain to learn), topaz gigapixel and video enhance (to output lossless tiff files), staxrip, handbrake.

    Film Dirt Cleaner (FDC) plugin for virtualdub2 is doing the most of the cleaning. However when there is a scene change, the last frame before scene change and the first frame after scene change doesn't get cleaned. FDC requires comparison of previous and next frame for the present frame to be cleaned or not. Therefore last frame before scene change and the first frame after scene change makes FDC unworkable.

    Using virtualdub2 I can go from one scene change to another but then I to type out the frame numbers in notepad++ of frames that require cleaning in photoshop. This is very time consuming.

    Is there a way to output all the frame numbers of all scene changes automatically to a txt file. I have been searching for a week and do not think anyone ever thought of creating this potentially time saving software/plugin whatever. Virtualdub2 is somewhat 3/4 of the way there.
    Don't use any topaz products
    Quote Quote  
  4. Originally Posted by poisondeathray View Post
    Originally Posted by coolgit View Post
    Is there a way to output all the frame numbers of all scene changes automatically to a txt file. I have been searching for a week and do not think anyone ever thought of creating this potentially time saving software/plugin whatever. Virtualdub2 is somewhat 3/4 of the way there.
    There are several avisynth variations on writefileif (txt file) and scenechange detection in these threads

    https://forum.doom9.org/showthread.php?p=1678941
    https://forum.doom9.org/showthread.php?t=168697
    Ok I have added mvtools2.dll, DePan.dll, DePanEstimate.dll and tried your first link and tried both Scene Detection Using MVTools2 and Scene Detection Using Depanestimate and both gave an error - AviSource: Could not decompress frame 0.

    This is over my head.
    AVI file is

    Format : AVI
    Format/Info : Audio Video Interleave
    Format profile : OpenDML
    File size : 10.8 GiB
    Duration : 42 min 5 s
    Overall bit rate mode : Variable
    Overall bit rate : 36.8 Mb/s
    Writing library : VirtualDub2 build 44282/release

    Video
    ID : 0
    Format : FFV1
    Format version : Version 3.4
    Format settings, GOP : N=1
    Codec ID : FFV1
    Duration : 42 min 5 s
    Bit rate mode : Variable
    Bit rate : 36.8 Mb/s
    Width : 1 024 pixels
    Height : 576 pixels
    Display aspect ratio : 16:9
    Frame rate : 25.000 FPS
    Color space : YUV
    Chroma subsampling : 4:2:0
    Bit depth : 8 bits
    Scan type : Progressive
    Compression mode : Lossless
    Bits/(Pixel*Frame) : 2.495
    Stream size : 10.8 GiB (100%)
    coder_type : Range Coder
    MaxSlicesCount : 4
    ErrorDetectionType : Per slice
    Quote Quote  
  5. Member Skiller's Avatar
    Join Date
    Oct 2013
    Location
    Germany
    Search PM
    I use XviD via VirtualDub for this (yes, that old MPEG4 Codec that nobody uses anymore).
    I just let it run a 1st pass of a 2-pass encode and it'll generate a tidy text file during that first pass containing all the scene changes it found. You can also alter the threshold if needed.

    I needed this a couple of times for touching up subtitles; to find subtitles that are leaking just a few frames across a scene change because that's a big no-no in subtitles.
    Quote Quote  
  6. AVISource using VFW FFV1 decoder is incompatible with FFV1 v3
    Originally Posted by coolgit View Post
    AviSource: Could not decompress frame 0.

    Format : FFV1
    Format version : Version 3.4
    Use FFVideoSource (FFMS2) , or LWLibavVideoSource (LSmash) as the source filter
    Quote Quote  
  7. Member Cornucopia's Avatar
    Join Date
    Oct 2001
    Location
    Deep in the Heart of Texas
    Search PM
    If you wanted to get FDC, or other processes that require frame-1 and frame+1 as well as frame0 to fully work, you could always cheat by temp pre-adding dupes of the edge frames (e.g. new subclip = old subclip w dupe of 1st frame prepended & dupe of last frame appended) then, post-process remove those extra frames. Lots of extra scripting, but doable.


    Scott
    Quote Quote  
  8. Member
    Join Date
    Jul 2018
    Location
    Italy
    Search Comp PM
    BUt if you add before the first frame that you want to clean, a duplicate frame that has the same "spot", than the temporal filter cannot remove the defect, or am I missing someting?
    Quote Quote  
  9. Originally Posted by Skiller View Post
    I use XviD via VirtualDub for this (yes, that old MPEG4 Codec that nobody uses anymore).
    I just let it run a 1st pass of a 2-pass encode and it'll generate a tidy text file during that first pass containing all the scene changes it found. You can also alter the threshold if needed.

    I needed this a couple of times for touching up subtitles; to find subtitles that are leaking just a few frames across a scene change because that's a big no-no in subtitles.
    Tried it and it worked.

    However all I got is 7 columns and no bleeding header to tell me what each column is. Useless.
    Quote Quote  
  10. Originally Posted by poisondeathray View Post
    AVISource using VFW FFV1 decoder is incompatible with FFV1 v3
    Originally Posted by coolgit View Post
    AviSource: Could not decompress frame 0.

    Format : FFV1
    Format version : Version 3.4
    Use FFVideoSource (FFMS2) , or LWLibavVideoSource (LSmash) as the source filter
    Finally got this to work. Replace AVISource with FFVideoSource in your MVTools2 example.

    It worked.

    Clicked open scenes.txt and got.... drum roll..... NOTHING!!!

    So what next??
    Quote Quote  
  11. Originally Posted by coolgit View Post

    Clicked open scenes.txt and got.... drum roll..... NOTHING!!!

    So what next??
    Did you uncomment out the writefileif line ?

    "#" is a comment symbol in avisynth - it means ignore the line . ie. write nothing.

    You preview the script and adjust the thresholds when the writefileif line is commented out - so you can see if the scenechange detection is correct. If threshold is too high or too low - you might miss some scenechanges, or get false positives. Then when you want the actual file with the adjusted correct threshold, you uncomment out that line to write the file
    Quote Quote  
  12. Originally Posted by poisondeathray View Post
    Originally Posted by coolgit View Post

    Clicked open scenes.txt and got.... drum roll..... NOTHING!!!

    So what next??
    Did you uncomment out the writefileif line ?

    "#" is a comment symbol in avisynth - it means ignore the line . ie. write nothing.

    You preview the script and adjust the thresholds when the writefileif line is commented out - so you can see if the scenechange detection is correct. If threshold is too high or too low - you might miss some scenechanges, or get false positives. Then when you want the actual file with the adjusted correct threshold, you uncomment out that line to write the file
    This is the script so far.

    #Script to find scene changes

    Loadplugin("C:\Program Files (x86)\AviSynth\plugins64+\mvtools2.dll")

    filename = "E:\currser\BABYLON5\new\scenes.txt"
    BlockChangeThresh = 350 # Default: 350. Increase to reduce number of scenes detected;
    # Determines whether a block has changed from prev frame.
    Num_blocks_changed = 90 # Default: 90. Increase to reduce number of scenes detected;
    # How many changed blocks (given threshhold) must change to trigger scene change

    source=FFVideoSource("E:\currser\BABYLON5\new\Soul Hunter.avi").AssumeTFF().killaudio().colorYUV(auto gain=true) #add this to increase contrast
    source_fields=source.convertTOYV12(interlaced=fals e)
    source_super = source_fields.MSuper(pel=2, sharp=0)

    backward_vec = MAnalyse(source_super,isb = true, delta = 1, blksize=16,search=0)

    SceneChange = MSCDetection(source_fields, backward_vec,thSCD1=BlockChangeThresh,thSCD2=Num_b locks_changed)

    #Uncomment following line for troubleshooting. It puts the scene change threshold numbers on the screen.
    ScriptClip(source_fields,"Subtitle(String(AverageL uma(SceneChange ) ),align=5)")

    #This line writes frame numbers to file. These can be imported into Vegas or other editor
    WriteFileIf(source_fields,filename, "(AverageLuma(SceneChange)>20)" , "current_frame+1", flush=false)

    When # is put back in on the last line I can see the video showing the scene change. When I remove # the frame numbers doesn't come up. I pressed vdub forward scene button and the frame numbers start to appear.

    How exactly am I suppose to operate the script? Is it automated or do I need to constantly do something?
    Quote Quote  
  13. For any of the writefileif variations (scenechange detection or otherwise) , you can use open the avs in vdub/vdub2 and use file=>run analysis pass and it will write the text file

    You can comment out the script clip line as per the instructions when you're ready to write the actual text file . You toggle one or the other. Scriptclip is just to overlay the information (so you can "see") so you can debug and get more accurate values.
    Quote Quote  
  14. OK done that.

    What does AverageLuma(SceneChange)>20 do?

    Also the scene change between live frame and the next fade out frame isn't detected. Anything i can do?

    #Script to find scene changes

    Loadplugin("C:\Program Files (x86)\AviSynth\plugins64+\mvtools2.dll")

    filename = "E:\currser\BABYLON5\new\scenes.txt"
    BlockChangeThresh = 200 # Default: 350. Increase to reduce number of scenes detected;
    # Determines whether a block has changed from prev frame.
    Num_blocks_changed = 60 # Default: 90. Increase to reduce number of scenes detected;
    # How many changed blocks (given threshhold) must change to trigger scene change

    source=FFVideoSource("E:\currser\BABYLON5\new\Soul Hunter.avi").killaudio().colorYUV(autogain=true) #add this to increase contrast
    source_fields=source.convertTOYV12(interlaced=fals e)
    source_super = source_fields.MSuper(pel=2, sharp=0)

    backward_vec = MAnalyse(source_super,isb = true, delta = 1, blksize=16,search=0)

    SceneChange = MSCDetection(source_fields, backward_vec,thSCD1=BlockChangeThresh,thSCD2=Num_b locks_changed)

    #Uncomment following line for troubleshooting. It puts the scene change threshold numbers on the screen.
    ScriptClip(source_fields,"Subtitle(String(AverageL uma(SceneChange)),align=5)")

    #This line writes frame numbers to file. These can be imported into Vegas or other editor
    #WriteFileIf(source_fields,filename, "(AverageLuma(SceneChange)>50)" , "current_frame+1", flush=false)


    You had Subtitle(String(AverageLuma(SceneChange) ),align=5)") is the space between ) ) important?
    Quote Quote  
  15. Originally Posted by coolgit View Post

    What does AverageLuma(SceneChange)>20 do?

    Also the scene change between live frame and the next fade out frame isn't detected. Anything i can do

    On that version, Use the debug mode with ScriptClip to overlay the info . Use that information to adjust values of BlockChangeThresh or Num_blocks_changed until you get ideal values for your source . It tells you can increase or decrease the value to detect more or less scenes in the comments

    The next frame is written when that threshold value is surpassed. So if you have >20 , then when it prints value >20 , it will write
    current_frame+1 on the text file when writefileif is enabled

    Fades can be difficult to categorize, if you include them and it's a slow fade, you're going to get a lot of false positives (scene change frame number printed, when it shouldn't be) . eg. if a camera moves from a shadowy corridor to a lit corridor, that resembles a type of fade.

    You're supposed to preview the script and adjust the values before you write the text with writefileif - because each source is different, different types of movies, content, styles of scene changes

    If mvtools2 detection is not ideal for a particular source, you can try dozens of other variations. Some might be more accurate than others for a given source. But none will be perfect for everything

    You had Subtitle(String(AverageLuma(SceneChange) ),align=5)") is the space between ) ) important?
    That's John Meyer's post. And no it doesn't matter. (And you can just test it yourself, preview the script with or without the space)
    Quote Quote  
  16. Also, he has coloryuv(autogain=true) in that version

    You might try disabling that if you wanted to include fades, because that could interfere with a fade scene transition (it should brighten the darker frames more than the brighter frames during the fade)
    Quote Quote  
  17. Ok my latest.

    #Script to find scene changes

    Loadplugin("C:\Program Files (x86)\AviSynth\plugins64+\mvtools2.dll")

    filename = "E:\currser\BABYLON5\new\scenes.txt"
    BlockChangeThresh = 200 # Default: 350. Increase to reduce number of scenes detected;
    # Determines whether a block has changed from prev frame.
    Num_blocks_changed = 60 # Default: 90. Increase to reduce number of scenes detected;
    # How many changed blocks (given threshhold) must change to trigger scene change

    source=FFVideoSource("E:\currser\BABYLON5\new\Soul Hunter.avi").killaudio().colorYUV(autogain=false) #add this to increase contrast
    source_fields=source.convertTOYV12(interlaced=fals e)
    source_super = source_fields.MSuper(pel=2, sharp=0)

    backward_vec = MAnalyse(source_super, isb = true, delta = 1, blksize=16, search=0)

    SceneChange = MSCDetection(source_fields, backward_vec,thSCD1=BlockChangeThresh,thSCD2=Num_b locks_changed)

    #Uncomment following line for troubleshooting. It puts the scene change threshold numbers on the screen.
    ScriptClip(source_fields,"Subtitle(String(AverageL uma(SceneChange)),align=5)")
    ScriptClip("subtitle(string(current_frame))")
    ScriptClip("Subtitle(String(YDifferenceFromPreviou s),align=1)")

    #This line writes frame numbers to file. These can be imported into Vegas or other editor
    WriteFileIf(source_fields,filename, "(AverageLuma(SceneChange)>30)", "current_frame", """ ":" """, "current_frame+1", flush=false)

    Last frame and first frame of scene change are printed out correctly using "current_frame", """ ":" """, "current_frame+1"

    I have adjusted the settings to get the desired result. So far so good.

    Just one more stumbling block. Close action is no good for FDC and they will show up like this series of numbers

    19068:19069
    19069:19070
    19070:19071
    19071:19072
    19072:19073
    19073:19074
    19074:19075
    19075:19076

    Is there a way to group this to output to

    19068:19076


    Possible bug somewhere.

    Run video analysis pass goes through the video to the end but the output stopped at 40000ish frames leaving 20000ish frames left to go. I pressed F5 key vdub script editor and then it finishes the job.
    Quote Quote  
  18. Originally Posted by coolgit View Post


    Just one more stumbling block. Close action is no good for FDC and they will show up like this series of numbers

    19068:19069
    19069:19070
    19070:19071
    19071:19072
    19072:19073
    19073:19074
    19074:19075
    19075:19076

    Is there a way to group this to output to

    19068:19076
    Not sure with internal avisynth functions; maybe with some macro or other spreadsheet function. You can also ask at doom9


    Possible bug somewhere.

    Run video analysis pass goes through the video to the end but the output stopped at 40000ish frames leaving 20000ish frames left to go. I pressed F5 key vdub script editor and then it finishes the job.

    Are you commenting out ("#") and disabling the scriptclip lines for the actual writing runs ? scriptclip can cause memory and other issues

    Did you try avsr64 (to rule out if it's vdub issue) ? It's slightly faster than vdub analysis pass, and it's easier to batch run concurrent instances (simultaneous execution of multiple scripts if you have enough memory/resources)

    Code:
     
    avsr64 script.avs
    Quote Quote  
  19. Sorted it by closing vdub2 and reopening it and then run the script straight away without any changes to the script codes.
    Quote Quote  



Similar Threads