VideoHelp Forum

+ Reply to Thread
Results 1 to 10 of 10
Thread
  1. Hi,
    I have two sources, a 720p50 and a 1080p50 stream, 1hr long.
    These are identical frame to frame for the most part.

    The 1080p50 stream has sporadic artifacting (bright white macro blocks).
    This occurs couple of times a minute or so.

    I would like to replace the broken frames in the 1080p50 stream with the 720p50 frames (as the 1080p50 is much sharper otherwise).
    Two ways I can see to do this:
    Extract all the frames to images (bmp or png?), manually replace the broken ones. (This will require a lot of storage and setup time)
    Use an NLE to replace the frames when necessary, but picking out an individual frame will likely be difficult. (Kdenlive or resolve)

    Alternatively some kind of script that would allow me to press a button and save a frame number to a file would at least speed it up significantly. (might be possible with mpv?)

    If anyone can suggest a more streamlined workflow or tool for this that would be great.

    Thanks.
    Quote Quote  
  2. Make sure it's not some decoding error first, eg. try different players, mpv, mpchc, ffplay etc...


    Originally Posted by ajingo View Post
    The 1080p50 stream has sporadic artifacting (bright white macro blocks).
    This occurs couple of times a minute or so.

    How many consecutive frames do the "bright white macro blocks" occur in? How much area do they cover per frame?

    Use an NLE to replace the frames when necessary, but picking out an individual frame will likely be difficult. (Kdenlive or resolve)

    Why would it be difficult ?

    A NLE would be the method of choice for most people that do this manually

    Alternatively some kind of script that would allow me to press a button and save a frame number to a file would at least speed it up significantly. (might be possible with mpv?)
    How would that speed it up? If you save a frame number you still have extra steps to perform to replace frames

    Alternatively you can use avisynth or vapoursynth and specify the frames to replace using RemapFrames. avspmod has a macro to insert current framenumber.py, and you can map it to a hotkey (so you don't have to type frame numbers) - so you could play/preview or go frame by frame and write the replacement script at the same time


    If the defect frames are very different from surrounding frames, lasting 1 frame, you could probably script automatic detection and replacement using conditionalfilter

    Or if the brightness is very different than "normal" frames, you could probably script automatic detect/replace as well
    Quote Quote  
  3. Make sure it's not some decoding error first, eg. try different players, mpv, mpchc, ffplay etc...
    Sadly not a decoding error. It definitely appears like a decoding error further up the chain (Such as terrestrial TV when the signal is intermittent and the motion vectors get applied to the wrong I frame) but it is baked into my source.

    How many consecutive frames do the "bright white macro blocks" occur in? How much area do they cover per frame?
    1 to 4 frames, anywhere from 10% to 90%.

    Why would it be difficult ?

    A NLE would be the method of choice for most people that do this manually
    It's a lot of clicks for each frame to replace - of which there's a lot.
    For each frame -
    1. Hide 1080p50 track
    2. Right click -> Export Image to Project
    3. Click Save
    4. Drag Image from Project Bin to Timeline
    5. Show 1080p50 track


    How would that speed it up? If you save a frame number you still have extra steps to perform to replace frames
    If I were replacing images in a folder I could make a list of frames to replace and then write a quick script to replace all listed frames, and as you say this should work for AviSynth as well. This means I can play back the video normally, pressing "x" when I find a corrupted frame and continue.

    Then with the frame numbers I can use a script to replace.
    "insert current framenumber.py" looks like it should work well for this.
    Otherwise I might try adapting this .

    If the defect frames are very different from surrounding frames, lasting 1 frame, you could probably script automatic detection and replacement using conditionalfilter

    Or if the brightness is very different than "normal" frames, you could probably script automatic detect/replace as well
    The artifacts at times can be quite subtle, and the video is full of flashing lights to trip up detection.
    Normally I end up shooting myself in the foot when I try heuristics anyway.

    Thanks for the pointers.
    Quote Quote  
  4. Originally Posted by ajingo View Post
    A NLE would be the method of choice for most people that do this manually
    It's a lot of clicks for each frame to replace - of which there's a lot.
    For each frame -
    1. Hide 1080p50 track
    2. Right click -> Export Image to Project
    3. Click Save
    4. Drag Image from Project Bin to Timeline
    5. Show 1080p50 track
    Well if you do it that way it's much slower. Why would you export individual frames ?


    The way you would do this is slightly different in different editors -

    One way is : Put 720p50 on top layer (scaled to 1080) . Hide the visibility of the layer, so you only see the bottom 1080 layer. when you see bad frames on the bottom 1080 layer, make cuts on the scaled 720 top layer to "cover up" the bad frames on the bottom 1080 layer. Another way you can do it is with opacity keyframes. 100% when you want top layer to show, 0% when you want bottom to show. Set keyframes to no interpolation (or on/off) - basically you're toggling visibility of the layer

    Or multicam edit. The 2 "cams" are 720(scaled to 1080) and the 1080 version. This would probably be the easiest/fastest/ least number of keys/ and most interactive for most people (hotkey switches the cam), but not all free versions have multicam editing, I can't recall if the non studio version of Resolve has it





    How would that speed it up? If you save a frame number you still have extra steps to perform to replace frames
    If I were replacing images in a folder I could make a list of frames to replace and then write a quick script to replace all listed frames, and as you say this should work for AviSynth as well. This means I can play back the video normally, pressing "x" when I find a corrupted frame and continue.

    Then with the frame numbers I can use a script to replace.
    "insert current framenumber.py" looks like it should work well for this.
    Otherwise I might try adapting this .
    You don't need images, or to replace images in a folder when you use avisynth or vapoursynth - everything is frameserved and referenced to the video(s). You don't need a "physical" image sequence or to waste any space

    Just like you don't need images for a NLE...you can do it directly in a NLE
    Quote Quote  
  5. One way is : Put 720p50 on top layer (scaled to 1080) . Hide the visibility of the layer, so you only see the bottom 1080 layer. when you see bad frames on the bottom 1080 layer, make cuts on the scaled 720 top layer to "cover up" the bad frames on the bottom 1080 layer. Another way you can do it is with opacity keyframes. 100% when you want top layer to show, 0% when you want bottom to show. Set keyframes to no interpolation (or on/off) - basically you're toggling visibility of the layer
    I did explore this as an option, but for doing single frame cutting then zooming in to toggle visibility (too small to click otherwise) then zooming out took more precise control than exporting frames, which I managed more quickly.

    Or multicam edit. The 2 "cams" are 720(scaled to 1080) and the 1080 version. This would probably be the easiest/fastest/ least number of keys/ and most interactive for most people (hotkey switches the cam), but not all free versions have multicam editing, I can't recall if the non studio version of Resolve has it
    KdenLive does support Multicam but all it appears to do is provide a grid of images that when clicked on will jump to the correct track.

    You don't need images, or to replace images in a folder when you use avisynth or vapoursynth - everything is frameserved and referenced to the video(s). You don't need a "physical" image sequence or to waste any space
    Yeah, I forgot to mention my intention to use RemapFrames, I just have to decipher the file format first.
    It seems that "10 10" would replace frame 10 of A with frame 10 of B.
    My first instinct was the folder of frames as that was how upscaling with some of the ML stuff used to be done.


    If I was doing scene by scene or similar scale, a NLE would be the obvious choice, it's just that at single frames the scale is so small and frequent it becomes unweildy (at least for me) to use a NLE for this task.
    Basically, more than one key press for each frame was more than I was willing to do. I only have so much dedication.
    Last edited by ajingo; 25th Sep 2021 at 17:30.
    Quote Quote  
  6. Wait... Did you mean the frames don't align ? I assumed when you said "These are identical frame to frame for the most part." that you meant they were identical , except the 1080 version has some artifacting ?

    Another option to remapframes is clipclop. You might like the syntax better (or not)
    Quote Quote  
  7. Assuming you have videos that align:

    If you have only a few "bad" frames, but mostly "good", it's better to swap the tracks, 1080 on top, 720 on bottom. You're poking "holes" in the top track for the 720 version to appear.

    I don't understand why you need to zoom at all ? You actually don't have to toggle visibility, and you can edit by keyboard shortcuts . Hit play or frame step, whenever you see bad frame, mark in, mark out when frames end. Delete. Go to next... repeat


    Not sure how kdenlive does multicam edit, but that's basically what you want, and it stays on the current track until you toggle it. The hotkey is toggling the "cam", ie. camera switching. e.g. if the "1" key is 720, "2" key is 1080 , you start with "2". The entire video is 1080 until you hit "1" , then from then on it's 720, move the playhead as soon as you hit 2, it switches back to 1080. In some editors, you have multiple views - so one is looking at say the 1080 version only, the other is looking at the final multicam edit. So you use the 1080 view to guide the multicam edit. You go along, once you see a bad frame you hit 1, if there is more than one, keep on going, as soon as 1080 is ok, you hit 2 (it switches back to the 1080 track). This is interactive and the fastest/ fewest number of keystrokes by a large margin if your videos align, because you don't have to mark in/ out or delete - everything is "built in" to that 1 key stroke
    Quote Quote  
  8. Upload a sample of the video with "bright white macro blocks" and an overlapping segment from the good 720p video. That will help determine if the fix can be automated.
    Quote Quote  
  9. And if you're doing the replace in avisynth route, ReplaceFramesSimple is probably easier, if the videos align , because only 1 number is needed for singe frame replacement; so if you combine with the insert current framenumber.py hotkey, it's faster than RemapFrames
    Quote Quote  
  10. Alternatively some kind of script that would allow me to press a button and save a frame number to a file would at least speed it up significantly.
    Sawbones from master StainlessS helped me a great deal to salvage footage recorded with a camera which had a defective stabilizer about 50000 frames to examine one by one, with thousands of blurry frames to fix, seemed like a nigh impossible task, but using Sawbones (based on AutoIt if I remember correctly) a keyboard shortcut (rather awkward I must say flexible fingers are a must !) would write a command for FrameSurgeon with the current frame number to a text file opened in Notepad, which was a huge time saver (other text editors can be used as well after some tweaking, I would recommend one with an auto-save feature, like the excellent TED Notepad). If the 1080p50 version is that much better, and if the frames which need to be fixed are mostly static and/or not too many at a time (beyond 3-4 contiguous frames the interpolation can look weird unless there's very little motion), you might want to use frame interpolation instead, and FrameSurgeon is an excellent interpolation plugin for Avisynth (I've tested quite a few, FrameSurgeon usually gave the best result, except for tricky frames with a lot of motion going on, for which I used either Morpheus also from StainlessS or the more basic frame blending function Morph). Once you have the list of interpolation commands you can tweak the script and preview the outcome with AVSPMod. Then you can encode it directly with ffmpeg or any GUI video converter that recognizes Avisynth scripts (MeGUI and Staxrip for sure, perhaps others will be suggested). Another upside is that it gives you a better control over the compression quality (in particular, x264 is generally considered significantly more efficient than the H.264 encoders implemented in video editors, usually from Mainconcept, meaning that the quality loss will be less for a similar file size, or the file size will be less for a similar quality).

    EDIT : To get Sawbones / FrameSurgeon, as it's not made very clear in the linked Doom9 thread :
    https://www.mediafire.com/folder/hb26mthbjz7z6/StainlessS
    Last edited by abolibibelot; 4th Oct 2021 at 02:25.
    Quote Quote  



Similar Threads