VideoHelp Forum
+ Reply to Thread
Results 1 to 4 of 4
Thread
  1. Hi, I'm playing a bit around with vsCodeFormer.
    I start out with suzie_lossless and then apply 'SwinIR-L x4' and vsCodeFormer, which frame-by-frame looks nice (see: SwinIR+CodeFormer_lossless).
    The problem is the temporal instability of the process.

    Anyone got suggestions on how to stabilize this flickering?
    (QTGMC maybe?)

    Cu Selur
    Image Attached Files
    Last edited by Selur; 2nd Apr 2023 at 07:27.
    users currently on my ignore list: deadrats, Stears555
    Quote Quote  
  2. The temporal aliasing is a common problem with single image algorithms, especially upscaling ones

    The differences in prediction each frame are too large in that example. QTGMC inputtype 1/2/3 can only do so much, and there are usually side effects incl. detail loss, possible ghosting . If you stack some temporal filters to combat the temporal aliasing/inconsistencies , you begin losing too much detail and blurring everything to mush

    Another approach is to pre scale with something temporally consistent. You can experiment BasicVSR++ or Tecogan somewhere in the chain, either before /after codeformer, +/- downscaling

    Here is an experiment with BasicVSR++ 4x => bicubic downscale /2 => GPEN *2 => QTGMCp inputtype=1 , compared to lanczos3 at 1/2 speed. I like the finer hair details improved on BasicVSR++ . I don't like that the BG textures are smoothed away, eye specular reflections are way too enhanced, contrast and saturation changed (in this example, they were crudely matched back) , other small details smoothed away compared to BasicVSR++ alone

    Not a fan overall of the "look" of codeformer , or most "face" upscalers - I find that they change the "facts" and actual details too much. In that codeformer example, the hair (e.g. hair style changes, curls are straighened), eyes (eyes change, look almost transplanted), nose & lip shape change too much

    My cutoff for acceptable tolerance is if the image is "plausible" and similar in underlying structure compared to say a lanczos4x
    Image Attached Files
    Quote Quote  
  3. Yeah, BasicVSR++ often simply is too smooth/aggressive and codeFormer for normal content only is useful when the weight is near 1.
    Here's an example of what weight 0 can do: https://imgsli.com/MTY2NDI2 *gig*

    Cu Selur
    users currently on my ignore list: deadrats, Stears555
    Quote Quote  
  4. What would be ideal is training a model with the same person for inference. e.g. if you were doing a family member or some famous person - train a specific model using other videos of the same family member or famous person . The temporal flicker would still be there for single image algorithms to some extent, but at least the face would match
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!