Any possibility of repairing this video? (screenshots in link)

[newtovideo]

I recorded footage using a capture card and OBS studio. I've recorded over 70 videos previously without issue, but this time I made a typo when adjusting the resolution resulting in a severely distorted video. Screenshots linked here, or see attachments: https://imgur.com/a/g3MoRtU

I know the resolution was the issue, because I tried correcting the resolution settings and recorded a test video which came out fine. For detail, the resolution I entered was 3820x2160 when the source was 3840x2160. The video is in .mkv format, and converting it to .mp4 does not change anything.

Not sure if there's any way to salvage and repair this footage, but would be very grateful if anyone has any ideas. Thank you in advance.

[jagabo]

It looks like the codec you used does't support 4:2:2 or 4:2:0 chroma subsampling at a mod 4 frame size. It may be possible to fix. What coded did you use? Can you upload a video sample (don't reencode)?

[newtovideo]

It looks like the codec you used does't support 4:2:2 or 4:2:0 chroma subsampling at a mod 4 frame size. It may be possible to fix. What coded did you use? Can you upload a video sample (don't reencode)?

Hi jagabo! Appreciate the response. I used the x264 video encoder. Here's a link to a sample clip from the video. It's also in the attachments below.

https://imgur.com/a/jO6v1wG

I also shared some screenshots of the color settings I'm using in case you have any feedback. Thanks again for having a look!

[jagabo]

This is pretty close:

LWLibavVideoSource("sampleclip1.mp4", cache=false, prefer_hw=2) 

Crop(10,0,-10,-0)
StackHorizontal(last, last, last)
src = last

HShear(63.45)

MergeChroma(src, last)
ChromaShiftSP(x=2158)

Crop(0,0,width/3, height)
#add back borders if you want

[Darth Moore]

Dear Jagabo,

May I ask if this trick would work in case of our problem, or there would be any solution to fix this ?:


[Attachment 80575 - Click to enlarge]
[Attachment 80576 - Click to enlarge]
(2nd screenshot need to open large to see the same problem can be easily spotted on the black areas of first screenshot)

We recorded a stratospheric flight with my Son, as Father and Son project, but one of the camera produced these strange "red random straps" all over and ruin the spectacular recording. I am wondering if would be possible to fix this somehow, as impossible to repeat it and would make my Son and of course me very very happy.

I had uploaded the the first chunk of the video as well here:

https://master365office-my.sharepoint.com/:v:/g/personal/admin_master365office_onmicro...oR2cw?e=IU3QAu

May I ask if you could have a look at it in hope we could fix this ? (we are not experienced in such video manipulation, so would be lost without help.

VERY much appreciate your time and help in advance!

[Selur]

Nice! I'm impressed. Chapeau!
(for those wondering, HShear comes with http://avisynth.nl/index.php/Rotate)

[jagabo]

Thanks, Selur. I forgot to mention hShear() was included with the Rotate filter. And ChromaShiftSP() can be found here:

http://www.avisynth.nl/index.php/ChromaShiftSP

[Selur]

May I ask if this trick would work in case of our problem, or there would be any solution to fix this ?:

No, rotating the chroma and overlaying multiple copies of it and shifting the chroma won't help with your clip.

We recorded a stratospheric flight with my Son, as Father and Son project, but one of the camera produced these strange "red random straps" all over and ruin the spectacular recording.

Seems like there are issues in the V channel throughout the whole clip.
Best look at the YUV channels separately to better see the problem.
Try some heavy denoisers or artifact removers on the V channel. (multiple DeScratch calls probably can help with the lines)

[Darth Moore]

Dear Selur,

Many thanks for your quick reply and help!

May I ask if you could guide us how to do this: "Try some heavy denoisers or artifact removers on the V channel.". We are not experienced in video editing.

BIG thank you for any help and sorry for the newbie question.

[Selur]

Sorry, on my way to bed, but here's what multiple (here 4) Descratch calls

DeScratch(clip=clip,mindif=1,mindifuv=1,modey=0,modev=3,maxwidth=7,minlen=1,left=20)

can do: https://imgsli.com/Mjc4MTE1

Cu Selur

[jagabo]

I've got something that removes a lot of the stronger red lines. But not the fuzzier blotches. I will continue playing with it but here's a sample with a crop from the original video on the left, the cleaned version on the right.

Basically, I looked for vertical edges and reddish colors in the video. Anywhere that had both those properties had the V channel replaced with a blend of colors taken from the left and right.

[Darth Moore]

Oh God, so much better already, we are so tanksful, you cant imagine! If may there be a solution, what we could ran over all the MOV files recoded by this camera, would be a true miracle. Thank you so so so much ! Waiting for your update with all of fingers crossed for luck. Once again, thank you!!!

[jagabo]

Here's a longer clip (same crop) where I also desaturated dark areas. Some of the bluish black areas have become pure black. You can see the filtering doesn't work as well on some parts of the video. I can desaturate the brightest areas too but the blue sky may turn white. If there are any areas of the video you are particularly interest in let me know. I'll take a look at those areas.

[Darth Moore]

BIG thank you again, amazing result!!! I think this is a huge step forward.

May I ask how can we reproduce this step by step, so we can ran on the 4+ hours recording ? Or this need to be down manually at every scene ?

I sent you the complete video in private msg, but not to make you the hard work as we are not lazy, just to show you other critical areas you asked, like this:


[Attachment 80633 - Click to enlarge]
[Attachment 80634 - Click to enlarge]
[Attachment 80635 - Click to enlarge]
[Attachment 80636 - Click to enlarge]
[Attachment 80637 - Click to enlarge]

Many many many thanks in advance!!

[jagabo]

You'll need to install 64 bit AviSynth+

https://avs-plus.net/

And a few third party filters (64 bit versions go in AviSynth+'s plugins64+ folder):

http://avisynth.nl/index.php/Masktools2
https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

Then copy the following script to a text file named with the extension .AVS instead of .TXT.

###############################################################################################
#
# An alpha mask based on Y.  Like mt_binarize except has optional range for softer transitions
#
###############################################################################################

function YMask(clip v, int "min", int "max")
{
    min = default(min, 128)
    max = default(max, min)

    Levels(v, min, 1, max, 0, 255, coring=false).GreyScale()
}


###############################################################################################
#
# Desaturate UV in bright and dark areas
#
###############################################################################################

function DeSaturateBrights(clip c, int low)
{
    ymask = YMask(c, low, 235)
    Overlay(c, ColorYUV(c, cont_u=-256, cont_v=-256), mask=ymask)
}

function DeSaturateDarks(clip c, int high)
{
    ymask = YMask(c, 16, high).Invert()
    Overlay(c, ColorYUV(c, cont_u=-256, cont_v=-256), mask=ymask)
}


###############################################################################################

LWLibavVideoSource("FILE240709060421F.MOV") 
src = last

# build a mask of high V (as in  YUV) areas
cmask = VtoY().BilinearResize(width,height).ColorYUV(off_y=-128).ColorYUV(gain_y=10000)

# red vertical lines will usually have the normal colors to the left and right
# get an average of those colors to replace the red lines
shiftleft = Crop(6,0,0,0).AddBorders(0,0,6,0)
shiftright = Crop(0,0,-6,0).AddBorders(6,0,0,0)
shiftcenter = Merge(shiftleft, shiftright)

# find sharp vertical edges in the V channel
edges = VtoY().Sharpen(1.0, 0.0).mt_edge(mode="-1 2 -1   -1 2 -1  -1 2 -1", thy1=5, thy2=5).BilinearResize(width, height).mt_expand()#.Blur(1.0).Blur(1.0).Blur(1.0)

# fix only areas that are red and have a vertical line in the V channel
fixareas = Overlay(cmask, edges, mode="multiply")
Overlay(last, shiftcenter, mask=fixareas)

# Use the original luma and the new chroma
MergeChroma(src, last)

DesaturateDarks(50)

Then you can open the script with any editor/encoder that supports AviSynth script. For example VirtualDub2, x264 CLI, ffmpeg...

[Darth Moore]

Ok, so:

1) downloaded and installed:

[Attachment 80638 - Click to enlarge]

2) downloaded:

[Attachment 80639 - Click to enlarge]

[Attachment 80640 - Click to enlarge]

and moved to plugin dir:

[Attachment 80641 - Click to enlarge]

3) downloaded and started VirtualDub2_44282

4) Tools/Script editor, I loaded your previously saves script:

[Attachment 80642 - Click to enlarge]


May I ask what to do next? I mean I tried to play the video and export the video, but it doesnt seems to apply your script on it, so being a newbie, I am surely missing some simple trivial step, which very much apologize for, but could you point me to what should be done to get save the video applying your script but keeping the original video framerate, size, etc ?

Many many thanks for your patience and help!

[jagabo]

You want just the dlls for masktools2 and LSMASH in the plugins64+ folder, not in subfolders therein. Ie, look in the x64 folder for the masktools2.dll and move it up to the plugins64+. Same for LSMASHSource.dll.

Open the AVS script in AviSynth as if it a video file: File -> Open Video File...

[Darth Moore]

Thank you ! I moved .dll s and loaded AVS file (after I edited the video name in it) and seems to work. THANK YOU !

There are only 2 areas, which still very problematic, the ground greenish, brownish and the water blueish, greenish area, like these imagesbelow.
Since these are the key in this video, may I ask if you could have a look at it, when you have a spare moment?
We dont want to take advantage on your kindness, just you are literally the last hope to have this fixed.
(the full video link is in your forum inbox, where you can see these parts as being 70% of the total video, since this camera was the bottom camera.)


[Attachment 80643 - Click to enlarge]


[Attachment 80644 - Click to enlarge]

Big heartfelt thank you !

[jagabo]

I'll see what I can do with those remaining areas. It will probably be a few days before I can start. Can you give me some frame numbers or time stamps so I don't have to hunt?

[Darth Moore]

Many many thanks in advance, VERY much appreciate it !!! Have a lovely day!

[Darth Moore]

Good day to you, may I ask kindly, if you had any chance to look at the video ?

Not to rush you by any means, just didn't gave up the hope and I was not exaggerated when I said, this is our last hope.

Oh, and please forgive us, we just realized you asked some time stamps. No idea how did we overlook it. Here they are sir:

00:32:30
00:50:04
02:32:27
02:59:19
03:25:12

Unfortunately, no need to hunt for it as it is all over throughout the whole footage, when the balloon is in the air and filming the ground and the lake. So you will see it. Lines, spots and mixed forms. I know it sounds stupid, but it seems, when we play the video multiply times, the problems shows different forms, but maybe this is just our eyes as we looking too much. LOL.

Big heartfelt thank you for your time, help in advance!

[jagabo]

The earlier script gets rid of the strong, thin, red lines in those areas. But not the weaker reddish blotches. I tried several things but they damaged the rest of the picture too much. I may try a few more things over the next several days...

[Darth Moore]

Many many many thanks! VERY much appreciated!!

[rgr]

This is pretty close:

LWLibavVideoSource("sampleclip1.mp4", cache=false, prefer_hw=2) 

Crop(10,0,-10,-0)
StackHorizontal(last, last, last)
src = last

HShear(63.45)

MergeChroma(src, last)
ChromaShiftSP(x=2158)

Crop(0,0,width/3, height)
#add back borders if you want

Good. I wonder why it doesn't work for me.

After crop:
https://i.imgur.com/jZZOyNr.jpeg

After stack:
https://i.imgur.com/ygvgxt5.jpeg

After SHear:
https://i.imgur.com/tSLEIaB.jpeg

After MergeChroma:
https://i.imgur.com/KD5YqQR.jpeg

After ChromaShiftSP:
https://i.ibb.co/tPS1PL7/image.png

[jagabo]

Your ChromasShiftSP is shifting the chroma too far to the left. Maybe a version issue? Or color format issue? Did you resize the frame at some point? Just use a lower value for the shift.

[rgr]

No, I didn't change anything.

http://avisynth.nl/index.php/ChromaShiftSP

#ChromaShift_SP: Shift chroma with subpixel accuracy, basic function by IanB, made standalone by McCauley

function ChromaShiftSP (clip clp, float "X",float "Y") {

X = default(X, 0.0) # positive values shift the chroma to left, negative values to right
Y = default(Y, 0.0) # positive values shift the chroma upwards, negative values downwards

w = clp.Width()
h = clp.Height()

clp.MergeChroma(clp.Spline16Resize(w, h, X, Y, w+X, h+Y)) }

[rgr]

Your ChromasShiftSP is shifting the chroma too far to the left. Maybe a version issue? Or color format issue? Did you resize the frame at some point? Just use a lower value for the shift.

Before ChromaShift it looks like the shift should work. But it ends up like this:

https://i.imgur.com/ZSLBwN4.jpeg

LWLibavVideoSource("sampleclip1.mp4", cache=false, prefer_hw=2)

Crop(10,0,-10,-0)
StackHorizontal(last, last, last)
src = last

HShear(63.45)

MergeChroma(src, last)
#ChromaShiftSP(x=2158)
ChromaShiftSP(x=2058)

Crop(0,0,width/3, height)
#add back borders if you want

[rgr]

OK, I suppose Spline (in ChromaShiftSP.avsi) causes a mess, but I haven't analyzed it.

MergeChroma(Crop(0,0,width()-2158,height()), Crop(2158,0,width()-2158,height()))

[jagabo]

Your chroma planes are too narrow for the luma plane.

I just looked at the ChromaShiftSP() I have installed. There's a note in it that I fixed a bug. You can see the fix on the last line:

# bad shifting fixed by JEH 2023/6/3
#ChromaShift_SP: Shift chroma with subpixel accuracy, basic function by IanB, made standalone by McCauley


function ChromaShiftSP (clip clp, float "X",float "Y") {

X = default(X, 0.0) # positive values shift the chroma to left, negative values to right
Y = default(Y, 0.0) # positive values shift the chroma upwards, negative values downwards

w = clp.Width()
h = clp.Height()

#clp.MergeChroma(clp.Spline16Resize(w, h, X, Y, w+X, h+Y)) } # original
clp.MergeChroma(clp.Spline16Resize(w, h, X, Y, w, h)) } # JEH fix

[rgr]

Impossible to detect if someone doesn't know the details of how the Resize function works.
You can update the thread
https://forum.doom9.org/showthread.php?t=139959
(the wiki has the wrong version)