Manually fix pan, shift pixels, with Avisynth

[lordsmurf]

I have a damaged video, where bad frames had to be duplicated and removed.
It looks good now, aside from a small jerks of a few pixels.
So I want to shift pixels to repair.

For example, frames 0-21677,
new frame 21678-30878 can all be shifted vertical 3 pixels,
new frame 30879-50001 can all be shifted left 4 pixels

I don't care about pixels lost on frame edges, the central content is what matters.

I've looked around, but I have no guess on what to even try here.

[poisondeathray]

a static / non changing shift ? how about crop and addborders ?

[lordsmurf]

a static / non changing shift ? how about crop and addborders ?

Correct, static, non-changing.

Crop + AddBorder was my thought as well. But how to incorporate that for frame ranges? (I have a feeling it'll be some of that backwards variable scripting, which I have a hard time wrapping my head around.)

[poisondeathray]

If it's just 3 sections , use trim() to specify ranges. Aligned splice to stick them back together (A ++ B ++ C)

Or use remapframes if you have more sections or types of filtered sections . Or other options like clipclop . Trim is the fastest and easiest IMO for a few sections

[manono]

I do such things using ReplaceFramesSimple (part of the RemapFrames filter). For example, "new frame 30879-50001 can all be shifted left 4 pixels"

A=Last
B=A.Crop(0,0,-4,0)
B=B.AddBorders(4,0,0,0)
B=B.BorderControl(XLS=4,XLSF=4)
ReplaceFramesSimple(A,B,Mappings="[30879 50001] ")

BorderControl gets rid of the black at the edge. There are other border filters also, such as FillBorders.

[johnmeyer]

Agree with manono (and others): crop/addborders. StainlessS' RTStats is your friend for things like this. He's also worked out many ways to operate on ranges of frames. You might want to check out his CallCmd v1.04 - x86 & x64 -Execute command. From his description: "Plugin to execute command on selectable frames or at startup or closedown."

[_Al_]

btw these sorta things make a difference between avisynth and vapoursynth. If knowing python and using vapoursynth, in no time you just create mapping yourself, meaning using a filter for certain frame range, because its language has a tools to do that. Easily you even overlap those ranges for different filters.

[poisondeathray]

vertical 3 pixels might be an issue depending on your chroma subsampling, and interlaced vs. progressive

http://avisynth.nl/index.php/Crop#Crop_restrictions

[manono]

vertical 3 pixels might be an issue depending on your chroma subsampling, and interlaced vs. progressive

Yep, that's why I chose the 4-pixel one to demonstrate.

[lordsmurf]

I do such things using ReplaceFramesSimple (part of the RemapFrames filter). For example, "new frame 30879-50001 can all be shifted left 4 pixels"

Hmm... I'm not sure how to add that in.

This video has some nasty jitter issues that really mess up some sections. The overall scene is fairly static, fairly wide angle scene, and you're watching the presentation. Any jitter is extremely distracting, somewhat breaks the mood, so hard to enjoy.

I'm having to take some aggressive stabilizations, tested QTGMC settings quite a bit, and then replace frames.

Because the fixed-jitter areas have some obvious issues, I think the best solution here is to drop framerate, make it less noticeable.

After correcting the panning problem, I may attempt to double frame rate. Then take the video, run it without stablization or replacements, to get good sections at 59.94. Then merge it with the fixed half>double framerate bad sections. I will need to re-run a pixel shift, to match repair version. This may give the best viewing.

I've had this on my drive for 6 years now. It's time.

Here's my script so far.

avisource("N:\FS.avi")
ConvertToYV12()
StabMod()
QTGMC(Preset="Faster")
#QTGMC(Preset="Fast", SourceMatch=3, Lossless=2, MatchEnhance=0.75, TR2=1, Sharpness=0.1)
StabMod()
RemapFrames(mappings="[21677 21677] [21676 21676]")
RemapFrames(mappings="[21678 21678] [21679 21679]")
RemapFrames(mappings="[28418 28419] [28417 28417]")
RemapFrames(mappings="[28420 28422] [28423 28423]")
RemapFrames(mappings="[30874 30875] [30873 30873]")
RemapFrames(mappings="[30873 30873] [30872 30872]")
RemapFrames(mappings="[30876 30878] [30879 30879]")
RemapFrames(mappings="[30879 30879] [30880 30880]")
RemapFrames(mappings="[30881 30882] [30883 30883]")
SelectEven
RemapFrames(mappings="[15438 15438] [15437 15437]")
RemapFrames(mappings="[15437 15438] [15436 15436]")
RemapFrames(mappings="[15439 15439] [15440 15440]")
RemapFrames(mappings="[15440 15440] [15441 15441]")
ConvertToYUY2()

What I was hoping for was something similar
example: ShiftFrame([15438-22222] 4px) ... or something.

This script style confuses me, I don't know what's happening here. And for now, forget the border fixes. That doesn't matter, adding to my confusion.

A=Last
B=A.Crop(0,0,-4,0)
B=B.AddBorders(4,0,0,0)
B=B.BorderControl(XLS=4,XLSF=4)
ReplaceFramesSimple(A,B,Mappings="[30879 50001] ")

Let's assume I take the output of the above script, new AVI file.
What does a full script look like?

AviSource("N:\FS-output.avi") 
???

Two sections need to be pixel shifted.

[johnmeyer]

I'd be happy to take a look at the jitter problem, if you want to share a few seconds of footage. You know your stuff, so I assume you've ruled out field reversal. It is quite common.

[lordsmurf]

The jitter damage is blended/ghosted.
At first, I thought it was NR creating the blending, but a recapture with NR off changed nothing.
I highly doubt the TBC was causing it, but I can recapture the tape again. It's been years, but I probably tried this already.
It's not clean fields.
The jitter moves across several lines. One frame is violent, massive jump of lines on both fields.
And yet, I don't think the error is baked in from the master. I think it's this tape.

[poisondeathray]

For 3 segments, it's easier to do with Trim().

Think of it as mixing and matching versions. 3 versions of the video. (1) Original, (2) A shifted left 4 , (3) A down shifted 3 (I'm assuming "down" since "vertical" is not clear) . In this example, they are called orig, SL4, SD4 . You can call them whatever name you want

3px vertical won't work for 4:2:0 subsampling; you have to convert to 4:4:4 for that step. I assume you have a YV12 progressive source. Interlaced 444 still has mod2 restrictions, so it won't work. Interlace is the problem.

You can write a helper function to shift up/down/left/right . Just a shorthand for crop and resize. I included them below.


In this example I use the framenumbers in the 1st post

#progressive YV12 source
orig=last

SL4 = orig.shiftleft(4)
SD3 = orig.ConvertToYV24().shiftdown(3).ConvertToYV12()

orig.trim(0,21677) ++ SD3.trim(21678,30878) ++ SL4.trim(30879, 50001)





function ShiftLeft(clip c, int x)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(x,0,0,0,true).addborders(0,0,x,0)
}

function ShiftRight(clip c, int x)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(0,0,-x,0,true).addborders(x,0,0,0)
}

function ShiftDown(clip c, int y)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(0,0,0,-y,true).addborders(0,y,0,0)
}

function ShiftUp(clip c, int y)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(0,y,0,0,true).addborders(0,0,0,y)
}

[jagabo]

Or, if you don't like lots of named streams:

WhateverSource("filename.ext")

trim(0,21677) ++ trim(21678,30878).shiftdown(3) ++ trim(30879, 50001).shiftleft(4)

[lordsmurf]

I'm assuming "down" since "vertical" is not clear

It's actually up. I'm not sure of the exact pixel offset, 3 was an intentional odd number to see if odd caused issues. It's not longer interlaced, so I'm assuming I could just change to 444. Or are those filters YV12/YUY2 only?

Interlace is the problem.

It's always a problem with restoration, isn't it?

For 3 segments, it's easier to do with Trim().

Think of it as mixing and matching versions. 3 versions of the video. (1) Original, (2) A shifted left 4 , (3) A down shifted 3 (I'm assuming "down" since "vertical" is not clear) . In this example, they are called orig, SL4, SD4 . You can call them whatever name you want

3px vertical won't work for 4:2:0 subsampling; you have to convert to 4:4:4 for that step. I assume you have a YV12 progressive source. Interlaced 444 still has mod2 restrictions, so it won't work. Interlace is the problem.

You can write a helper function to shift up/down/left/right . Just a shorthand for crop and resize. I included them below.

In this example I use the framenumbers in the 1st post

#progressive YV12 source
orig=last

SL4 = orig.shiftleft(4)
SD3 = orig.ConvertToYV24().shiftdown(3).ConvertToYV12()

orig.trim(0,21677) ++ SD3.trim(21678,30878) ++ SL4.trim(30879, 50001)


function ShiftLeft(clip c, int x)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(x,0,0,0,true).addborders(0,0,x,0)
}

function ShiftRight(clip c, int x)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(0,0,-x,0,true).addborders(x,0,0,0)
}

function ShiftDown(clip c, int y)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(0,0,0,-y,true).addborders(0,y,0,0)
}

function ShiftUp(clip c, int y)
#positive integer values only
#no chroma subsampling mod restrictions, or interlaced error checking built it
{
  c.crop(0,y,0,0,true).addborders(0,0,0,y)
}

I shall see how this goes soon. Will post back either way.

Or, if you don't like lots of named streams:

WhateverSource("filename.ext")

trim(0,21677) ++ trim(21678,30878).shiftdown(3) ++ trim(30879, 50001).shiftleft(4)

Thanks for this.

Seeing both versions of a script helps my brain comprehend the backward variable version. The non-variable version just seems so much easier, more elegant, if not creating some massive multi-callback sort of script. (johnmeyer's script drives me nuts to edit, but I think it at least has multiple calls to variables, so it makes sense there.)

"last" implies it scans the entire video to the end, and then returns to start position

I understand functions. I sometimes write or edit those.

[jagabo]

"last" implies it scans the entire video to the end, and then returns to start position

"last" is just the default name used whenever you don't specify a stream name. So

AviSource("filename.avi")
TDecimate()

is just shorthand for

last = AviSource("filename.avi")
last = TDecimate(last)
return(last)

I implicitly used poisondeathray's shift functions in my earlier sample script. And there was a bug: the vertical shift by 3 would fail if the video had 4:2:0 chroma.

[poisondeathray]

It's actually up. I'm not sure of the exact pixel offset, 3 was an intentional odd number to see if odd caused issues. It's not longer interlaced, so I'm assuming I could just change to 444. Or are those filters YV12/YUY2 only?

It works with everything, but has the same crop restrictions listed in that link . Avisynth will give an error.

In that example, it was converted to 444 , filter applied, then back to 420 to be able to join the other segments

[jagabo]

The shifts could be done without converting to 444, like this:

function ShiftUp(clip c, int y)
{
  AddBorders(c, 0, 0, 0, ((y+1)/2)*2)
  Spline16Resize(c.width, c.height, 0, y, c.width, c.height)
}

Of course, you still get chroma resampling.