How to get DePan for AviSynth to fix Screen Shake?

[VideoFanatic]

I have a 720 x 480 interlaced MPEG2 video HERE which I've denoised. The original is HERE. The video moves up several pixels every few seconds which was probably caused by old VHS equipment. You can see the "USA" logo move up and down.

I found this code but it just gives wavey artefacts:

  Mpeg2Source("E:\1\Screen Shake.d2v")
  i = ConvertToYV12()
  mdata = DePanEstimate(i)
  DePanStabilize(i, data=mdata, initzoom=1.1)

I looked at the DePan documentation but I couldn't understand it.

Could someone please suggest a filter and script to fix the screen shake. I'm not bothered if it's for DePan or not, just as long as it works. Thanks

[jagabo]

You need to deinterlace first.

That's some really sad video -- interlaced with repeated frames. You need to remove the repeated frames. Deshaking won't work.

[VideoFanatic]

I thought you're not supposed to deinterlace an interlaced video? I was told that several times when asking questions elsewhere.

Could you suggest a script please - how do I deinterlace it?

What do you mean by repeated frames? Did you download the clip and watch it in Windows Media Player? I thought it looked fine apart from the screen shake?

Thanks

[jagabo]

DePan only works with progressive material. So you will have to deinterlace, depan, reinterlace. The DePan docs recommend:

AssumeTFF()
SeparateFields()
mdata = DePanEstimate(last)
DePanStabilize(last, data=mdata, initzoom=1.1)
Weave()

But you will get better results with a better deinterlace method:

AssumeTFF()
QTGMC()
mdata = DePanEstimate(last)
DePanStabilize(last, data=mdata, initzoom=1.1)
SeparateFields()
SelectEvery(4,0,3)
Weave()

And none of this will help with the random horizontal jitter from VHS.

You'll still have problems with back-and-forth movement because of the duplicate frames. Open your MPEG file with VirtualDub and step though frame by frame. You'll see that frames 68, 80, 83, 89, 110... are duplicates of the preceding frames.

[jmac698]

You can use dedupe plugin to fix that, and my dejitter script to fix the jitter. All in all doable.
http://akuvian.org/src/avisynth/dedup/
http://forum.doom9.org/showthread.php?p=1531324

[VideoFanatic]

If you look at the VIDEO I get from the following script you'll notice that the whole picture now moves left and right quite a lot (look at the WF logo). Is that what's supposed to happen? The original video only moves up and down a few pixels. The new video kind of gives me motion sickness looking at it. The picture seems to move left and right a lot - why does it do that if it's just the picture moving up and down that I want to fix?

			 				AssumeTFF()
QTGMC()
mdata = DePanEstimate(last)
DePanStabilize(last, data=mdata, initzoom=1.1)
SeparateFields()
SelectEvery(4,0,3)
Weave()

Will the script have no effect on parts of a video that don't have the screen shake issue?
Also, is there any way to speed up the script as it's very slow?

[jmac698]

Try dxmax=0
This prevents horizontal compensation.
You were given the wrong arguments, the stabilize defaults were meant for shaky cameras, not jumping up and down.
There's probably a better way to do this..

[VideoFanatic]

OK thanks but where do I put that line of code?

[VideoFanatic]

I tried the following:

Mpeg2Source("E:\1\Screen Shake.d2v", CPU=6)
DeGrainMedian(limitY=2, limitUV=3, mode=1, interlaced=true)
DeGrainMedian(limitY=2, limitUV=3, mode=1, interlaced=true)
AssumeTFF()
QTGMC()
mdata = DePanEstimate(dxmax=0)
DePanStabilize(last, data=mdata, initzoom=1.1)
SeparateFields()
SelectEvery(4,0,3)
Weave()

The top line deblocks and gets rid of most noise. The 2nd line gets rid of any remaining noise. The rest of the code fixes the screen shake. Please let me know what you think. HERE is the new video.

The QTGMC script takes over 24 hours! Is there any way to speed the script up? Or is there another filter I could use which is quicker?

[VideoFanatic]

You can use dedupe plugin to fix that, and my dejitter script to fix the jitter. All in all doable.
http://akuvian.org/src/avisynth/dedup/
http://forum.doom9.org/showthread.php?p=1531324

I don't even see any duplicate frames when I watch the video so I don't see why I would need to remove them with dedup.

I looked at your other link but the page is a bit messy, what am I supposed to do? What script do I use?

Does it just fix horizontal jitter because I can't really see any, I just see the vertical problem as previously mentioned - the picture moves up and down every few seconds.

[jagabo]

The default settings for Depan are for large scale camera shake. For your slight up/down bounce you can limit dymax too. dymax=4 is probably sufficient. Maybe even 2. The logo moves because the stabilization is based on the entire picture. Stab() is script that uses Depan for small up/down and left/right motions.

##############################################################################
#Original script by g-force converted into a stand alone script by McCauley  #
#latest version from December 10, 2008                                       #
##############################################################################

function Stab (clip clp, int "range", int "dxmax", int "dymax") {

range = default(range, 1)
dxmax = default(dxmax, 8)
dymax = default(dymax, 8)

temp  = clp.TemporalSoften(7,255,255,25,2)
inter = Interleave(temp.Repair(clp.TemporalSoften(1,255,255,25,2)),clp)
mdata = DePanEstimate(inter,range=range,trust=0,dxmax=dxmax,dymax=dymax)

DePan(inter,data=mdata,offset=-1)
SelectEvery(2,0) }

Stab() works well for projector bounce. It should work here too.

Yes, QTGMC is very slow. It has simple presets for faster or slower deinterlacing. For example:

QTGMC(preset="fast")

There are also lots of specific settings you can control. See the HTML manual. Of course, the faster settings you use the less accurate it is.

[VideoFanatic]

OK I tried the Very Fast setting on the QTGMC script and it takes about 8 hours so that's OK. You're script looks interesting as well however I tried it and the picture still moves up and down a few pixels. Is there a stonger setting I should use?

[jagabo]

You're script looks interesting as well however I tried it and the picture still moves up and down a few pixels. Is there a stonger setting I should use?

Are you talking about Stab()? Try increasing the range:

Stab(range=3)

Range is the number of prior and future frames to compare to.

[VideoFanatic]

OK so I've got this code:

Mpeg2Source("E:\1\Screen Shake Denoised new.d2v")
function Stab (clip clp, int "range", int "dxmax", int "dymax") {

range = default(range, 3)
dxmax = default(dxmax, 8)
dymax = default(dymax, 8)

temp  = clp.TemporalSoften(7,255,255,25,2)
inter = Interleave(temp.Repair(clp.TemporalSoften(1,255,255,25,2)),clp)
mdata = DePanEstimate(inter,range=range,trust=0,dxmax=dxmax,dymax=dymax)

DePan(inter,data=mdata,offset=-1)
SelectEvery(2,0) }

I changed the underlined line - was that correct?

The picture still jumps up and down. I even tried different settings up to 12 but no difference.

Also I'm not sure what's happened here but yesterday when I tried the following script it took 8 hours on the Ultra Fast setting, now it takes 32 hours:

Mpeg2Source("E:\1\Screen Shake Denoised new.d2v")
Load_Stdcall_plugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll")
AssumeTFF()
QTGMC(preset="Ultra Fast")
mdata = DePanEstimate(dxmax=0)
DePanStabilize(last, data=mdata, initzoom=1.1)
SeparateFields()
SelectEvery(4,0,3)
Weave()

[jagabo]

The desakers may be getting confused by the bouncing of the ropes.

QTGMC(preset="ultra fast") may not work any better than Yadif. And Yadif is faster. Try using that instead:

Mpeg2Source("E:\1\Screen Shake Denoised new.d2v", CPU=6)
Load_Stdcall_plugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll")
AssumeTFF()
Yadif(mode=1, order=1)
mdata = DePanEstimate(dxmax=0, dymax=4)
DePanStabilize(last, data=mdata, initzoom=1.1)
SeparateFields()
SelectEvery(4,0,3)
Weave()

If you don't need interlaced video you can leave out the last three lines.

Oh, if you have a multicore CPU and multithreaded AviSynth installed you can get better performance by enabling multithereading. Add SetMtMode(2,0) at the top of your script.

[VideoFanatic]

Thats brilliant, it now only takes 4 hours! Thanks. Just wondering though, could you please explain to me what the following lines mean and how you decided on those settings:

DePanStabilize(last, data=mdata, initzoom=1.1)
SelectEvery(4,0,3)

I notice that half way in my video the bottom border moves up. Is there a way to prevent that happening?

Does the initzoom setting just zoom in the picture so you don't see the borders?

[Gavino]

OK so I've got this code:

Mpeg2Source("E:\1\Screen Shake Denoised new.d2v")
function Stab (clip clp, int "range", int "dxmax", int "dymax") {

range = default(range, 3)
dxmax = default(dxmax, 8)
dymax = default(dymax, 8)

temp  = clp.TemporalSoften(7,255,255,25,2)
inter = Interleave(temp.Repair(clp.TemporalSoften(1,255,255,25,2)),clp)
mdata = DePanEstimate(inter,range=range,trust=0,dxmax=dxmax,dymax=dymax)

DePan(inter,data=mdata,offset=-1)
SelectEvery(2,0) }

I changed the underlined line - was that correct?

The picture still jumps up and down. I even tried different settings up to 12 but no difference.

Is that your complete script? If so, you're doing it wrong.
That code just defines the function Stab() - you also need to call it in your script.
After Mpeg2Source(...), add the line:
Stab(range=3)

Also, although it's not wrong, it's cleaner to keep the original code, rather than change the default inside the function as you have done.

[VideoFanatic]

Thanks Gavino but I tried that and the picture still moves up and down. I just use the following script instead:

Mpeg2Source("E:\1\Screen Shake Denoised new.d2v")
  Load_Stdcall_plugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll")
  AssumeTFF()
  Yadif(mode=1, order=1)
  mdata = DePanEstimate(dxmax=0)
  DePanStabilize(last, data=mdata, initzoom=1.0)
  SeparateFields()
  SelectEvery(4,0,3)
  Weave()

I was just wondering though, how do you come up with the following numbers?:

SelectEvery(4,0,3)

[Gavino]

The sequence SeparateFields().SelectEvery(4,0,3).Weave() is used to interlace a progressive clip (often to reinterlace a clip that has been deinterlaced for filtering, as here).
SeparateFields() splits each frame into two fields, top field first if TFF.
SelectEvery(4,0,3) takes each group of 4 fields (from 2 frames) and selects the first and last field, ie the top field of the 1st frame and the bottom field of the 2nd frame. Then Weave() puts them together to give the interlaced result.

[VideoFanatic]

OK thanks. I've noticed that most of the time the top and bottom borders stay where they are but sometimes they move up and down quite a lot which gives the feeling of motion sickness.

To fix that problem I completely crop them out by cropping the top by 30 and the bottom by 42. This seems strange since the original picture didn't move and down that much so why do the borders on the corrected video move up and down by that much? Is there a way to limit how much the borders move?

[jagabo]

OK thanks. I've noticed that most of the time the top and bottom borders stay where they are but sometimes they move up and down quite a lot which gives the feeling of motion sickness.

Use more zoom, or limit the y motion.

To fix that problem I completely crop them out by cropping the top by 30 and the bottom by 42. This seems strange since the original picture didn't move and down that much so why do the borders on the corrected video move up and down by that much? Is there a way to limit how much the borders move?

Say one frame is much lower than the two surrounding it. To remove that bounce it has to be shifted up. But there's no information in the frame to fill the bottom area that's "exposed" after moving the frame. It's left blank.

VirtualDub's DeShake filter can attempt to fill those empty areas with data from prior or subsequent frames. I don't think DePan has that ability.

[VideoFanatic]

Use more zoom, or limit the y motion.

To fix that problem I completely crop them out by cropping the top by 30 and the bottom by 42. This seems strange since the original picture didn't move and down that much so why do the borders on the corrected video move up and down by that much? Is there a way to limit how much the borders move?

Say one frame is much lower than the two surrounding it. To remove that bounce it has to be shifted up. But there's no information in the frame to fill the bottom area that's "exposed" after moving the frame. It's left blank.

VirtualDub's DeShake filter can attempt to fill those empty areas with data from prior or subsequent frames. I don't think DePan has that ability.

Zoom just zooms in the whole picture and you still see the borders move. How do I limit the y motion?

I am currently using the script below. Is that DeShake filter just for VirtualDub or does it work with AviSynth? Could you please provide a script for it. Thanks

Mpeg2Source("E:\1\Screen Shake Denoised new.d2v")
  Load_Stdcall_plugin("C:\Program Files\AviSynth 2.5\plugins\yadif.dll")
  AssumeTFF()
  Yadif(mode=1, order=1)
  mdata = DePanEstimate(dxmax=0)
  DePanStabilize(last, data=mdata, initzoom=1.0)
  SeparateFields()
  SelectEvery(4,0,3)
  Weave()

[jagabo]

You could try reading the documentation.

http://avisynth.org.ru/depan/depan.html

Depan does have the ability to fill borders with information from prior and later frames. Use the prev and next options. You can use dymax to limit the y motion. Some combination of initzoom, addzoom, and zoommax should be able to eliminate the black border without fill from prev/next frames (which probably doesn't work well under some circumstances).

DeShaker filter is a VirtualDub filter. You don't use it in a script, you use it in VirtualDub.

[VideoFanatic]

I did read the documentation at http://avisynth.org.ru/depan/depan.html but I can't figure it out. I can't even find the functions you mentioned. If I could understand the documentation I wouldn't be here!

[jagabo]

Try some variations of:

DePanStabilize(last, data=mdata, initzoom=1.0, prev=5, next=5, dxmax=0, dymax=8)

[VideoFanatic]

That had no effect. The borders still move up and down like crazy.

[jagabo]

OK, I downloaded your video again and ran Depan. This script was enough to eliminate the vertical jitter and gave no black borders:

Mpeg2Source("Screen Shake.d2v", CPU=2)
AssumeTFF()
QTGMC(preset="fast")
mdata = DePanEstimate(dxmax=0)
DePanStabilize(last, data=mdata, initzoom=1.05, dxmax=0, dymax=4)

[VideoFanatic]

Thanks. It works perfectly now.

[VideoFanatic]

Jagabo, I've since watched my fixed video on my TV rather than on my PC and I've noticed horizontal jitter. What setting would I need to get rid of that?

[jagabo]

Change dxmax. dxmax=0 disables horizontal deshaking.

If you mean horizontal jitter caused by time base errors, you're not going to fix that in software.