Stabilizing wobbly cartoon

[Mephesto]

h t t p : / / w w w . s e n d s p a c e . c o m / f i l e / 9 f m l q p

I really wish those that do the digital transfers to Blu-ray would capture frames from the 35mm film strip in a slower, more individual fashion to make sure they are perfectly lined up. Carelessly capturing from free-rolling projector always brings this unwanted wobbling, even more unnecessary for films made recently or ones produced purely digitally that never should've been captured from 35mm film in the first place.

Anyway, I've noticed the wobbling here isn't only in the horizontal-vertical domain but the film strip was slightly warped so there is skewing of parts of the frame that are more noticeable upon deshaking.

I've tried Stab() which seems to reduce it but for some reason has 25% larger filesize than the shaky original when encoded with the same settings. VDub deshaker fixes it a bit better but I really hope to avoid it because A. it takes forever B. might run out of RAM and C. produces annoying artifacts around the edges that can't really be avoided in many scenes. Even if I set the max correction to 1%, 1% of 1280x688 is 13 and 7 pixels which is a lot to crop, and some scenes, especially close-ups get incorrectly deshaked and up to 13 pixels of shake gets INTRODUCED, so this would require selective trimming of the original and deshaked clip which is extremely annoying to script in avisynth. On my South park movie encode I ended up needing to write over 200 trims, never thinking there would be that many.

What are your professional recommendations?

[aedipuss]

frame by frame scanning of the 35mm film.
http://www.imagine-it-entertainment.com/index.html

[jman98]

aedipuss - I really think by "What are your professional recommendations?" he means to say "What better AviSynth filters can I use to fix this?". Based on reading his post I'm pretty sure he's just a fan rather than working for a studio with access to the source and money to pay someone like Imagine IT Entertainment.

[aedipuss]

lol - that wouldn't be a pro recommendation.... if it's just a dabbler with a complaint about a studio's low quality anime crud, then bollocks live with it, they couldn't give rat's ass about it to begin with. or buy yourself neatvideo if you're dying for better cartoons than a normal kid can live with.

[Mephesto]

aedipuss - I really think by "What are your professional recommendations?" he means to say "What better AviSynth filters can I use to fix this?". Based on reading his post I'm pretty sure he's just a fan rather than working for a studio with access to the source and money to pay someone like Imagine IT Entertainment.

lol, some unintentionally misleading semantics in my post I see. Jman's correct. I don't work in a studio, though I am giving it some thought considering what incompetent Down Syndrome babies seem to work at them today that force me to have to post-process the videos to perfection because their stupid-asses can't with studio-grade equipment, or because they source digitally-produced films from analog sources and post-process it with automated algorithms that end up destroying the picture and only removing half the dirt and scratches.

But yeah, my source is the retail Blu-ray, I don't have the 35mm master and likely never will. Instead of "professional recommendations" I should've just asked if anybody had any brighter ideas.

I plan to rip out all the duplicate frames to make it VFR so properly minimizing the projector wobble is necessary and minimizing illegitimate "differences" on duplicate frames that makes it harder to detect as duplicate is also preferred. Recommendations for this are welcome too. I usually use two RemoveSpots() or any temporal filter for this purpose since they are fast but RemoveSpots is problematic for scenes like subtle blinking of eyes that gets inferred as being a spot and removed, turning a non-duplicate into a duplicate.

Honestly though, why would Stab() stabilize the video a bit but then take more bitrate than the original? Go ahead and try Stab with my video and see what I mean.

[2Bdecided]

I'd try it, but I can't remember how to load that mp4 snippet properly into AVIsynth.

It's unlikely that stab() itself is increasing the bitrate. I'm guessing if you re-encode without stab(), you'll find a similar increase in bitrate from whatever settings you are using.

Hope this helps.

Cheers,
David.

[Mephesto]

I'd try it, but I can't remember how to load that mp4 snippet properly into AVIsynth.

ffmpegsource("c:\vid.mp4")
stab()

It's unlikely that stab() itself is increasing the bitrate. I'm guessing if you re-encode without stab(), you'll find a similar increase in bitrate from whatever settings you are using.

Lossy compressed sizes:
Original: 1740KB
VDub deshaker: 1719KB
Stab: 1911KB

Lossless:
Original: 56,461KB
Vdub deshake: 59,909KB
Stab: 71,789KB

[poisondeathray]

Stab's results might have to with the green frame bug

depan , and stab (which relies on depan) , cause a sort of green flicker on stablized frames. You can search for more info, I don't think it's ever been resolved or the environment in which it happens has been clearly defined. It affects some people more than others. In some situations is very severe (very green), in others, it's just a minor flicker. Might have to do with some specific dll's

The lossless original vs. deshake results are unexpected too. Why would vdub/deshake lossless be larger than original lossless ?

What kind of "lossless" did you use? All I frame or long GOP lossless? That would make a difference too

[Mephesto]

I don't see any green flicker, nothing any more noticeable than the original anyway. Where is the flicker?

The lossless original vs. deshake results are unexpected too. Why would vdub/deshake lossless be larger than original lossless ?

That surprised me too, but it doesn't compare to the dramatic 25% increase with Stab. The lossy encodes managed to stay consistent, though.

What kind of "lossless" did you use? All I frame or long GOP lossless? That would make a difference too

x264lossless, no B-frames, 50 keyint and all other settings tweaked for speed over compression. I would still not expect a video with less entropy to be 5% bigger though. Weird.

[poisondeathray]

Well there's flicker in the original, but stab tends to bring out more flicker (it might be from non cropped edges, or it might be from green frames depending if you are affected or not)

Regardless, do coloryuv(analyze=true) and stackvertical/horizontal on each and see the fluctuations in Y, U, V are larger in the stabb'ed version (at least it is here) . For temporal compression schemes, any differences will require more bitrate for coding. Not sure if the low range Y differences might be from non cropped edges (ie. black border)

(I did this without removing duplicates, which selecteven() or selectodd() should work)

e.g

a=ffvideosource()

a1=a.coloryuv(analyze=true)
b1=a.stab().coloryuv(analyze=true)

stackvertical(a1,b1)

[poisondeathray]

Without a doubt (at least here, with this combination of versions of .dlls), I can see slight flickering green tinged frames with stab.

Compare them in different scripts in avspmod, look especially at the "brown" desk

It's not pure green, like some cases , but it's a long standing, poorly understood "bug". Some people are more affected than others. I have no idea why
http://forum.doom9.org/showthread.php?t=159478

Here is frame 66 comparison (open the pngs in different tabs & flip back & forth)

Is it enough to explain the bitrate differences? Maybe the combinations of border/edge changes and flicker is enough for temporal compression schemes to cause increased bitrate requirements. I don't know.

[Mephesto]

You sir are a goddamn genius, I would've never noticed this on my own. I can definitely see the green in the snapshot, I also see accentuation of the macroblock edges on his hat and shirt, what's up with that?

It's weird that this barely-noticeable flicker would add 10% more bitrate though. Just shows how flawed even the best video codec is and I'll expect the same from H.265.

What temporal filter should I use? I don't need strong filtration. Between TTempsmooth and temporal cleaner which is the better choice? I forgot which one I used last time.

[poisondeathray]

It's weird that this barely-noticeable flicker would add 10% more bitrate though.

Not just the flicker, but the edge changes, black borders with stab. Same idea as ruddy dirty edges eating up bitrate (but not as bad) . Are those changes enough? I don't know

Just shows how flawed even the best video codec is and I'll expect the same from H.265.

How is this a flaw of the codec? It's working harder to preserve those changes. It's up to you to give it a clean source without flicker

What temporal filter should I use? I don't need strong filtration. Between TTempsmooth and temporal cleaner which is the better choice? I forgot which one I used last time.

I don't know. I don't have time to fiddle with it. I guess it depends what you plan on using to stabilize this. Fiddle with the settings and find out what works. For flicker, donald grafts deflicker for vdub works well and is another option. (And you should definitely use some deflicker somewhere as the original has flicker as well)

[Mephesto]

Not just the flicker, but the edge changes, black borders with stab. Same idea as ruddy dirty edges eating up bitrate (but not as bad) . Are those changes enough? I don't know

They shouldn't be. I cropped off the edges when I encoded the stab video.

How is this a flaw of the codec?

A barely-noticeable very insignificant change adding 10% more entropy is a flaw because it obviously isn't properly modeled on human psychovisuals. With something like MPEG-2 it would probably be worse. In fact, when optimized for PSNR H.264 can have comparable quality to MPEG-2, as exemplified by Quicktime.

I don't know. I don't have time to fiddle with it. I guess it depends what you plan on using to stabilize this. Fiddle with the settings and find out what works. For flicker, donald grafts deflicker for vdub works well and is another option. (And you should definitely use some deflicker somewhere as the original has flicker as well)

If Stab introduces flicker, its probably best to deflicker afterwards. I'm just gonna use TTempsmooth with low settings.

[poisondeathray]

And it should go without saying that you should decimate the duplicates , maybe with dedup or similar functions, in a 1st stage processing step beforehand if you plan to use any temporal filters (I know you mentioned that there were pure 24p sections as well) . Otherwise filters like ttempsmooth or deflickering plugins will not be as effective

What was your full script, what crop values etc... and if you resized back, did you use a sharp or soft resize? because that makes a difference as well

[poisondeathray]

But even an I-frame lossless intermediate like lagarith sees about 6% increase with stab (no crop or resize) over original encoded to lagarith

This suggests more than just temporal component like flicker is causing increase in bitrate requirements.

[poisondeathray]

I think I know what it is. If you look closely at the green section on the back wall, there is a sort of posterization noise in the "stabbed" version

You can see it more clearly with histogram("luma") , as the block edges seem to be "enhanced" . Histogram("luma") is an exaggerated luminace view

Can you verify these results with your versions of plugins (as I said earlier, some people seem to be more affected than others, some people get very green frames, some get mild green frames etc...)


That still doesn't explain the deshaker results you got, because to my knowledge it's not affected by the green bug or this posteriztion effect . Which settings or edge fill options did you use?

Lossless:
Original: 56,461KB
Vdub deshake: 59,909KB
Stab: 71,789KB

[Mephesto]

And it should go without saying that you should decimate the duplicates , maybe with dedup or similar functions, in a 1st stage processing step beforehand if you plan to use any temporal filters

Yeah, I plan to decimate duplicates before deflickering.

What was your full script, what crop values etc... and if you resized back, did you use a sharp or soft resize? because that makes a difference as wel

avisource("m:\hv.avi").stab.crop(2,2,-2,-2).lanczosresize(1280,688)

Necessary due to mod16 but I guess this can be compared more objectively by cropping 8 on all sides for both videos and encoding. I'll try that.

But even an I-frame lossless intermediate like lagarith sees about 6% increase with stab (no crop or resize) over original encoded to lagarith

You should crop black flickering borders without question, they add entropy.

I think I know what it is. If you look closely at the green section on the back wall, there is a sort of posterization noise in the "stabbed" version

You can see it more clearly with histogram("luma") , as the block edges seem to be "enhanced" . Histogram("luma") is an exaggerated luminace view

Can you verify these results with your versions of plugins (as I said earlier, some people seem to be more affected than others, some people get very green frames, some get mild green frames etc...)

AAH, NOW I NOTICE. Stab() ******* SHARPENS the video in addition to depanning. I verified right now with my own. Mine don't have banding because the one I sent you was encoded with crf16 but it clearly sharpened the chroma gradients as well as all the other edges are thicker and darker. This explains everything.

That still doesn't explain the deshaker results you got, because to my knowledge it's not affected by the green bug or this posteriztion effect . Which settings or edge fill options did you use?

I think this is the configuration I used, don't remember if I used 2 or 10 pixels for edge softening because I only wanted to quickly test results.

[poisondeathray]

Did you confirm the lossless export from deshaker used same pixel format / colorspace ? as you know vdub will convert to rgb (although a lossless RGB format, even stabilized, I would expect to be much larger)

[2Bdecided]

stab() is just calling DePan. stab() adds a smoothed frame between each real frame, which is subsequently discarded after DePan, and limits the corrections allowed to just 4 pixels, but does nothing else that I can see. So I wonder what in DePan causes all these problems?

I guess I've either been lucky or undiscerning, because I've never noticed them here.

Cheers,
David.

[manono]

...and limits the corrections allowed to just 4 pixels, but does nothing else that I can see.

Yes, but that behavior can be modified. I have mine set for 8. And you can also have it do fill-in, rather than having the edges jump all around afterwards.

[2Bdecided]

Yeah, I know - I meant the problem must be in DePan because there's nothing to cause it in stab(). I wasn't criticising.

[unclescoob]

https://forum.videohelp.com/threads/348759-Moving-dots-I-cant-seem-to-get-rid-of?p=2185...=1#post2185368

[Mephesto]

Did you confirm the lossless export from deshaker used same pixel format / colorspace ? as you know vdub will convert to rgb (although a lossless RGB format, even stabilized, I would expect to be much larger)

No, because I used x264lossless which doesn't support RGB, so colorspace conversions have happened.

Anyway, I've decided to simply use Stab for the wobbling. But there is a new problem now with spot removal.

RemoveDirtMC even on a setting of 99 failed to remove the simplest, smallest etches of black spots (it did remove white ones, which is ironic because in another cartoon it ignored the white ones while aggressively killing the black ones).

Look on frame 92 on the video on the far left. How does that spot get past the filter?

Sccript: avisource("M:\hvstab.avi").removedirtMC(99,false)

[poisondeathray]

But there is a new problem now with spot removal.

RemoveDirtMC even on a setting of 99 failed to remove the simplest, smallest etches of black spots (it did remove white ones, which is ironic because in another cartoon it ignored the white ones while aggressively killing the black ones).

Look on frame 92 on the video on the far left. How does that spot get past the filter?

If we're referring at the same spot, there are duplicate frames, so it's not detected as a "spot"

Spot/dirt removal filters look at differences between frames. Duplicate frames means no differences, so filter does nothing.

On a static background scene like this, another approach is to mask out and replace the background with a still

[jagabo]

Since every other frame is a duplicate:

ffVideoSource("hvstab.mp4") 
Merge(SelectEven(), SelectOdd()) # or just SelectEven() or SelectOdd()
RemoveSpots()

Of course, the Merge() will not work if there are dropped or extra duplicate frames. But I did it here to reduce the compression artifacts.

[Mephesto]

Ha, yeah the duplicates were the problem. I could've sworn it failed to filter non-duplicated spots though. Oh well.

Should I apply Stab before or after dedupping? The log and times have already been acquired with a stab-processed input. My gut tells me to Stab after.

[jagabo]

I could've sworn it failed to filter non-duplicated spots though.

RemoveSpots() isn't perfect. It will miss some spots. And it will sometimes remove valid picture elements. Look at the the thin lines in your cartoon. Especially on the pockets of the shirt, on the man's face, and right hand.

Should I apply Stab before or after dedupping? The log and times have already been acquired with a stab-processed input. My gut tells me to Stab after.

It depends on how you dedup. If simple decimation works, Stab afterward will be faster. But if you use a "smart" deduplicator it may get confused by the film bounce.

[Mephesto]

RemoveSpots() isn't perfect. It will miss some spots. And it will sometimes remove valid picture elements.

Removespots() blows, I don't use it.

It depends on how you dedup. If simple decimation works, Stab afterward will be faster. But if you use a "smart" deduplicator it may get confused by the film bounce

I already did the first pass where I added Stab() before the deldup analysis pass so the bounce doesn't interfere with flagging the duplicate frames.

Question is, should I append or prepend Stab to the second pass of the Deldup?

[Mephesto]

*FACEPALM*

I got a little too ahead it seems. I do remember why I never despot cartoons after all. And this is on the lowest strength.

RemovedirtMC needs a parameter to specify maximum spot size or something, this is just funny. Removespots temporally blurs without removing any actual spots and RemovedirtMC does... this.