converting 60 unique fields ps

[spiritgumm]

you guys make this look so easy. When you say lossless intermediates, do you mean uncompressed avi output which will be large files?

[jagabo]

When you say lossless intermediates, do you mean uncompressed avi output which will be large files?

Uncompressed AVI or a lossless encoder like HuffYUV, Lagarith, MSU Lossless, etc. Yes, the files will be big.

[spiritgumm]

I'm finally getting to this jumpy video (2.m2v). Poison, do you recollect what settings you used for Deshaker? I did a sample following an online guide, and set the rotation/pan to 0 in Pass2, but still got and moving borders, and video still looked bad.
Also, I read an uncompressed avi output can be 90 gigs per hour. What codec would produce the smallest lossless video? Would that be UT video 420 you recommended in another thread? Does that need to be configured when chosen?

[poisondeathray]

I'm finally getting to this jumpy video (2.m2v). Poison, do you recollect what settings you used for Deshaker? I did a sample following an online guide, and set the rotation/pan to 0 in Pass2, but still got and moving borders, and video still looked bad.
Also, I read an uncompressed avi output can be 90 gigs per hour. What codec would produce the smallest lossless video? Would that be UT video 420 you recommended in another thread? Does that need to be configured when chosen?

detect rotation, detect zoom should be uncheckmarked in pass 1 parameters

The scale and pixle options below will be slower but more accurate. You can skimp on these options, but the results might differ a bit.

scale: full (most precise)
use pixels: all (most robust)

On pass 2 parameters , in post #59 , I said "I used adaptive zoom and no borders, but you might prefer the other border options like mirror or fill in with past/future frames "

Also, did you read that entire post regarding dupes and srestore placement order in the script ?

Smallest lossless YV12 (ie. best compression) would probably be FFV1 (you can access through FFDshow) or MSU . But better compression means slower to encode and decode. Tradeoffs you have to make. Some compresson formats can be 10-20x slower. So if you have a monster script, it might take weeks instead of days for you . UT is the fastest (for multithreaded computers, might only marginally faster than huffyuv for single thread, but the difference is larger with more cores) . A newer computer will help

[spiritgumm]

the online guide (out of date) specified scale=full, and pixels=all, so I did do that.
I added 0 in the zoom/rotation in Pass 2 (to mimic your "deshaker(norotation.nozoom).xvid.q3.avi" video), while increasing the other parameters to 3000 (suggested in guide). Here's pics of my 2 pass settings, hopefully you can identify my mistakes.

Pass1.JPG

Pass2.JPG

I dont know what part of the script you kept/omitted while using Deshaker. I tried a basic script, omitting Depan and Stab to see if Deshaker would replace their functions. Omitted noise reduction to get a faster sample done:
assumetff
bob()
SRestore(23.976)
trim(25571,25983)

The result had moving borders, and vertical shaking that Stab might correct.

[poisondeathray]

your pass 2 settings say: edge compensation none (large borders)

large borders mean large black borders. If you re-read my post, it says I used "adaptive zoom (no borders)"

no borders means.... no borders . The trade off is you lose pixels. It zooms in , so you lose active image area.



I deleted everything and don't recall what I used before deshaker, but it was probably something like this
If you asked back when this was posted I would have posted the exact script

MPEG2Source()
AssumeTFF
QTGMC() #some settings don't recall . You can use some other bobber here
Srestore(23.976)
AssumeFPS(24000,1001)

I may have cropped some of the borders horizontally (keeping mod16) before bobbing, and after bobbing cropped vertically, for slightly faster processing. I probably split that into a lossless 1st stage (video=>fast recompress in vdub, some lossless codec), then used something like this for the input into deshaker

AVISource("1stpass.avi")
Crop() #can't recall exact values
ConvertToRGB()

Deshaker works better when you don't have black borders to begin with. I can't recall what the exact crop values were

Then run that in deshaker, export out lossless RGB

Then clean it up with other denoising filters, and resize (and or add borders if you're going back to dvd). I think I used RemoveDirtMC

AVISource("deshakerRGB.avi")
ConvertToYV12()
#some noise filters
#some resize or add borders

[spiritgumm]

Thanks for revisiting this video problem. At the time, I thought about asking for your script, but I wanted to check out Deshaker first for the inevitable questions, then got sidetracked by that grainy dvd I thought would be a quick fix...

I got mixed up - thought I would get 'no borders' (none) when I picked "None [Large borders]."

With Deshaker 2.7, the wording is slightly different. There is no "Adaptive zoom (no borders)." Options are:
None (Large Borders)
Adaptive zoom (some borders)
Adaptive zoom only
Fixed Zoom (no borders)
Adaptive and Fixed (no borders)
Adaptive and Fixed only

I would think "Adaptive zoom only" would be what you're suggesting - "Adaptive zoom[no borders]", but if that were true, the last two options should be the same (therefore redundant). So something is amiss.

I will be encoding to dvd, so I would add borders later. With Stab and DePanStabilize, I cropped afterwards, but I will crop before bobbing and deshaking. I originally used BOB (and cant realistically use Qtgmc with my PC).
Why do I need the line: AssumeFPS(24000,1001) ?

[poisondeathray]

I think it was Adaptive and Fixed (no borders)

There might be better options, like the edge fill options. You can experiment with them

Why do I need the line: AssumeFPS(24000,1001) ?

23.976 is only an approximation ; 24000/1001 is the exact value (more correct). I don't know if srestore frate parameter accepts fractional input

[spiritgumm]

I've tried all the "Edge Compensation" options and results are not stable like your last one ("no rotation or zoom" avi). I'm using Vdub 1.9.9 with Deshaker 2.7. Can you take a look at my settings again? Are you able to replicate your "no rotation or zoom" avi?
Is it possible you still had some stabilizing function in the earlier script so when it got to Vdub, it didnt need to deshake as much?
I'll save my question about the framerate change (24 vs 23.976) for later.

[poisondeathray]

I just looked at it again and I get very similar results with default settings , the only things changed from the defaults were the aspect ratio (same as yours NTSC 0.912) and uncheckmarked detect zoom and rotation (same as yours), and Adaptive and Fixed (no borders)

(there might have been slightly different crop values for preprocesing), but the end result was very similar to "no rotation or zoom avi"

The difference in your screenshot is the 2nd tab parameters in mine were default settings (so rotation and zoom smoothness was at 1000) - but that shouldn't matter because detect zoom and rotation were unchecked in the pass 1 settings , and horizontal panning/ vertical panning were both at 1000 (default values, not 3000 like in your screenshot)

[spiritgumm]

Could Qtgmc vs. BOB be making the difference?
I'm using a slightly longer clip from the video. I added cropping to the basic script.
assumetff
bob()
SRestore(23.976)
crop(10,8,-10,-12)

result with "Adaptive and fixed (no borders)." "Fixed zoom" alone is stable but always zooms in too much.
sample.avi

[poisondeathray]

sample.avi is messed up

that isn't from the difference between bob and qtgmc

are you sure you re-ran pass 1 with the new settings (uncheckmarked zoom and pan) ?

don't reuse the old deshaker log file from previous results

[spiritgumm]

yes, I'm sure.
I also closed/reopened Vdub to see if that would help, but same results. Should I be deleting a log somewhere?

I did not re-run Pass 1 when I was trying various "Edge compensation" options in Pass 2 - I assumed each Pass 2 run would reuse the Pass 1 info(which didnt need to be changed).

[poisondeathray]

post the longer sample source then

are you sure the pass 1 was done on the same clip or script? the log only applies to that same clip. If the frames have different number (longer or shorter clip), or it's slightly different clip (maybe you added some crop values but didn't re-run pass 1) you can get the screwed up results you see there. You can change pass2 settings, but they only apply to the same clip that had pass1 done on it. If you change the input clip in any way, you have to re-run pass 1

i ran bob() instead of qtgmc() on the 2.m2v sample (even used skimpy pass 1 settings, like every 4th pixel) and it has the same motion characteristics at the "nozoom norotation avi"

Here it is (no denoising filters), just resize filter in vdub to 640x480, applied after deshaker filter

[spiritgumm]

Well, this is weird. I just ran Pass2 a few more times, and the samples all look good (no shaking or zooming). Either my PC is buggy, or Vdub/Deshaker are. Do you know which Deshaker you're using? It might be more stable.

I know you said there would be some pixels missing in one of settings. I noticed some cropping in the "Adaptive and Fixed (no borders)" option.
"Adaptive only" would be a better option (no Deshaker cropping) except there's some noise rising up from the bottom and then down from the top (when the woman sits on the bed). Is there a better option, or a way to minimize the noise?

Your Bob clip didnt crop as much as my Adaptive/Fixed version. Perhaps the longer the clip (with more bouncing issues), the more Deshaker needs to crop?

[poisondeathray]

my deshaker version says 2.5 . My guess would be that you were mixing up different sources without re-running pass 1

"Adaptive only" would sometimes have black borders or dirty edges creep in (if you didn't crop them)

When you leave black borders in, thats when "use previous & future frames to fill" comes in. Sometimes it can look weird, other times it can be excellent

I find adaptive + fixed no borders usually give the best results (at the expense of losing active image area by zooming in)

[spiritgumm]

I tried "Adaptive only (some borders)" and ticked the "Use Previous frames to fill in borders" and the result was perfect. I wonder if it will work as well on the whole video, or when there's more complex panning. One of these videos has some film glitches I will have to edit around; I hope it wont throw off Deshaker.

I really did re-run Pass 1 when I made any changes in that part of the filter, or original script. After selecting the Pass 1 in the filter, I ran it by clicking on FILE and selecting: Run Video Analysis Pass. Perhaps that wasnt the right way?

I might try using Deshaker 2.5 instead. The real problem is probably my PC which hasnt been the same since I downloaded (and removed) some malware a few weeks ago. Several video-related programs dont work quite right anymore.

[poisondeathray]

yes, every source is different

the artifacts you want to look for in "the previous frames to fill" option - they look like a staggered deck of cards, where the edges of frames overlap on each other. As I said, sometimes it works great on some material, others time: not so good

If you want to tweak it you can manually do it in sections, or even use depan. I tested some custom depan settings, and there is less zoom so more active image area (no green frame issue - I think it has to do with the mirror edge option)

I usually use adaptive + fixed no borders, because it fixes everything without having to worry about bad edges creeping in (you don't have to look at every frame manually) , but the side effect is less active image area. You can customize the zoom limits or process in sections to improve the results to regain more area

For running pass 1, I usually hit the play button (the button with the "o" for output pane). Never tried it any other ways so I can't comment if other ways work or not

[spiritgumm]

I guess I wont know what the edges look like until I run the whole thing.

So it looks like you suggested at least 3 stages or scripts.
Crop horizontal/Bob mpg to lossless file
Crop vertical/Deshake lossless file to new lossless file
Denoise/Addborders lossless file to mpg encoder

If the first stage creates a large file, wont I need just as much space to accommodate the 2nd file being generated from it?
Is there a list or resource which shows codec output size & processing speed, so you can predict how big a file will be?

[poisondeathray]

Iif you're using bob() you can probably use 1 lossless stage , avs for the bob/srestore/crop input into deshaker, lossless export out of deshaker , then apply your final denoise/borders/resize in a script to that 2nd export into mpeg2 encoder .

But if you're using slow deinterlacer like QTGMC with slow settings, then it might make sense for 2 stages (because deshaker uses 2 passes) ; so apply your 1st script to generate a lossless intermediate for input into deshaker . But if your denoisers filters are slow, you might want to yet add another lossless stage before the mpeg2 encoder

It's simple - whenever you have a slow bottlenecking filter, you might want to use a lossless stage when the next stage is going to be using 2 passes. The rationale is you endure the slow filter once instead of twice

Yes, each lossless export will require lots of space. They will vary in size as you apply filters - e.g. a clean denoised lossless export will be smaller than the "dirty" version of the same video when using the same compression

I don't have any links handy for lossless compression comparisons, but you can search google and a bunch are on doom9 , but some of them are older. It's almost impossible to predict the size, so just have lots of HDD space ready

[spiritgumm]

Everything is slow on my PC, so in the last stage - Denoising - would that script get processed twice if its getting fed into an mpg encoder doing 2-pass (specifically HCenc)?

I'll see if there are any good codecs available for my purposes.
Thank you VERY MUCH for all your work and attention to my problems!

[poisondeathray]

Everything is slow on my PC, so in the last stage - Denoising - would that script get processed twice if its getting fed into an mpg encoder doing 2-pass (specifically HCenc)?

Yes, it gets processed twice.

That's why I said if you are using a slow denoiser, then consider using another lossless stage at that point before the mpeg2 encoder (most people use 2 or more passes when going to DVD, unless you have enough room for 1pass CBR - short duration content)

But if you're using a fast denoiser, it might not be worth it

EDIT: I cleaned up the post above, I hope it's more clear

[spiritgumm]

Thanks for clarifying. I realize it probably seemed obvious, but I wasnt sure if the mpeg encoder 2-pass would double-dip the incoming script, same as Vdub, or whether you were recommending a lossless stage because of some other reason.

I wanted to try out some codecs, but keep having the same issues with Deshaker (apparently Pass1 is still zooming in/out). I've tried versions 2.5, 2.6, 2.7 with a couple Vdub versions. I read 2.6 fixes a crash from prefilter cropping; I dunno if cropping in the script has anything to do with it. Very frustrating.

[spiritgumm]

Hi Poison,
I figured out what I was doing wrong - mainly because you were doing it right. I only use Vdub in Fast Recompress mode for making xvids from AVS. I completely forgot about the colorspace conversion requirement, but you included it in your sample scripts for me, so I eventually caught on.

I have two questions. I've read there's some degradation between doing multiple back and forth colorspace conversions of the same video - if true, can I avoid it?

These videos are TV episodes I'll be Deshaking (52 minutes each). I was thinking of processing them in 5 parts with TRIMs (about 10 minute segments) because of my slow PC (and little memory). I would combine the dvd encodes at the end (and mux with one large audio stream, generated from a script without trims). Does that make sense, or do you have a better suggestion?

[poisondeathray]

I figured out what I was doing wrong - mainly because you were doing it right. I only use Vdub in Fast Recompress mode for making xvids from AVS. I completely forgot about the colorspace conversion requirement, but you included it in your sample scripts for me, so I eventually caught on.

I have two questions. I've read there's some degradation between doing multiple back and forth colorspace conversions of the same video - if true, can I avoid it?

Yes, there is degradation going back & forth YV12<=>RGB , so avoid uncessary trips back & forth.

Many vdub filters require RGB and full processing mode (e.g. deshaker)

If you convert to RGB, do it only once, then apply all your RGB filters while you are still in RGB

These videos are TV episodes I'll be Deshaking (52 minutes each). I was thinking of processing them in 5 parts with TRIMs (about 10 minute segments) because of my slow PC (and little memory). I would combine the dvd encodes at the end (and mux with one large audio stream, generated from a script without trims). Does that make sense, or do you have a better suggestion?

That might work, but you need to select "good spots" to cut them up , like a scene change or fade to black - because the joins might not match if the cutsites are in the middle of a scene (deshaker will process them differently and there might be a "blip" where the cuts are)

[spiritgumm]

I was going to trim at fades when possible, but I will do it for sure. Sounds like I might need to generate audio for each trim to make sure it's all synced.
So as long as the lossless codec I use outputs RGB, I dont need to convert back to YV12 until I encode with HCenc.

[spiritgumm]

I finally started the two videos I need to deshake. As predicted, it's taking alot of time and space, so I'm doing it in small increments. Have had at least two problems so far:
A fast moving car moving left to right, kept relatively in center screen by camera panning right to follow, has some horizontal jerks in it.
The other problem (artsy opening credits were zooming in/out instead of staying static) I can eliminate by reducing the Max Correction limits (horizontal & vertical) from 15 to 1 in Pass#2. Does not help with the first issue.

Also wondering if I should encode the short lossless AVI files to dvd as I go, or wait until they're all done, and combine together in an AVS and encode as one complete episode? Will be using 2pass in HCenc.

[poisondeathray]

I finally started the two videos I need to deshake. As predicted, it's taking alot of time and space, so I'm doing it in small increments. Have had at least two problems so far:
A fast moving car moving left to right, kept relatively in center screen by camera panning right to follow, has some horizontal jerks in it.
The other problem (artsy opening credits were zooming in/out instead of staying static) I can eliminate by reducing the Max horizontal & vertical smoothness to 1 in Pass#2. Does not help with the first issue.

What does the original look like ? Do you even need deshaker for this bit ? Maybe process it in segments (eg. use different settings for the credits)

You can try disabling zoom in the detection pass

Also wondering if I should encode the short lossless AVI files to dvd as I go, or wait until they're all done, and combine together in an AVS and encode as one complete episode? Will be using 2pass in HCenc.

If you have enough HDD space, I would do 1 episode in HCEnc

[spiritgumm]

Thanks for the dvd encode suggestion. I'll just have to make room on HD.
I still have the first video I did with Depan and Stab, and the brief car shot is fine in it. After I mux the audio into each version, I'll edit together the best shots from each in VideoRedo. And add any re-done clips like the credits.

I originally de-activated the zoom/rotation settting in Pass#1. I reactivated it to see if that would help with two problems, but didnt work.

[poisondeathray]

What was that AVI you uploaded ? That was the source or processed ?

What is it supposed to look like ?