weird jitter at film edits in Pal retail dvds

[spiritgumm]

I bought a TV show (which was filmed, not videotape) on PAL retail dvd, and when there's an edit in the film (or scene change), sometimes there's a slight horizontal jitter involving 3 frames - bottom half left, then top half left (& bottom half back to normal), then top half back to normal. When it's does this every edit, it's pretty annoying. It's visible on a hacked-for-R2 dvd player, and in a PC, so I don't think it's my equipment (or is it?). Is this a glitch made by the company when they digitized the films?
Is there a quick fix for it if I'm reencoding with avisynth?

[filmboss80]

Is it an older TV series? Sometimes there are jitters at the film splices, and nothing is done to fix it during the film-to-video transfer. I have all the Monty Python episodes, and the out-of-studio skits were filmed in 16mm with these sorts of flaws. As far as I can tell, there were no significant digital repairs when preparing the series for DVD release. If you have a similar British TV series, I can imagine it might be the same for you.

You could try an image stabilization plug-in for Avisynth, but I really don't know if the results will be worth the effort. You may just have to learn to live with the flaws.

[spiritgumm]

it's a 1959 TV series, but the films look otherwise perfect. It's hard to believe it's the film itself. I have seen this once before - on a home recording (which happened to be Pal) of a broadcast movie (which was an American movie, not UK).
I've used Depan before, but am not an expert with the settings. Was hoping there was some standard method for fixing this type of flaw.

[lordsmurf]

The Robotech TV series is really bad about this, too, on the initial release box sets from 2002. I don't know if the later "remastered" releases fixed this, as I only bought it, and did not fully watch it. Going back to the most recent ADV Macross dub, the film had been re-scanned and lacks bad jumps. (Watch both for free on Hulu.)

Transformers VHS releases had been this way for some episodes, too, but the first Rhino DVD releases worked off newly-discovered broadcast masters, if I remember correctly. More recent releases were probably from the same transfers, but without the crappy audio effects additions made by Rhino in the early seasons.

The flaws are generally uncorrectable after the fact -- better film scanning is needed.

[jman98]

I love how people like spiritgumm post about problems with commercial DVDs and then refuse to say what it is they bought. Of course, if you want to eliminate the possibility that someone out there in the internet might know something specific to what you have to cover up your shame about admitting what you bought, that is your business. Personally, I don't care if you ever get your problem fixed or not. But if you insist on being all mysterious about it you just take away the possibility that someone might know something specific to your problem.

[poisondeathray]

There is no avisynth plugin or "standard way" of fixing this.

The stabilizers like depan work on whole frames, not part of frames (like top left, etc..)

You can try dejitter, but it doesn't work very well IMO. Maybe you'll get lucky fiddling with the settings

The only other approach would be frame replacement (either duplicate, blend or interpolate) as outlined in this thread from one of your other problem videos
https://forum.videohelp.com/showthread.php?t=324743

[spiritgumm]

Thanks yall for confirming this phenomena exists (and isn't my imagination). And thanks for the info, poisondeathray. I don't think I'll be using Badframes, etc. because of the manual work involved. In regards that other thread, I did look into Nuke - I suppose it's not as difficult as it looks, but I was intimidated. I'm looking into Adobe After Effects.
jman98, I don't understand your generalizing complaint. lordsmurf and filmboss80 used specific examples. If you want to know the series title, just ask. I didn't mention it because it's irrelevant to solving my problem, and for all I knew, it was some issue with my equipment. I'm also paranoid about getting deleted for being specific.

[jagabo]

Could you post a small segment that shows the problem?

[spiritgumm]

from the recent dvd release "Four Just Men"(1959), in case anyone is thinking of buying it. Might seem imperceptible on a PC, but on TV it's like a nervous tic.
jitterbug.m2v

[jagabo]

Thanks spiritgumm. I've never seen it at every edit like that.

AviSynth's Stab() or VirtualDub's DeShaker help with the overall horizontal shift but don't fix the flagging at the top of the frame. DeShaker:

[poisondeathray]

What's bizarre about this case, is you don't have black borders. When the frame warps out in 1 direction, the opposite side actually has content with active image pixels instead of black borders. This suggests the master or whatever preceded this PAL DVD was wider than 720 resolution

When attempting repairs on parts of frames (instead of whole frames), the other types of software that can handle this type of restoration warping are tracking clone type software, like imagineer mokey . Essentially what they do is track parts of "good frames" and replace them into the warped out sections. It still requires lots of manual work for the masks and area selection , but parts of it the process like the tracking is automated

[filmboss80]

After watching the clip, I can rule out film splice points. That's just plain weird. It may be either a time-based correction problem or an encoding error at each scene change, but I'm at a loss on that one. It is an otherwise very crisp, sharp film transfer.

[lordsmurf]

Wow, that's worse than Robotech was. it had a vertical jump with obvious scanning noise at the top of the frame. The solution was to dupe a frame, but took time. I did it as proof-of-concept once, for the first few minutes.

This one appears as a mix of timing and film alignment. I wonder if it was really made from a film scan, or a latter tape master. It almost looks like the latter, honestly.

Deshaker could fix the sideways motion. The timing waves really can't be fixed.

[poisondeathray]

as in the other thread , you can fix it with the interpolation technique

this one is actually easier to do than the other one, because the pattern is regular, and only 3 frame segments

it's not a conventional jitter, because the displacement is non linear warping (not just shifted over) - you can't just move half the frame back into place because it's displaced non linearly - you would need a very fine mesh displacement to fix it, or the clone method mentioned earlier. So it would be way easier to just fix the entire frame

[spiritgumm]

Thanks for all the input. I haven't looked at the video samples yet, but in the meantime, thought I'd pass along this reply from NetworkDvd:

This is to do with the way that the film physically behaves when it travels through the scanning gate – though there is some judder in evidence this was the best possible transfer taken from the original film elements, undertaken by experts at BBC.
[name deleted]
Account Manager

[jagabo]

I guess the splices cause the film to "bounce" as it passes through the scanning machine. The flagging occurs because the frame is being scanned as it's bouncing. Sounds like they need a better machine.

[poisondeathray]

I always thought when it gets caught in the machine , that the warp/bump in the scan results in vertical, not horizontal displacement - like a squeeze in some frames. But this one is horizontal

Also why would it occur every consistently a few frames into each scene change ?

I forgot to disable some settings in deshaker in above , and it screws up the motion (e.g. when the guy is nodding the camera goes up & down when it's not supposed to - I guess I'm a deshaker noob) , so I did it with a different stabilizer so it emulates the original more closely

[DB83]

I always thought when it gets caught in the machine , that the warp/bump in the scan results in vertical, not horizontal displacement - like a squeeze in some frames. But this one is horizontal

Also why would it occur every consistently a few frames into each scene change ?

I forgot to disable some settings in deshaker in above , and it screws up the motion (e.g. when the guy is nodding the camera goes up & down when it's not supposed to - I guess I'm a deshaker noob) , so I did it with a different stabilizer so it emulates the original more closely

Out of curiousity, what did you use to achieve that ?

The deshakers I have seen have always cropped the image but this looks complete (maybe I am wrong on that)

But it serves to ask the question. if you can do this then why cannot the dvd producers also do it ?

[poisondeathray]

Out of curiousity, what did you use to achieve that ?

The deshakers I have seen have always cropped the image but this looks complete (maybe I am wrong on that)

I explained in more detail the process in the link above with the other thread , with limitations , pros/cons of each method etc..
https://forum.videohelp.com/showthread.php?t=324743

Basically it involves replacing bad frames (the entire frame, instead of part of it), and the method I used here is interpolation (as opposed to replacing with dupes or blended frames). You can do it with free methods through avisynth, and I gave examples of how to do that in the other thread with mflowinter(). I used nuke in this example, and nuke's stabilizer for the 2nd example

I used Deshaker in the 1st example in this thread, because the newly generated frames were symmetrical after the interploation, but still horizontally displaced. I forgot to disable rotation and some other settings in deshaker, so it caused some new distortions. All you really needed was horizontal stabilizing after the warping was fixed, and that could have been done just as easily with avisynth as well (with depan, depanstabilize, stab etc....)

Deshaker can fill in black borders (ones that would normally be cropped in other programs) with past/previous frames, it has various border fill options, and they are quite good. In this example I didn't use any fill method except for scaling up the image slightly and a left black border - the amount of displacement wasn't that great in the first place, and nuke's fill options are harder to use than deshaker's (the latter of which are super easy to use and basically automatic)

But it serves to ask the question. if you can do this then why cannot the dvd producers also do it ?

For one like this, it's quite easy to fix, but other transfers can be really bad. There are severe limitations to the interpolation methods (I discussed them in the other thread) - It because this is a low motion clip that it works well. But for difficult transfers and multiple consecutive bad frames there are other tracked cloning methods and software specialized for that (like mokey).

In this case it's probably that the dvd producers don't want to put the time/effort into it (either a better machine transfer, or fixing it up after the fact)

[DB83]

@poisondeathray

Thank you for taking the trouble of clarifying this.

It really does seem that the 'machine' that the producers used is not up to the mark. But IMHO to blame the BBC for this is total bullsh*t when they did not produce the tv series in the first place.

[poisondeathray]

If that email reply from NetworkDVD was truthful, you would imagine the "experts at the BBC" would have access to the top of the line transfer equipment. And I wouldn't call that "minor judder" - especially if it was on every scene change

Is it possible that "the original film elements" had the problem , and it isn't the transfer causing the problem ?

This review mentions some issues with early episodes but doesn't go into any details
http://www.myreviewer.com/default/a132625/The_Four_Just_Men_DVD_Review#

whilst the quality of some of the early episodes on this DVD (including the pilot) are merely ‘perfectly acceptable’, many episodes are flawlessly transferred with little or no sign of wear and tear. The picture has a rich, stylish high contrast that is both a tribute to the original cinematography as well as to the careful transfer on this set.

[jagabo]

Is it possible that "the original film elements" had the problem , and it isn't the transfer causing the problem ?

I don't see how those distortions could occur on film. I'm sure the BBC has state of the art equipment but they probably have some lesser equipment too. In all likelihood they didn't deem this project worth the time and effort and used some older cheaper transfer process.

[DB83]

Having re-read the OPs first post again, there is a possible issue here. He does state that he is using a 'hacked' player to play the PAL dvds.

So I do wonder whether there is some timing issue here. The 'proof' would be to play one of these disks in genuine PAL equipment.

Just a thought.

[poisondeathray]

But a PC software player wouldn't exhibit playback issues with PAL DVD, especially if it was ripped to HDD

[DB83]

But a PC software player wouldn't exhibit playback issues with PAL DVD, especially if it was ripped to HDD

Like I said, just a thought. And I saw both the 'before' and 'after' rips.

But if the reviewer actually watched these he would surely have noticed some glitch. Unfortunatley, this series is only available as a box-set so I am not about to prove or disprove this.

[poisondeathray]

filmboss80 said "After watching the clip, I can rule out film splice points. "

what kind of characteristics determine this? (ie. what steps did you take to arrive at this conclusion) ?

Does "original film elements" refer just to the film itself, the negative , internegative ? or something else ?

[jagabo]

filmboss80 said "After watching the clip, I can rule out film splice points. "

what kind of characteristics determine this? (ie. what steps did you take to arrive at this conclusion) ?

I suspect he meant the problem frames weren't exactly at the splices, but a few frames later.

The extra thickness of the splice hitting something further along the film path, causing it to jog to the side a little, could explain this.

Does "original film elements" refer just to the film itself, the negative , internegative ? or something else ?

Who knows. And the BBC isn't likely to elucidate. In short, their answer said "Live with it."

[filmboss80]

I now believe this may be an A-roll / B-roll alignment issue in the film-to-video transfer. When editing the film negative for making the release print, alternating shots were on placed on separate film reels to conceal splice points. Thus, shots 1,3,5,7, etc. were on Reel A, while shots 2, 4, 6, 8, etc. were on reel B. In between shots 1,3,5,7, etc. on Reel A were sections of black film the exact length of shots 2,4,6,8, etc. on the other reel. The inverse applied for the B reel. (I don't want to get into a long, technical explanation about this, so if I'm not making sense, you can do a Google search on how motion picture film was edited and printed to copies a few decades back.) The A and B reels were aligned and run in sync through an optical printer to produce a release print.

The crisp quality of the sample clip causes me to believe that the video transfer might have been struck from the A/B negative reels. I have a feeling the scanner had an alignment issue between the reels, creating a slight timing shift at each scene change. Because this is not a well-known, in-demand TV series, there was likely not a very big budget devoted to cleaning everything up to perfection when making the digital master.

That's my theory, and I'm sticking to it.

You have a choice as to whether to enjoy the content and not allow the flaws to irritate you, or jump through lots of filtering and reencoding hoops to improve it.

[poisondeathray]

Thanks for the explanation

I have some old DVD's that I vaguely remember exhibit similar characteristics, now I have to go back and check

I find as I learn more about video, films etc... than I enjoy movies less nowadays , and I notice imperfections everywhere even in new productions - stuff I would have never noticed before

[spiritgumm]

Thanks spiritgumm. I've never seen it at every edit like that.

AviSynth's Stab() or VirtualDub's DeShaker help with the overall horizontal shift but don't fix the flagging at the top of the frame. DeShaker:

Will Stab() really minimize the horizontal shift? I didnt have much luck tweaking the function. Thought it would be quicker than Deshaker.