Video Problem, Shift? Drift? Not sure what to call it....

[Owen Plato]

So I think I figured out what caused the problem and have eliminated it (a 90 degree BNC adapter). But now I have a video, that in its current state, is unusable. My questions are: What is this type of problem called and can it be edited to fix it? In the first picture the problem isn't so bad, you can see the area in the lower left hand side where the top/bottom and left/right edges of the video meet. In the second picture the effect is a lot worse and more towards the center of the frame. Any thoughts?

[johnmeyer]

Looks like good old-fashioned vertical roll. I'm not sure how this could happen in the digital world.

Fixing it will depend a lot on how uniformly consistent the roll rate is, although even if it is not constant, there are ways to track and recover it.

With the help of "StainlessS" over at doom9.org, I was able to track and then alter noise bars that resulted from using a movie camera to film a TV screen to which it was not synced. Here is the result showing what I started with, and then the result:

https://www.youtube.com/watch?v=qx26T6WOZ_4

and here is the thread where I was able to develop the software that tracked the noise bars:

Bad 1950s Kinescope - Hopeless?

You will see, if you read that thread, how it was done.

So, I'm sure it can be done, but it will require developing some unique software to track the bar in between the top and bottom of the video, and then "gluing" the two halves together.

[jagabo]

If the problem is stationary you can easily fix it like (the second image):

ImageSource("vlc.png") 
StackHorizontal(Crop(576,0,-0,-0), Crop(0,0,576,-0))
StackVertical(Crop(0,774,-0,-0), Crop(0,0,-0,774))

If it's rolling around you have to figure out some way of tracking the misalignment.

[johnmeyer]

Jagabo's code gives you the key piece of the puzzle: that will let you put the bottom on the to and the top on the bottom, and crop out the bar in between.

Because that blanking bar enters into the picture, you may end up with a video that isn't quite full height, except when the blanking bar scrolls into its normal position, off the screen. Thus, you'll have to figure out what you want to do with the slightly too-short height. I'd just crop it and leave a horizontal bar since, these days, people are more accustomed to seeing horizontal bars on their video as, for instance, when they watch an academy formatted movie on their 16:9 display.

[Owen Plato]

The problem is stationary for the most part. It doesn't roll from bottom to top or vice versa. I believe the example in the second picture is as bad as it ever gets, but it does change position from time to time not as a gradual move but by 'popping' (?) from one spot to another. But the problem isn't just that the top is now on the bottom, the right is now on the left. For instance, in the first picture, on the left side is a guard holding a sword (he's looking to the left) but he is supposed to be on the right side of the picture standing behind the guy with the staff.

As I said earlier, I'm pretty sure it was a BNC 90 degree adapter that caused the problem, signal loss most likely. We had three cameras feeding into an Apollo by Convergent Design via SDI (BNC) connections. And while I used BNC 90s on the other cameras as well, the base of those cameras never moved since they were remote controlled (only the 'eye' moved). But this camera was a handheld mounted on a tripod, so it got panned around a little bit.

*johnmeyer - I have started reading the forum and watched the YouTube video, just need a little more time to really dig into the Doom9 Forum.

*jagabo - While I've done a little bit of video editing in Premiere Pro and Photoshop, I have no idea how to use the code you posted.

I just want to say thanks so much for taking the time to read my post and throw some ideas my way, I was beside my self until now.

[jagabo]

Here's a fix for the first image (AviSynth, a text script based filtering system):

ImageSource("vlc1.png", start=0, end=23, fps=23.976) 
xpos = 192
ypos = 1070
StackHorizontal(Crop(xpos,0,-0,-0), Crop(0,0,xpos,-0))
StackVertical(Crop(0,ypos,-0,-0), Crop(0,0,-0,ypos))

[Attachment 40802 - Click to enlarge]

It's not quite perfect -- there are few lines at the top left corner that are wrong. You can move the cut point two lines lower to eliminate that but then there will be two lines at the bottom that are wrong. So you're going to lose a few lines one way or another. You'll have to get the xpos and ypos values for each segment with a different offset.

If there's motion during the shot the repair probably has to involve two consecutive frames. Because the lower portion is probably the top of the next frame.

The video is repairable. It will just be manually intensive.

[Owen Plato]

Thanks so much for that. I can breath a little easier now. Time to learn AviSynth!!

[jagabo]

Can you upload a few seconds of the video with some motion around the tears?

[johnmeyer]

If the problem is constant, and merely shifts from time to time, then jabago's code is all you need. No need to look at all that stuff I linked to.

Actually, since it is constant for seconds or minutes at a time, you could also easily do this in your NLE. I could do this in less than a minute in Vegas. That program also lets you mirror the image (i.e., flip it horizontally) which would fix your second problem. That can also be done with one command in AVISynth (FlipHorizontal, I believe).

That advantage to doing it in Vegas is that you can use its feathering and other tools to clean up any small issues where the two images are joined back together.

[Owen Plato]

Here is about a 5 sec clip with alot of dancing and movement through the left/right tear. No audio for some reason, Photoshop being finicky I suppose. Doesn't like some MOV files, worry about that later....

[manono]

That program also lets you mirror the image (i.e., flip it horizontally) which would fix your second problem. That can also be done with one command in AVISynth (FlipHorizontal, I believe).

I thought it was FlipHorizontal as well, at first. But it's not. You can kind of see that only the guy on the left is on the wrong side of the picture and he has to be moved over to the right. But the rest of the picture is okay once you put the bottom back on the top. Never seen anything like that before. Can an NLE really fix that kind of thing? My guess is it's an AviSynth-only fix. As jagabo says, in principle it's not all that hard, but will take a ton of manual work, depending on how often the image changes. And those join points can be feathered as well. Just more work.

[jagabo]

Here is about a 5 sec clip with alot of dancing and movement through the left/right tear. No audio for some reason, Photoshop being finicky I suppose. Doesn't like some MOV files, worry about that later....

Attached is the result of this script:

LSmashVideoSource("CLIP0000014_A.mp4") 
xpos=576
ypos=756
StackHorizontal(Crop(xpos,0,-0,-0), Crop(0,0,xpos,-0))
StackVertical(Crop(0,ypos,-0,-0), Crop(0,0,-0,ypos))

But there's some other problem with the video. The part below (in the original video) the horizontal tear is interlaced and has the field positions are swapped. There may be something more than that too. Unfortunately, there very little motion in that part of the frame (just a few finger tips), and it's very dark, so it's hard to say for sure what it is. But at least one field of the lower section is from the next frame. Here I intentionally brightened the top (after fixing) part of the frame so it's easy to see where the tear was. Inside the green square you can see some combing artifacts, and that one field is from the next frame (the fingers are ahead of the rest of the hands):


[Attachment 40805 - Click to enlarge]

Adding SwapFields() and vInverse() to that portion of the frame helps with those problems. But there's probably a better fix.

LSmashVideoSource("CLIP0000014_A.mp4") 
xpos=576
ypos=756
StackHorizontal(Crop(xpos,0,-0,-0), Crop(0,0,xpos,-0))
StackVertical(Crop(0,ypos,-0,-0).SwapFields().vInverse(), Crop(0,0,-0,ypos))

[Owen Plato]

Thanks jagabo. I showed your rendered video to my boss and he was very satisfied with the end result. Now I just have to do that for a little over 2 hours of video.

[Owen Plato]

BTW, are you using VirtualDub or some other application? Since I'm going to need to scrub through the video and find the frames for each change in the x & y positions, what do you recommend?

[jagabo]

The result of simply rearranging the quandrants may not look as good on shots where there is significant motion above and below the horizontal tear.

Yes, I use VirtualDub and a screen magnifier to view the result of AviSynth scripts. For encoding I usually use the x264 command line encoder.

[Owen Plato]

We understand. It's our best option at this point though, at least I think it is. And while we could see what you mentioned in the test footage, we are willing to live with it if for no other reason than we are a bit desperate.

[jagabo]

while we could see what you mentioned in the test footage, we are willing to live with it if for no other reason than we are a bit desperate.

But with the tear in the middle of peoples bodies they will be torn in half (see attachment with simulation). That is where it will be necessary to take the top/bottom parts of the picture from different frames -- requiring a more complicated script.

[johnmeyer]

As I suspected, this is really easy to do in Vegas. Here is the result:

Vegas_test.mp4

If you have Vegas (version 10 or later), here is the project (VEG) file I used:

Test.VEG

Finally, here is a screen shot showing how simple it is:



You simply put four copies of the same video file onto the timeline. You use the "track motion" to move each copy in the X-Y direction until each quadrant is where it is supposed to be. Vegas then combines them together automatically because of how it composites between tracks.

I did this quickly (a couple of minutes) so I only did a rough job of aligning the video. You can zoom in and make it as perfect as you like.

At each point where there is a "jump" change to a different arrangement, you can just add two keyframes to each track, one for the end of the original arrangement and then, one frame later, one for the new arrangement. Once you have the initial setup, it won't take long to do the rest, as long as you don't have more than a few dozen.