Aligning together frames of different aspect ratio

[Mephesto]

So I have an HD and SD version, one 4:3 one 16:9. The SD video seems cropped on the sides as proven by the HD version, but the HD version is vertically cropped to make it widescreen.

I wanna combine the top and bottom crops of the SD video to mitigate the unnecessary crops, but I wanna know how realistic this is, because I can't align the SD frame properly and even then there will be empty space in the top right and bottom right corners, which are small and may or may not be easily filled in with some extrapolation tool like Region Remove.

And then there still would be color/brightness/contrast correction because the SD video is darker etc.

Here are the frames. Should I dream on or what?

[Cornucopia]

Dream. or "what". But don't expect these to work together. You maybe could blend using the lowest common denominator, but what you want is the highest possible multiple, which must include data that doesn't exist (except for in the original cinema master). Even going L.C.D. would require a great deal of positioning adjustment & color correction, just like you said, and for what?

Are these downloads? If so, look for a better master (BD/DVD/VoD).

Scott

[Mephesto]

I don't know what you're talking about. Expound.

I don't have a better BD. The SD version is a DVD rip but a good one, although I can't know if it's a re-reencode.

[poisondeathray]

This is just a rough idea of what is missing from the HD version

I just manually resized the width, height disproportionately of the top and bottom and position it to "fit" in after effects. It can be done in any NLE, photoshop, etc... . If you spend time on it you can probably get it to match up better in terms of fit and color, I just quickly brightened it with curves. Then you could crop and resize to 1280x720 . Maybe use content aware fill or impainting filters on the top right and bottom right corners

Notice the SD version is missing vertical dark lines on the bottom , but has the horizontal lines on the top. It makes you wonder what else is missing in other scenes. Probably not worth it IMO

[Mephesto]

How can you scale it unevenly and get it to match? I spent an hour with GIMP re-scaling and overlaying over the other picture with 50% transparency and couldn't match it perfectly. It seems the frame is a bit warped. Of course, I locked the aspect ratio while scaling.

Can you tell me the width and height of the SD frame? I was thinking of stacking the tops and bottoms with avisynth and fixing the borders and right corners with Region Remove vdub filter.

Does After Effects have content-aware fill for video? I'll download and install AE if it does. Is there a free version? I used AE once or twice in the past for a quick fix then uninstalled so a demo version might detect the old installation and refuse to run.

Notice the SD version is missing vertical dark lines on the bottom

Yes, thats because the screenshot you see is a still from scrolling, and with Xvid sucking as bad as it usually does, it didn't preserve that detail. I take back what I said, this is not a good DVD rip. I thought H264 was used because it was an .mp4 but it's MPEG-4 ASP.

It probably isn't worth it as you say, but it's an interesting experiment to try.

I resent this dumbass widescreen trend where they butcher the movies so they'll fit in 16:9 screens. I'm choosing not to play their game.

[poisondeathray]

How can you scale it unevenly and get it to match? I spent an hour with GIMP re-scaling and overlaying over the other picture with 50% transparency and couldn't match it perfectly. It seems the frame is a bit warped. Of course, I locked the aspect ratio while scaling.

Can you tell me the width and height of the SD frame? I was thinking of stacking the tops and bottoms with avisynth and fixing the borders and right corners with Region Remove vdub filter.

Like you said, the frame is warped. And I said "disproportionately" above - so that means unlock the aspect ratio scaling. I just "eyeballed" it until it approximately fit. I did the top and bottom separately. The numbers used w:h (from your jpg) were 156%,152% for the top section, 155%,154% for the bottom section. Note I scaled from the center point. You might have scaled off center for perhaps a better fit - there is evidence that this would actually work better (things weren't processed evenly somewhere along the line)

To blend the demarcation line, you can apply a vertical blur through a feathered mask

I didn't spend much time on the color (You can do a much better job matching) , I just made a quick curves adjustment

Does After Effects have content-aware fill for video? I'll download and install AE if it does. Is there a free version? I used AE once or twice in the past for a quick fix then uninstalled so a demo version might detect the old installation and refuse to run.

No it doesn't. You would batch process photoshop or other content fill software. Even avisynth has inpainting filters like avs inpaint . There are several logo removal filters that use inpainting as well

There are other software that fill in backgrounds from motion tracking adjacent frames e.g. mocha pro , but there is a quite of bit of manual work required in addition

[Mephesto]

Oh, if you did them separately then it makes sense, because they align almost perfectly. I see little imperfection. But this makes my plan even more discouraging.

Batch process with photoshop? I doubt it. This will be a purely spatial domain and won't take temporal elements into account, so I expect flickering and other weird artifacts in the filled corners during movement.

Screw it, I'll have to leave aside this project for another time when things improve.

Thanks for the wisdom as always.