Conversion 4:3 to 16:9

[Jaglo]

Welcome!

I wanted to share my way of converting 4:3 format materials to 16:9 format. This is useful for converting old movies or TV series or sports broadcasts (generally anything made before 2000).

Simply cutting out part of the image, i.e. removing the border at the bottom and top of the screen usually gives unsatisfactory results. In this way a significant part of the image is lost. I was looking for a tool on the Internet that I could convert the video differently, but I found nothing. That is why I invented the conversion method myself. Only a small part of the image is removed. If I would stop at that, there would of course be black bars on the left and right side. To avoid this, the image is stretched - only that it is not just a simple stretch of the whole, but a gradual one. The image in the center is kept in standard proportions, but the further away from the center, the bigger scaling is used. Thanks to this, the faces of actors or characters appearing in the center have unchanged proportions and look natural. Closer to the edge there is an extension, which, however, is not noticeable, because most of the time you look at the central objects. You can say that there is a trick to image perception.

In my opinion, the results are better than just cutting out or using black bands on the sides or filling an inactive area with a blurred image. Basically, when watching such prepared material on a widescreen TV one can forget that its source was 4:3 format material.

And now how to do it -> I used ffmpeg and wrote special scripts.


1) Software conversion of PAL material

Of course you need to have the ffmpeg.exe file and save the following script in a file named e.g. conversion.bat. Such an executable file is launched from the command line, giving the name of the input and output file as parameters.


ffmpeg -c:v mpeg2video -i %1.mpg -vf "yadif=1:-1:0,hqdn3d=8,crop=720:480:y=48,scale=1152:720:flag s=bicubic,split=11[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11];[v1]crop=96:720:x=0,scale=96+20:720:flags=bicubic[v1];[v2]crop=96:720:x=96,scale=96+16:720:flags=bicubic[v2];[v3]crop=96:720:x=192,scale=96+12:720:flags=bicubic[v3];[v4]crop=96:720:x=288,scale=96+8:720:flags=bicubic[v4];[v5]crop=96:720:x=384,scale=96+4:720:flags=bicubic[v5];[v6]crop=192:720:x=480[v6];[v7]crop=96:720:x=672,scale=96+4:720:flags=bicubic[v7];[v8]crop=96:720:x=768,scale=96+8:720:flags=bicubic[v8];[v9]crop=96:720:x=864,scale=96+12:720:flags=bicubic[v9];[v10]crop=96:720:x=960,scale=96+16:720:flags=bicubic[v10];[v11]crop=96:720:x=1056,scale=96+20:720:flags=bicubic[v11];[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10]
[v11]hstack=inputs=11,pad=1280:720:x=4,setsar=1/1" -map 0:v -map 0:a:1 -f mp4 -c:v libx264 -preset veryfast -profile:v baseline -level 1.3 -b:v 5000k -c:a aac -ac 2 -b:a 256k -sn -map_metadata -1 -map_chapters -1 %1.mp4


The key place of the script is the video filter, which follows -vf in quotation marks. The image is divided into 11 fragments with different scaling and then merged into a whole. The encoding parameters can be changed freely (e.g. video or audio bitrate).


2) Hardware conversion of PAL material

The difference is that the method built into the NVidia graphics card, not yadif, is used for deinterlacing. Also the first scaling is done by the graphics card.

ffmpeg -hwaccel cuda -c:v mpeg2_cuvid -deint adaptive -crop 48x48x0x0 -resize 1152x720 -r 50 -i %1.mpg -vf "hqdn3d=8,split=11[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11];[v1]crop=96:720:x=0,scale=96+20:720:flags=bicubic[v1];[v2]crop=96:720:x=96,scale=96+16:720:flags=bicubic[v2];[v3]crop=96:720:x=192,scale=96+12:720:flags=bicubic[v3];[v4]crop=96:720:x=288,scale=96+8:720:flags=bicubic[v4];[v5]crop=96:720:x=384,scale=96+4:720:flags=bicubic[v5];[v6]crop=192:720:x=480[v6];[v7]crop=96:720:x=672,scale=96+4:720:flags=bicubic[v7];[v8]crop=96:720:x=768,scale=96+8:720:flags=bicubic[v8];[v9]crop=96:720:x=864,scale=96+12:720:flags=bicubic[v9];[v10]crop=96:720:x=960,scale=96+16:720:flags=bicubic[v10];[v11]crop=96:720:x=1056,scale=96+20:720:flags=bicubic[v11];[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11]hstack=inputs=11,pad=1280:720:x=4,setsar=1/1" -map 0:v -map 0:a:1 -f mp4 -c:v libx264 -preset veryfast -profile:v baseline -level 1.3 -b:v 5000k -c:a aac -ac 2 -b:a 256k -sn -map_metadata -1 -map_chapters -1 %1.mp4

3) Sofware conversion of NTSC material

American materials have a slightly different format, so you need to use a modified script.

ffmpeg -c:v mpeg2video -i %1.mpg -vf "yadif=1:-1:0,hqdn3d=4,crop=720:400:y=45,scale=1152:720:flag s=bicubic,split=11[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11];[v1]crop=96:720:x=0,scale=96+20:720:flags=bicubic[v1];[v2]crop=96:720:x=96,scale=96+16:720:flags=bicubic[v2];[v3]crop=96:720:x=192,scale=96+12:720:flags=bicubic[v3];[v4]crop=96:720:x=288,scale=96+8:720:flags=bicubic[v4];[v5]crop=96:720:x=384,scale=96+4:720:flags=bicubic[v5];[v6]crop=192:720:x=480[v6];[v7]crop=96:720:x=672,scale=96+4:720:flags=bicubic[v7];[v8]crop=96:720:x=768,scale=96+8:720:flags=bicubic[v8];[v9]crop=96:720:x=864,scale=96+12:720:flags=bicubic[v9];[v10]crop=96:720:x=960,scale=96+16:720:flags=bicubic[v10];[v11]crop=96:720:x=1056,scale=96+20:720:flags=bicubic[v11];[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11]hstack=inputs=11,pad=1280:720:x=4,setsar=1/1" -f mp4 -c:v libx264 -preset veryfast -profile:v baseline -level 1.3 -b:v 5000k -c:a aac -ac 2 -b:a 256k -sn -map_metadata -1 -map_chapters -1 %1.mp4


4) Hardware conversion of NTSC material

ffmpeg -hwaccel cuda -c:v mpeg2_cuvid -deint adaptive -crop 45x35x0x0 -resize 1152x720 -r 60000/1001 -i %1.mpg -vf "hqdn3d=4,split=11[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11];[v1]crop=96:720:x=0,scale=96+20:720:flags=bicubic[v1];[v2]crop=96:720:x=96,scale=96+16:720:flags=bicubic[v2];[v3]crop=96:720:x=192,scale=96+12:720:flags=bicubic[v3];[v4]crop=96:720:x=288,scale=96+8:720:flags=bicubic[v4];[v5]crop=96:720:x=384,scale=96+4:720:flags=bicubic[v5];[v6]crop=192:720:x=480[v6];[v7]crop=96:720:x=672,scale=96+4:720:flags=bicubic[v7];[v8]crop=96:720:x=768,scale=96+8:720:flags=bicubic[v8];[v9]crop=96:720:x=864,scale=96+12:720:flags=bicubic[v9];[v10]crop=96:720:x=960,scale=96+16:720:flags=bicubic[v10];[v11]crop=96:720:x=1056,scale=96+20:720:flags=bicubic[v11];[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11]hstack=inputs=11,pad=1280:720:x=4,setsar=1/1" -f mp4 -c:v libx264 -preset veryfast -profile:v baseline -level 1.3 -b:v 5000k -c:a aac -ac 2 -b:a 256k -sn -p_metadata -1 -map_chapters -1 %1.mp4


The output image has a resolution of 1280x720.

Finally, note to all scripts. There is a filter that removes noise hqdn3d (Higt Quality Denoiser 3D). Its use depends on the quality of the material we convert. As a general heuristic rule I can suggest that for materials created in the 80s or earlier I used hqdn3d=8, for materials from the 90s hqdn3d=4, and for those from the first decade of the 21st century it was unnecessary to use this filter (you can remove this fragment).

[DB83]

I have not tested your script(s). I have no motivation in doing so to be frank. Whilst there may be a perception that one does not notice the stretching there is also a perception that most will not notice, or be bothered by, the borders that appear on 4:3 material displayed on wider screens. And there are many out there that stretch the full image and really do not care that the image is wrong.


Of course some may welcome your work and I do not wish to be depreciating it. But what you may wish to do is enhance your own topic by providing actual examples.

[manono]

That is why I invented the conversion method myself.

You invented nothing:
http://avisynth.nl/index.php/WarpedResize

Crop the amount you want from top and bottom and then use WarpedResize on what's left.

The whole idea is absurd and here's why:
https://forum.videohelp.com/threads/363684-4-3-to-16-9-avisynth#post2314433

You signed up here just to promote this?

[johnmeyer]

I've been aware of this idea since 2005, when HD was first becoming popular. I even created some warp filters for Sonic Foundry Vegas. However, I could never get anything that looked good, and as the camera panned, the effect could be quite disconcerting.

However, I would certainly be interested in seeing a clip of something you have produced. No need to upload anything because for this, a simple upload to YouTube will suffice. Make sure you have a clip where you have motion moving horizontally across the frame, as well as a clip where the camera pans horizontally.

Who knows, despite the two skeptical comments above, maybe you've figured out some magic numbers for the warp. That is the key to getting something that looks good.

However, having tilted at this windmill myself, I'm pretty content with the black bars on the side and, when I watch a 4:3 movie, I do so on my home projector. I zoom it out and it fills the screen because my screen is from the 1980s, so it is more square than rectangular.

[DB83]

BTW I was not being sceptical. Just expressing an opinion.


But someone pls tell me if I got this wrong. Here in Pal-land, as the OP states, the typical way of 'converting' 4:3 material to 16:9 does indeed involve cropping away the frame from 576 pixels to 432. Yet reading the script the OP suggests cropping to 480 pixels. So that is not an insignificant amount of picture loss.

[davexnet]

I used to have a Yamaha rear projection TV (about 15 years ago) that had this technology built-in.
You could set it so that any 4x3 signal received could be stretched, as described by the OP,
where the stretching was evident on the sides but not the center.

Pretty sure all I had then was analog cable

[dvd3500]

PAL land comment here:
I have seen the cropping DB83 mentions, black borders left and right, borders made out of blurring the frame but keeping the originally aspect of the original picture and while there was a transition to 16:9 video there were quite a few shows in 14:9.

I encoded some private videos to 14:9 because my relatives had a mix of 4:3 and 16:9 TVs. Not sure it was worth the effort though.

[azmoth]

You only have to think of all those old tv series originally filmed in 4:3 and then re-broadcast on Netflix as 16:9 which look natural widescreen without squashing. X-Files being one of them. They must crop the picture and zoom it in to get a perfect 16:9 without distortion losing only slight image? The original pilot of X-Files being a classic example of this revamping. But HBO broadcasts some original filmed widescreen shows in 4:3 which is apparently making a comeback in some circles. I prefer the 2.00 aspect myself as a happy medium of all ratios.

[Jaglo]

I didn't know the WarpedResize and I don't promote it. The idea is the same, but different realization. My script does not divide smoothly according to the formula, but divides into 11 parts.The number of parts and scaling was chosen in such a way that the negative effects are as little noticeable as possible and do not distract attention. The key here is not to apply too much scaling to the borders of the image. This can be a major problem when using the WarpResize function.

Examples with a chessboard should not be discouraged, because geometric objects rarely appear in the real film. The strength of this method lies in the fact that the viewer usually does not know what proportions the objects have. The brain takes what it sees as the truth.

Examples:

Oryginal material 4:3

[Attachment 55827 - Click to enlarge]

Simple stretch of the image 16:9 - the players are wide

[Attachment 55828 - Click to enlarge]

Cut 16:9 - you can see half the captions

[Attachment 55829 - Click to enlarge]

Panoramic conversion 16:9

[Attachment 55830 - Click to enlarge]

When the camera zooms in, the results are even better. For film scenes also. When the camera moves horizontally, a variable scale is most noticeable. But it must be remembered that this does not happen often (in the movies) and, in my opinion, has been minimized. For example, when I watch a game, it does not get noticed at all.

When the camera's movement is horizontal, the adjacent frames of the film (which just show a wider perspective) should be used, but this is unlikely to be achieved with an ffmpeg script.

[johnmeyer]

The reason commercial TV shows can be re-released in widescreen without stretching and without much cropping is that they have access to the original negative. Most TV shows prior to HD were shot on regular film cameras, and most of those were designed for theatrical films (i.e., widescreen). When the show was cut for broadcast, the sides were cropped off. The studio can go back to the negative, and this time use that material at the edges. There can still be the need to crop a little at the top and bottom, depending on the actual aspect ratio of the original negative.

[manono]

Most TV shows prior to HD were shot on regular film cameras, and most of those were designed for theatrical films (i.e., widescreen).

Uh, while they may have been shot using film cameras and while there may have been a little less cropping than when beginning with a 1:33:1 source, these 35 mm cameras weren't 'widescreen', but Academy ratio (1.375:1).

Film originally created in the 4:3 aspect ratio does not need to be altered for full screen release ... Full screen aspect ratios in standard television have been in use since the invention of moving picture cameras. Early computer monitors employed the same aspect ratio. The aspect ratio 4:3 was used for 35 mm films in the silent era. It is also very close to the 1.375:1 Academy ratio, defined by the Academy of Motion Picture Arts and Sciences as a standard after the advent of optical sound-on-film. By having TV match this aspect ratio, movies originally photographed on 35 mm film could be satisfactorily viewed on TV in the early days of television (i.e. the 1940s and '50s).

https://en.wikipedia.org/wiki/Fullscreen_(aspect_ratio)

In 1932, in refining this ratio, the Academy of Motion Picture Arts and Sciences expanded upon this 1930 standard. The camera aperture became 22 by 16 mm (0.87 by 0.63 in), and the projected image would use an aperture plate size of 0.825 by 0.600 in (21.0 by 15.2 mm), yielding an aspect ratio of 1.375:1. This became known as the "Academy" ratio, named so after them. ... The image area for "TV transmission" is slightly smaller than the full "Academy" ratio at 21 by 16 mm (0.83 by 0.63 in), an aspect ratio of 1.33:1.

https://en.wikipedia.org/wiki/35_mm_movie_film#Widescreen
Therefore. instead of a total of 25% of the picture being removed to create "widescreen" from "fullscreen", you might wind up cropping "only" 24%.

I can't confirm it, but I doubt those earlier television programs were shot using 35 mm cameras with an anamorphic lens on them to create a true "widescreen" video which could be cropped for television. Why would they? They weren't thinking ahead 30 years to today's "widescreen" television sets.

[Bonnevue]

First of all, thank you for your hard and excelent work, about 3 weeks ago I decided to look at my dvd-collection in order to put them on my NAS but I dont like the 4x3 AR on my 65 inch tv so I dabbled with changing the AR to 16x9.
Your solution is a very good compromise and it works for me.
I changed a few things, not that they are any good, it's just the way I code in ffmpeg and the result is very satisfactionaly.

ffmpeg -i "The Stand 1.mkv" -map 0 -vf "yadif=1:-1:0,hqdn3d=4,crop=720:480:y=48,scale=1152:720:flag s=bicubic,split=11[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11];[v1]crop=96:720:x=0,scale=96+20:720:flags=bicubic[v1];[v2]crop=96:720:x=96,scale=96+16:720:flags=bicubic[v2];[v3]crop=96:720:x=192,scale=96+12:720:flags=bicubic[v3];[v4]crop=96:720:x=288,scale=96+8:720:flags=bicubic[v4];[v5]crop=96:720:x=384,scale=96+4:720:flags=bicubic[v5];[v6]crop=192:720:x=480[v6];[v7]crop=96:720:x=672,scale=96+4:720:flags=bicubic[v7];[v8]crop=96:720:x=768,scale=96+8:720:flags=bicubic[v8];[v9]crop=96:720:x=864,scale=96+12:720:flags=bicubic[v9];[v10]crop=96:720:x=960,scale=96+16:720:flags=bicubic[v10];[v11]crop=96:720:x=1056,scale=96+20:720:flags=bicubic[v11];[v1][v2][v3][v4][v5][v6][v7][v8][v9][v10]
[v11]hstack=inputs=11,pad=1280:720:x=4,setsar=1/1" -c:v libx264 -crf 20 -preset veryfast -c:a copy -sn "test.mkv"

Source is The Stand, a miniserie 4 parts 1994, I ripped it to my computer with MakeMKV, so MPEG 2
And after the conversion I added the subtitles (dvdsub) with MKVToolnix from the original rip.

[azmoth]

The reason commercial TV shows can be re-released in widescreen without stretching and without much cropping is that they have access to the original negative. Most TV shows prior to HD were shot on regular film cameras, and most of those were designed for theatrical films (i.e., widescreen). When the show was cut for broadcast, the sides were cropped off. The studio can go back to the negative, and this time use that material at the edges. There can still be the need to crop a little at the top and bottom, depending on the actual aspect ratio of the original negative.

That explains it well. Having that OCN or interpositive is a goldmine for later releases or restorations if in good nick. Some film directors were well ahead of their time and realized that widescreen was the future. John Carpenter was one of those who shot in it in the early days!

[johnmeyer]

Most TV shows prior to HD were shot on regular film cameras, and most of those were designed for theatrical films (i.e., widescreen).

Uh, while they may have been shot using film cameras and while there may have been a little less cropping than when beginning with a 1:33:1 source, these 35 mm cameras weren't 'widescreen', but Academy ratio (1.375:1).

I should have not said "most." It depends on whether they were shot on 3-perf or 4-perf 35mm. 4-perf, which is what Seinfeld used, does provide some additional material at the edges that was cropped to get to 4:3. However, you don't get enough extra material at the edges to permit a full 16:9 result without cropping. The following image, which you can find on hundreds of sites, shows the crop. You can see some additional image in the blue 16:9 result that was never seen in the original release, but you also see that some material is lost at the top and bottom.

Starting in the 90s, when most TV shows were still being shot on film, but when HD TV was beginning to be discussed, some shows had the foresight to switch to 3-perf. Those negatives are closer to 16:9 and when re-scanned, the result can be 16:9 with very little cropping.

[jagabo]

The key here is not to apply too much scaling to the borders of the image. This can be a major problem when using the WarpResize function.

WarpedResize can give almost exactly the same result (minus the stepwise motion distortions) as your technique if you crop before the resize.

LWLibavVideoSource("source.mkv") 
AssumeTFF()
Yadif(mode=1)
Crop(0,40,-0,-40)
WarpedResize(1280, 720, 0.93, 1.0)

I prefer to watch the original 4:3 video pillarboxed on the screen. But if I had to pick one of the 16:9 versions I prefer the WarpedResize() method to the Jaglo method because of the smoother motion.

[Jaglo]

I was wrong - WrapResize has no problem with scaling at the screen edge. However, there is a problem that can be described as a "lens". You can see that the image is curved like in a lens. By dividing the image into several fragments it is possible to reduce it.

Motion distortion is visible in both methods (7s, 16s and 21s), although it is indeed more visible on the jaglo.mkv file. In my opinion, the reason is different video encoder and/or lower bitrate (6000 kb/s vs 3600 kb/s).

[manono]

I was wrong...

You should have stopped there, rather than try and rationalize your 'misguided' opinions and inferior filter. You object to pillarboxing on what was originally 1.33:1 material. Do you also object to letterboxing on 2.39:1 films when shown on your 1.78:1 television set? Following your logic of not liking black bars and wanting your entire television filled with active video, you should object. Maybe develop a filter to stretch video vertically. Or develop a television that stretches to adapt to the aspect ratio of what's being shown, kind of like YouTube's adaptable video player these days.

I prefer to watch the original 4:3 video pillarboxed on the screen.

Although that might not be the consensus of the public at large (I have friends and relatives that mess up their television aspect ratios purposely), I expect it is the consensus of the people at video sites such as this one, meaning people that actually know something about this subject.

[butterw]

I prefer to watch the original 4:3 video pillarboxed on the screen.

Although that might not be the consensus of the public at large (I have friends and relatives that mess up their television aspect ratios purposely), I expect it is the consensus of the people at video sites such as this one, meaning people that actually know something about this subject.

Stretching 4/3 movies is out of the question, but some vintage video content (music videos, maybe sports ?) could possibly be stretched to 16/9.

ex: for Britney fans

or this

[jagabo]

If letterbox/pillarbox bars are so offensive to you do what they do in movie theaters. Install curtains and adjust them to cover the black borders.

The only real reason I see for eliminating the black borders is to eliminate burn-in problems with plasma TVs. And maybe OLED screens.

[aedipuss]

If letterbox/pillarbox bars are so offensive to you do what they do in movie theaters. Install curtains and adjust them to cover the black borders.

The only real reason I see for eliminating the black borders is to eliminate burn-in problems with plasma TVs. And maybe OLED screens.

wouldn't worry about plasmas, my samsung is going on 10 years old and no sign of any burn-in.

[jagabo]

If letterbox/pillarbox bars are so offensive to you do what they do in movie theaters. Install curtains and adjust them to cover the black borders.

The only real reason I see for eliminating the black borders is to eliminate burn-in problems with plasma TVs. And maybe OLED screens.

wouldn't worry about plasmas, my samsung is going on 10 years old and no sign of any burn-in.

Someone I know had a plasma and watched mostly 4:3 TV. It had noticeable burn in after only a month.

[dellsam34]

I respect your choice but technically any form of stretch is wrong, It is less noticeable in steady scenes but when the camera is panning it is easily noticeable, Just add blurred edges to the sides if screen burn-in is a concern.

[lordsmurf]

and as the camera panned, the effect could be quite disconcerting.

It reminds me of panning my head with reading glasses on. Urgh.

Move book, not head.