Filling black borders in Fullscreen mode

[butterw]

This post covers video player methods:
- to fill black borders that appear in fullscreen mode when the input has a different Aspect ratio than the screen.
- to crop the input to aspect ratio (and in windowed mode to resize the display window accordingly).

EDIT: any replies should be directly related to the topic of software playing.

In a recent thread about a 4/3 to 16/9 stretch method, it was suggested that filling the black pillarboxes that appear in full-screen mode might be preferable.
What video players/realtime methods can achieve this easily and efficiently (high-quality lightweight methods, no re-encoding required) ?

Black border replacement could be: a lighter solid color or a color gradient, an image, an unfocused blur (of an image taken from the video or an updated blur of the video).
Additionally, the option to mask/crop black/blurred borders when they have been burnt-in to the source video is desirable.

Why replace black bars ?
- Black bars have been used historically on TVs for pillarboxing/letterboxing movies. Black0 (RGB: 0, 0, 0) offers the best contrast ratio vs the frame and is the default option in current video players. However it should be noted that black is not typically used for internet portrait mode videos (1:1, 4:5, 9:16). Lighter colors or an unfocused blur background can provide a more casual viewing experience. An updated unfocused blur avoids a large part of the screen remaining static and provides an adaptative color accent vs the displayed scene.
- Independently, fullscreen black bars may look poor on PC monitors because of display limitations. This issue is most noticeable in dim lighting conditions: imperfect blacks are obvious with IPS panels with edge-backlighting, but are also present to a lesser degree with other types of panels. This issue is exacerbated by monitor/Windows display settings: the settings used on PC to reduce eye-fatigue / Night Mode do not produce the darkest blacks.

--
Current options: All good video players on PC and mobile offer crop-to-screen, custom crop-zoom and a basic stretch method. PC players have a windowed mode (optionally with a minimal/hidden interface) which displays the video at the source Aspect Ratio.

- In Mpc-hc and Mpc-be manual zoom-crop adjustement can be useful to crop some border-edge artifacts:
(+) Numpad 9
(-) 1
Reset Pan&Scan: 5
A similar feature is available in mpv.

- The following manipulation allows to use an image as the background for a bordeless mpc-hc window:
... Open the video in bordeless mode: View > Presets... > Minimal (1)
... Optionally, set the Windows taskbar to autohide in Windows taskbar settings (or better disable it)
... Display the desired image in fullscreen with an image viewer (ex: Xnview Classic), or use your desktop background with hidden icons.
... Alt-Tab to mpc-hc
... You can use 200% zoom if desired (Alt+3)
EDIT: It may be simpler to use an avisynth script instead.

--
Windowed crop feature (crops burnt-in black bars and resizes the display window):
It also can be useful when landscape mode was used for capture, but the relevant content is only present at the center of the screen (ex: tighten the frame by cropping to 1:1)

- In MPV: using the crop filter from ffmpeg, which works with (w, h, [x0, y0]) box coordinates for the output. The image width and height are available as iw, ih.
To crop sidebars to 9/16 Aspect Ratio: mpv --vf=crop=ih*9/16:ih "video.mp4"
To crop sidebars to 1.25 AR: --vf=crop=ih*1.25:ih

- In VLC: Video > Crop > 4:3. CTRL-H: toggles minimal interface.

VLC Command-line launch options:
# Windowed crop of burnt-in black bars (the crop parameter must be defined as the ratio of 2 integers)
vlc --crop=4:3 "video.mp4"

# Widescreen Zoom in Fullscreen (for 16/9 content encoded as 4/3):
vlc -f --crop=16:9 "video.mp4"
In mpv, this can be done automatically using an auto-profile based on filename (ex: video_wsz.mp4).

Mpv can also automatically crop all black borders at playback with portable_config/scripts/autocrop.lua.
It's based on (ffmpeg) cropdetect which provides the parameters to the crop video filter.

--
Setting video player background color to a value different than the default black0
- PotPlayer has a Preferences > Video > Background color setting.

- Mpv also has an option to set the screen background color, ex: gray160
mpv --fs --background=0.627 "video.mp4"
In addition to a gray level, color can be specified as R/G/B each in the range [0, 1] or with hexadecimal color #ff00aa
The background color change can be hotkey-triggered by customizing input.conf.


Background Color choice
A few colors seem to work quite well.

On my setup (VA panel) with non-daylight non-video-optimized monitor/windows settings, I've found that black16 (limited black) gives better results than the default black0 (tested with old black&white films).

--
Alternative solutions
- On PC, an avisynth script could be used. But it requires creating (or generating) a script, has a significant cpu overhead and loading mkv requires prior-indexing. These issues can at least partially be adressed by using a Avisynth Directshow filter (applies a script to the video loaded in the player).
EDIT: Mpc-hc, mpc-be, mpv can open .avs scripts with Avisynth installed. Realtime Avisynth script: blur/fill_bb.avs.

- A gpu pixel shader (barMask.hlsl) works. But user-shaders only have access to the post-resize black bars area in the EVR-CP renderer and the required information about the source AR isn't currently made available to the shader, meaning you need to specify the desired AR in the parameters. Simple color substitution can't be used because the color black is frequently present in the source video. A realtime unfocused blur isn't simple to implement and would be quite gpu intensive on a 1080p screen.

[jagabo]

What's wrong with black borders? They're easy to ignore. Whereas moving blurred crap is not.

[butterw]

Full screen (with no borders) provides the best viewing experience. Fullscreen with black borders is fine for movies, but for other types of content I typically prefer crop to screen on mobile and windowed mode on PC (no black borders, no upscale). I'll need do more testing to see if grey/pastel solid borders are an improvement over black in the latter case.

Unfocused blur is currently very popular for borders in internet/mobile videos and IMO constitutes a superior option to plain color. Part of the appeal is that the border is adaptative vs the displayed content. Updating every frame is overkill however and not at all efficient, a fixed blurred image seems like the preferrable option.

PS: A lot of internet videos do have burnt-in moving blurred borders, which some may find distracting. My pixel shader barMask.hlsl allows to mask them in black, solid color or gradient (use post-resize in mpc-hc/be with default EVR-CP renderer and specify the masking AR).

[Cornucopia]

I totally and vehemently disagree.

Viewing the videos AS DESIGNED, IN THEIR PROPER, ORIGINAL ASPECT provides the best viewing experience.
It is fine for movies AND EVERTHING ELSE.

With just a little bit of practice, it is possible to break former habits, and learn to NOT EVEN NOTICE the black bars.


Scott

[jagabo]

I'll need do more testing to see if grey/pastel solid borders are an improvement over black in the latter case.

It's not. Some lasma TVs had that to prevent burn-in problems. It looked terrible.

[butterw]

@Cornucopia: Nobody is going back to black borders. Some people like them, most people don't.
The topic in this thread is what options are currently available in video players to fill them.

For some types of content, improvements over black borders in fullscreen are possible:
https://forum.videohelp.com/threads/393762-ImageBorders-FrostyBorders-Scripts

[jagabo]

For some types of content, improvements over black borders in fullscreen are possible:
https://forum.videohelp.com/threads/393762-ImageBorders-FrostyBorders-Scripts

Possible, yes. Improvement, no.

[butterw]

From a quick test with some old 4/3 videos on a 16/9 monitor I find that masking with solid color grey160 (soft grey not dissimilar to the side bars on the videohelp website) provided an improvement over the default video player black bars (black0).

Tested in mpc-hc with barMask.hlsl pixel shader with the following parameters:
#define Mode 4
#define BorderColor 160/255.
#define yxRatio 4/3. //1.25
#define Xshift 0

Please note this Shader can be used Pre-resize for burnt-in bars, otherwise it must be used post-resize in fullscreen with EVR-CP renderer.

The alternative is to use "mpc-hc >Pan-and-Scan Widescreen Zoom". "Video Frame > Touch Window from outside" is not applicable because the black bars were burnt-in to the video). This is equivalent to VLC crop AR 16/9. Obviously the disadvantage of this method is that a significant amount of pixels from the top and bottom of the screen are cut. Some adjustement is possible and maybe necessary, but my general conclusion is that smaller black bars are less of an issue.

Finally I tested stretching to 16/9 which gave awful results because of the huge AR error.

[lingyi]

Off topic and 100% with Cornucopia and jagabo.

Avoiding or modifying black borders is a bastardization of a proper viewing experience.

Black borders are an issue because of an improperly darkened room and too small screen. In a properly darkened room, the black bars blend into the background. I've bumped int the side of my 40" set when leaving the room because I didn't know where the edge of it was. And I never do any casual viewing on anything smaller than that set. My main viewing is 6' from my 55" plasma, which properly fills my field of vision.

[lingyi]

Some people like them, most people don't.

It's not a matter of liking or not liking them. It's a matter of appreciating a good video experience or not.

This forum is Videohelp and caters to members of all levels of video appreciation, knowledge and experience. However, most the regulars like Cornucopia, jagabo and myself, hold to the belief that videos should be viewed as accurately as possible. Not bastardized in ways contrary to that.

[butterw]

Unless I'm mistaken 4/3 video was never intended to be viewed with black bars ? The videos in question weren't meant to be watched in a darkened room either. They are just some old video clips. Most could be watched on a phone cropped without degrading the viewing experience at all (smaller screen means less visible artifacts).

It is indeed a bastardization to encode with black bars (or nowadays to upload 9/16 video encoded with blurred bars).

Keep the replies on topic please.

[smrpix]

Unless I'm mistaken 4/3 video was never intended to be viewed with black bars ? .

Correct. They were intended to be viewed on a 4:3 screen.

The videos in question weren't meant to be watched in a darkened room either.

They were meant to be watched in a movie theater in a darkened room or on a tv where darkening was optional.

They are just some old video clips. Most could be watched on a phone cropped without degrading the viewing experience at all (smaller screen means less visible artifacts).

Easy solution. Encode them as 4:3 and let the display device handle the rest. And I doubt the creators think of them as "some old video clips."

It is indeed a bastardization to encode with black bars (or nowadays to upload 9/16 video encoded with blurred bars).

Keep the replies on topic please.

Now you've gone off the deep end.

[jagabo]

It is indeed a bastardization to encode with black bars (or nowadays to upload 9/16 video encoded with blurred bars).

Of course you don't include the black/blurred bars in the encoded video.

Keep the replies on topic please.

You're the one changing the topic. You originally asked about players that automatically filled the left and right with something other than black. Now you're talking about encoding.

[butterw]

The topic is about how to fill sidebars (in the case of original 4/3 material) to anything other than pure black without re-encoding, not whether anyone disagrees with the intent. Black borders work ok, but I would suggest that it is not always the optimal fill (vs. content, ambient light, display panel technology).

Burnt-in sidebars (black or blurred) are commonplace in currently available internet videos clips, video players need to handle this case also (by masking in fullscreen and cropping in windowed mode).

[ProWo]

I totally and vehemently disagree.

Viewing the videos AS DESIGNED, IN THEIR PROPER, ORIGINAL ASPECT provides the best viewing experience.
It is fine for movies AND EVERTHING ELSE.

With just a little bit of practice, it is possible to break former habits, and learn to NOT EVEN NOTICE the black bars.


Scott

Agree totally with you

[butterw]

Please feel free to your own thread on how 4/3 video should be viewed (maybe as a poll ?). However it's off-topic in this thread.

[Dreamer]

PotPlayer - Preferences - Video - Background color: []

[DB83]

Please feel free to your own thread on how 4/3 video should be viewed (maybe as a poll ?). However it's off-topic in this thread.

It's 'Off Topic' since there is no topic.


EVERY software player recognises that if you display 4:3 video 'as is' in a frame larger than that it will fill the redundant space in the color of its chosing. I have not researched this but I suspect that the default frame is always BLACK with the live video superimposed over that. NO player AFAIK does anything significanctly different to that.


I even wrote in the thread referred to in the OP that many will not even visually appreciate that there is a border around a live video.


I have seen instances where the 'empty' space beyond the original 4:3 has been filled by blurred images of the original (re-encoded so not valid to the discussion). I am no expert here but I would still argue that the human eye reacts to such images and focuses somewhat incorrectly on these to detriment of the actual 'live' image.


As has been pointed 4:3 video should be displayed 4:3. And the discussion has no value beyond that.

[lingyi]

They are just some old video clips. Most could be watched on a phone cropped without degrading the viewing experience at all (smaller screen means less visible artifacts).

This explains your level of appreciation or more correctly lack of appreciation of videos. Carry on with your pointless search of how to bastardize your videos. *Plonk*

[TimA-C]

Do you not think that if there was ANY advantage or improvement in picture quality that the "big boys" like Sony, Samsung, Panasonic, LG etc. etc. etc. etc. etc. etc. etc. would have been ALL over this with their TVs? You'd have a specific button, prominently displayed on your TV remote to control the background colour . . . The fact that they don't do this REALLY should tell you that you're barking up the wrong tree. Oh well, it's your time to waste.

[butterw]

@lingyi this may not be a thread for you.

@DB83
In a rare on topic reply (thanks @Dreamer), It was reported that Potplayer supports custom frame background color. The feature might not be useful to you, that doesn't make it useless.
A border is recognisable unless the border has the same color as the sides of the image. If the picture is blue around the edge, blue could be used as the border color.

> I have seen instances where the 'empty' space beyond the original 4:3 has been filled by blurred images of the original (re-encoded so not valid to the discussion). I am no expert here but I would still argue that the human eye reacts to such images and focuses somewhat incorrectly on these to detriment of the actual 'live' image.

Different people may perceive the same thing differently. Blurred borders are very popular in any case.

It's actually possible (but not simple) to implement realtime blur of borders with a shader in any video player supporting EVR-CP renderer. However if you weren't convinced by the Avisynth script result, there is no reason to believe that a shader would fare any better.

I have implemented a dynamic fill mode in my updated pixel shader barMask.hlsl v1.4 "PillarboxFill43". Not as good as a blur but very lightweight and it can works whether the input has burnt-in sidebars or not (it also works for different aspect ratio than 4/3). A static BorderColor (black or otherwise can also be used).

[butterw]

@TimA-C: I don't follow TVs.
Screen display technologies are relevant to this subject however (IPS bleed, OLED burn-in, black light level), see also Blue light filter, Bias lighting / Philips Ambilight.

My only claim is that on my setup changing the sidebar color was an improvement for me on some old video clips (I'm not claiming it's an overall improvement, and I likely prefer blurred borders anyway).

[lingyi]

Do you not think that if there was ANY advantage or improvement in picture quality that the "big boys" like Sony, Samsung, Panasonic, LG etc. etc. etc. etc. etc. etc. etc. would have been ALL over this with their TVs? You'd have a specific button, prominently displayed on your TV remote to control the background colour . . . The fact that they don't do this REALLY should tell you that you're barking up the wrong tree. Oh well, it's your time to waste.

But, but those "old videos" whatever that means, are meant to be viewed on tiny phone screens that don't accurately display the full quality of them and draw your eyes to the black bars because they don't properly fill your vision.

And we should be able to automatically override the carefully crafted effects in modern videos like these, because how DARE they not use full screen!

https://www.youtube.com/watch?v=-z3iyydFevQ
https://www.youtube.com/watch?v=tkBAvP85jvc

And while we're at it. We should be able to colorize any B/W video because everything should be be in COLOR. And, and, we should be able to change any non-CGI effects we don't like. Oh wait, George Lucas already did that!

*SIGH*

[butterw]

VLC offers a windowed crop feature. Video > Crop > 4:3
CTRL-H: switch to minimal interface

Useful if the input video has been encoded with burnt-in sidebars.

It's also useful if camera landscape mode was used, but the relevant content are in the center of the screen. 1:1 crop is available in the list.
It's possible to add 3:4 to the crop AR list (9:16 didn't work as desired in windowed mode).
Preferences (Advanced) -> Video -> scroll down to "Custom crop ratios list"
3:4
Restart VLC.

[jagabo]

MPCHC, MPCBE, PotPlayer, KMPlayer, SMPlayer all have zoom (with cropping and manual pan/scan) options.

[butterw]

Windowed-crop for 4/3 in VLC: the window is resized (no zoom).

In mpc-hc, mpc-be manual zoom-crop can be useful to crop some border artifacts:
(+) numpad 9
(-) 1
reset PnS: 5

[TimA-C]

Do you not think that if there was ANY advantage or improvement in picture quality that the "big boys" like Sony, Samsung, Panasonic, LG etc. etc. etc. etc. etc. etc. etc. would have been ALL over this with their TVs? You'd have a specific button, prominently displayed on your TV remote to control the background colour . . . The fact that they don't do this REALLY should tell you that you're barking up the wrong tree. Oh well, it's your time to waste.

But, but those "old videos" whatever that means, are meant to be viewed on tiny phone screens that don't accurately display the full quality of them and draw your eyes to the black bars because they don't properly fill your vision.

And we should be able to automatically override the carefully crafted effects in modern videos like these, because how DARE they not use full screen!

https://www.youtube.com/watch?v=-z3iyydFevQ
https://www.youtube.com/watch?v=tkBAvP85jvc

And while we're at it. We should be able to colorize any B/W video because everything should be be in COLOR. And, and, we should be able to change any non-CGI effects we don't like. Oh wait, George Lucas already did that!

*SIGH*

We DO have the right to do all those things - as does ButterW - but it doesn't mean that we should. (Thinking of the scene-by-scene re-make of Psycho . . . ! ) Oh well, at least it keeps him occupied and off the streets!

[butterw]

Couldn't bring myself to watch through those uninspired 16/9 music videos, but it's pretty clear they have nothing to do with the topic of this thread.

[smrpix]

Couldn't bring myself to watch through those uninspired 16/9 music videos, but it's pretty clear they have nothing to do with the topic of this thread.

If you don't understand the point in the context of this thread you started, you're beyond hope.

[butterw]

Software related replies only please.