HLSL SBS 3D shader filter

[Космик]

Forumers! Using HLSL shader below in MPC-HC to change between 2- and 3-dimensional SBS movie output on the fly. 2D conversion is essentially just showing 1 half of the SBS 3D image, stretched over the screen: [L|R] → [ L ].

sampler s0 : register(s0);
float4 p0 : register(c0);

#define width (p0[0])
#define height (p0[1])

float4 main(float2 tex : TEXCOORD0) : COLOR
{
    tex.x = tex.x / 2;
    
    float4 l = tex2D(s0, tex);
    float4 r = tex2D(s0, tex);
    
    float red = l.r;
    float green = r.g;
    float blue = r.b;
    
    return float4(red, green, blue, 1);
}

However, i need to not only show the L (or R) half of the SBS movie during playback, but also both original L and R halves, for a total of 3: [L|R|L]. Doing something like tex.x = tex.x + tex.x / 2; results in distorted image in the added one of the 3 halves. Can this be achieved by some simple manipulation of the above code?


ADD: solution below!

[butterw]

I don't know anything about SBS 3D.

Can you post a screenshot of the input/desired output and explain more clearly how the desired output should be obtained ?

[Космик]

Right, so SBS it looks like this:


[Attachment 55032 - Click to enlarge]


And the script above makes the picture 2D by displaying only 1 of the halves (tex.x = tex.x / 2;):


[Attachment 55040 - Click to enlarge]


But what is needed is the combi of 2, like this:


[Attachment 55041 - Click to enlarge]


And i haven’t been able to modify the above script to achieve that. For instance, tex.x = tex.x + tex.x / 2; results in this:


[Attachment 55042 - Click to enlarge]

[butterw]

/* L|R to L|R|L
with pixel shaders for each output pixel of coordinates (tex.x, tex.y), you need to return the value as a function of tex.
*/

sampler s0: register(s0);

float4 main(float2 tex: TEXCOORD0): COLOR{
if (tex.x < 2/3.) tex.x = 1.5*tex.x;
// else if (tex.x < 2/3.) tex.x = 0.5+1.5*(tex.x-1/3.);
else tex.x = 1.5*(tex.x-2/3.);
//could also write as:
//tex.x = (tex.x < 2/3.) ? 1.5*tex.x: 1.5*(tex.x-2/3.);
return tex2D(s0, tex);
}

[Космик]

Great results! While playing around with the filter, i understood that proportions have to be a bit altered as well. ¿Can this also be achieved:


[Attachment 55045 - Click to enlarge]

[butterw]

You can't change output resolution with a user pixel shader in mpc-hc, but you could change Aspect Ratio.
as in the previous example, you need to calculate the function that maps from output coordinates (tex) to input.
you could also just use mpc-hc features: I'm not sure you can define custom AR correction in mpc-hc, but custom PnS settings are possible:
https://forum.doom9.org/showthread.php?p=1918397#post1918397
You could also adjust AR manually.


Is this L|R to L ?

Edit: after further clarifications, it's not, it's the desired L|R|L output.

[Космик]

Resolution doesn’t need to be taken into account. Only the proportions between original [L|R] and the added [|L]. That is, in the new [L|R|L], the original part [L|R] needs to be 40 % of the total length and 60 % of the total height of [L|R|L], as per last image. Is that possible in the shader script?

[butterw]

Start from key tex points in [0 to 1.] coordinates (for a rectangle you need 2 corner points) to calculate the desired output/input mapping function.
You'll need to modify tex.y.

[Космик]

Sorry, the syntax is beyond me. That’s why i turned to the forum. Can you help, please?

[butterw]

The desired transformation isn't clear to me.

Define the input/output points in [0, 1] coordinates.

[Космик]

For instance, inside [L|R|L], according to the image above (corners upper left – lower right):

[L|R| ]　0,0 – 1280,720
[   |L]　1281,0 – 3200,1200

[butterw]

Please Label Input and Output Points. Normalise to [0, 1].

You want the third L to be larger ?

[Космик]

Yes. [L|R|] shall only be 40 % in width and 60 % in height of the whole [L|R|L]. Which would make [|L] 1½ times wider and 1200 / 720 times higher then [L|R|].

[butterw]

Resizing with pixel shaders

Going back to the original [L|R] to [L], I should point out that when there isn't a 1 to 1 mapping between an input and output pixel there is an implicit resize (stretching x-axis is a 1D-resize).
The default is nearest neighbour or hw linear sampling if your gpu support this. It's the fastest resize method but not the highest quality: bilinear is fast and should be ok here, but bicubic methods such as Catmull-Rom give sharper results.

Video player scaling is typically done using pixel shaders, the catch is that doing a 2D resize requires a 2-pass shader approach for an efficient implementation.

[Космик]

Quality is not a problem, so bilinear is perfectly fine. Can the code you provided above be modified to make [L|R|] 40 % in width and 60 % in height of the whole [L|R|L]?

[butterw]

Sure, do you want the area below the first L|R filled with white ?
I'll be using fastest resize.

#define bg 0 //black is float4(0, 0, 0, 0) which can be simplified as 0, white is 1

sampler s0: register(s0);

float4 main(float2 tex: TEXCOORD0): COLOR {
if (tex.x >= 0.4) {
tex.x = 0.5/0.6*(tex.x-0.4); // L|R|L
// tex.x = 0.5 + 0.5/0.6*(tex.x-0.4); // L|R|R
return tex2D(s0, tex);
}
if (tex.y < 0.6) {
tex.y = 1/0.6*tex.y;
if (tex.x < 0.2) tex.x = 2.5*tex.x; // 0.5/0.2=2.5
else if (tex.x < 0.4) tex.x = 0.5+2.5*(tex.x-0.2);
return tex2D(s0, tex);
}
return bg;
}

[Космик]

Splendid, that works great! Black color beneath would be more suited, i’d say. I don’t see colors mentioned in the code though, so is white the default one?
And if one would switch the layout to [L|R|R] instead, what needs to be changed? Line tex.x = 0.5/0.6*(tex.x-0.4); seems to be related to the right part, but i don’t see how it should be altered to switch the layout (in the first code iteration it was easy – just changing to 2/3, if i recall).
Same goes if proportions need to be edited (40 & 60 %) – just changing all instances of 0.4 and 0.6 to some other 2 values doesn’t seem to do it.

[butterw]

Try to understand the code, then modify it to introduce the required parameters.

for instance background Color:
#define bg 0 //black is 0 or float4(0, 0, 0, 0)
...
return bg; // instead of return 1


You need to calculate the function that maps from output coordinates (tex) to input.
The first LRL output example is an easier starting point.
for the first 1/3 L region:
output tex (0, 0) gets its value from input (0, 0)
output tex (1/3, 1) gets its value from input (0.5, 1)
so the mapping function is tex.x=tex.x*3/2 with unchanged tex.y.
you need to define a mapping function for each region.
say you wanted to start with R, the mapping function would become tex.x=0.5 + tex.x*3/2
the second 1/3 R region is the same as the first with a 1/3 offset on x:
tex.x=0.5 + (tex.x-1/3)*3/2

[Космик]

Right, got the color! So white is default. For [L|R|R], i don’t quite grasp it however. Do you mean changing tex.x = 2.5*tex.x to tex.x=tex.x*3/2 and tex.x = 0.5+2.5*(tex.x-0.2) to tex.x=0.5 + (tex.x-1/3)*3/2?

[butterw]

I've modified the code (but not tested it), uncomment the corresponding line for LRR output.
It is possible to introduce a parameter with value 0 for LRL and 0.5 for LRR
ex:
#define LRR 0.5 //0 for LRL
tex.x = LRR + 0.5/0.6*(tex.x-0.4);


background color is what you specify.

Any desired changes will be trivial once you understand the code and have introduced the right parameters.

[Космик]

Just + another ½ for R instead of L; so simple..! Sure, it works fine, and i’m almost finished. I now need to understand how to change the proportions for future use (currently 40 and 60 %). I’ve successfully changed all instances of 0.6 for another values to switch the vertical stretch of the [L|R|] part. However, changing 0.4 for the horizontal stretch is not as simple. Maybe you could give a complete code example below with, say, 0.2 and 0.3 (instead of current 0.4 and 0.6)? That will help in understanding of how to replicate that later for some other % values.

[butterw]

1. horizontal (x) and vertical (y) are independant.
2a. for vertical parameter #define yLR 0.6
2b. for horizontal parameter #define xLR 0.4
3. replace 0.4 instances by xLR
4. replace 0.2 instances by xLR/2
5. as commented 2.5 is 0.5/0.2 so should become 1/xLR
6. for future reference: best to figure out what parameters you need before coding

[Космик]

Sweet with switch vars for easy change! Have adjusted the code for 20%30, instead of 40%60:

sampler s0: register(s0);
#define xLR 0.2
#define yLR 0.3
float4 main(float2 tex: TEXCOORD0): COLOR {
if (tex.x > xLR) {
tex.x = 0.5 + 0.5/yLR*(tex.x-xLR);
return tex2D(s0, tex);
}
if (tex.y < yLR) {
tex.y = 1/yLR*tex.y;
if (tex.x < xLR/2) tex.x = (0.5/(xLR/2))*tex.x;
else if (tex.x < xLR) tex.x = 0.5+(0.5/(xLR/2))*(tex.x-xLR/2);
return tex2D(s0, tex);
}
#define bg 0
return bg;
}


Am missing something however, as the [|R] portion isn’t filling correctly:


[Attachment 55123 - Click to enlarge]

[butterw]

1. horizontal (x) and vertical (y) are independent.
tex.x = 0.5 + 0.5/yLR*(tex.x-xLR); is incorrect

0.5/0.6 is 0.5/(1-xRL)

[Космик]

Super! This is the final code for [L|R|R] with screen and film aspect ratio switches which i’ll be using:

#define BG  0
#define LRX 0.4
#define LRY 0.6
#define AR  1.7778

sampler s0: register(s0);
float4 main(float2 tex: TEXCOORD0): COLOR
{
    if (tex.x > LRX)
    {
        tex.x = 0.5 + ((0.5 / (1 - LRX)) * (tex.x - LRX));
        float migratio = 0.5 * (1 - (1.6 / AR));  // ↑↓
        if ((tex.y < migratio) || (tex.y > (1 - migratio)))  // ■□
            return BG;  // 16:10 / 16:9
        tex.y = (tex.y - migratio) / (1.6 / AR);  // ↕
        return tex2D(s0, tex);
    }
    
    if (tex.y < LRY)
    {
        tex.y = (1 / LRY) * tex.y;
        if (tex.x < (LRX / 2))
            tex.x = (0.5 / (LRX / 2)) * tex.x;
        else if (tex.x < LRX)
            tex.x = 0.5 + ((0.5 / (LRX / 2)) * (tex.x - (LRX / 2)));
        return tex2D(s0, tex);
    }
    
    return BG;
}

Thank you so much, butterw! It’s amazing what can be done with these shaders on the fly, without reencoding



I just have a theoretical question. You said earlier that MPC doesn’t change output resolution with user pixel shaders. That became very much noticeable during testing, so i switched to PotPlayer – which apparently does that, as it produces a much less grainy image. And VLC worked fine also. But MPC (both latest HC and BE) all don’t switch the res, resulting in a subpar image. Why was this a design decision in MPC, compared to Pot and VLC?

[butterw]

I'm not sure what causes your issue, all video players work the same.

Does the issue only happen with your pixel shader enabled ?
Check in fullscreen with input resolution matching the screen resolution (no frame resize required that way).
To my knowledge, VLC only supports a (single post-resize hlsl) pixel shader in legacy dx9 video output mode ?

mpc-hc/be: Please note that a pixel shader can work pre or post resize with different results.
Look into what renderer (default is EVR-CP) and what resize method you are using (bilinear is fast but will not produce a sharp image) and whether you have further postprocessing enabled (also in your other video players).

If this doesn't solve your issue, the issue could be the way the pixel shader/gpu handles resizing (code uses fastest resize).
For better quality with the initial LR to L code example, I would recommend using resizer_catmull4_x.hlsl.

[Космик]

Aye, it turned out to be related to whether filter is used in pre- or post-processing in MPC. Still, i like Pot for this better ((:


I just also saw that i sometimes actually need to vertically shrink the last [|R] part as a final adjustment. For instance, making it 0.9 in height to the original (didn’t notice the output difference to 16:9 and 16:10 screens until now). That without touching the left [L|R|] part. Is that possible? Differences marked in green (desired output on the right):


[Attachment 55133 - Click to enlarge]

[Космик]

[Космик]

Added a couple of lines like this:


#define BG 0
#define LRX 0.4
#define LRY 0.6
#define AR 1.7778

sampler s0: register(s0);
float4 main(float2 tex: TEXCOORD0): COLOR
{
if (tex.x > LRX)
{
tex.x = 0.5 + ((0.5 / (1 - LRX)) * (tex.x - LRX));
tex.y = (AR / 1.6) * tex.y;
return tex2D(s0, tex);
}

if (tex.y < LRY)
{
tex.y = (1 / LRY) * tex.y;
if (tex.x < (LRX / 2))
tex.x = (0.5 / (LRX / 2)) * tex.x;
else if (tex.x < LRX)
tex.x = 0.5 + ((0.5 / (LRX / 2)) * (tex.x - (LRX / 2)));
return tex2D(s0, tex);
}

return BG;
}


This kind of works, except that the reduced to 90 % [|R] part isn’t centered vertically, leaving a larger black bar at the bottom. And with movies, which aren’t letterboxed and already fill a 16:9 frame fully, at the bottom there are some artefacts instead (they are probably there with letterboxed movies as well, just aren’t visible since they are just black). I need help with this, please.


[Attachment 55237 - Click to enlarge]

[butterw]

Vertical: Shrink (stretch), Shift, Mask


#define LRRY 0.9
...
//tex.y = (AR / 1.6) * tex.y;
float offset=0.5*(1-LRRY);
if (tex.y < offset || tex.y > (1-offset)) return BG;
tex.y = (tex.y-offset)/LRRY;
...