Side by side SBS or top bottom 3d video and modern codecs efficiency

[Bernix]

Hi,
iam very new to all this, just curious, because are small change between left eye and right eye, if codecs and its specs count with this. For example i have video for simplicity 2000x1000. Each field (left and right) is 1000x1000. And in most cases there is not big difference (is moreless but is otherwise 3d effect shouldn't work).
So is compression of modern codecs somehow prepared for this. Benefit from similarity of both eyes? I mean is there something in H.265 that has to be enabled to get better compression and benefit from same frame similiraties, or it doesn't work this way, probably in future specification or more likely never?


Sorry for as usual from me for majority stupid question, but iam really interested in this.


Thank you all.


Bernix

[johnmeyer]

I'm afraid I cannot understand, probably due to the translation software used.

[Bernix]

Hi,
no, my brain translated. Is in poore condition
In 3d movies there is similarity between left and right eyes. Left and right part of frame. So if modern codecs benefits from these similiraties not in time but in frame itself. I think i cant explain it more sorry. Probably will make a picture tomorrow.
Thanks for reply


Bernix

[Cornucopia]

What you are probably alluding to is the kind of spatial redundancy thing that MVC (3d-enhanced version of AVC) codec and similar takes advantage of.
It is analogous to efficiencies of P & B frame types, except spatially instead of temporally.

*Note that sbs, etc are 2d frame-compatible, so don't take advantage of this efficiency, whereas mvc is "service-compatible" which is like 2d base (compatible) plus (non-compatible) enhancement layer. And sbs, etc doesn't allow for transparent 2d use.

Scott

[hello_hello]

I have no clue for x265. I googled and found a build with an option called "sei-frame-packing-interpretation", but it was old and appears not to be a standard thing. x264 has a similar option for saving the SEI info when encoding, however as far as I know that's all it does. A player may or may not pay attention to it but as far as I know it makes no difference to how the video is encoded.

http://x264.janhum.alfahosting.org/fullhelp.txt

--frame-packing <integer> For stereoscopic videos define frame arrangement
- 0: checkerboard - pixels are alternatively from L and R
- 1: column alternation - L and R are interlaced by column
- 2: row alternation - L and R are interlaced by row
- 3: side by side - L is on the left, R on the right
- 4: top bottom - L is on top, R on bottom
- 5: frame alternation - one view per frame

My understanding is that to the human eye, vertical resolution is more important than horizontal resolution, so from that perspective side by side would probably be better than top/bottom.

The 3D fad has pretty much come and gone anyway. Is anyone still manufacturing 3D TVs?
Even "real" 3D increases the viewing effort by a fair amount compared to 2D, although "real" 3D can look fairly good. Unfortunately though, the majority of 3D releases are just 2D to 3D conversions and require even more effort to watch and tend not to look all that great.

[Cornucopia]

@hello_hello, you have it reversed: since human (and most other hunting animals) have eyes spaced apart horizontally but not vertically, the difference between their perspectives (IOW, the horizontal difference) is the determining factor in depth perception, Thus, formats which give the best horizontal resolution, even at the compromising of the vertical, will give the best 3D perspective. For frame-compatible file arrangements, this is the Top/Bottom, or TAB, format. As the Full version is more compatible with most codecs (which may hit a max resolution limit sooner using Full SBS), and the 1/2rez version still gives full resolution in the horizontal dimension.
The reason SBS became more popular has more to do with peoples' natural habit of putting of the 2 images side-by-side, with the 1:1 correspondence (and transcoding) with freeviewing (whether parallel or crosseyed) images, and with the fact that SBS is more compatible with Interlaced broadcast formats, while TAB really only works properly with progressive streams.

SEI is metadata, and the kind you mentioned is really designed to describe 2D-frame-compatible formats, not true BD3D "Frame Packing" hdmi format, nor Service-compatible formats such as MVC.

h265 already does have variations which support the equivalent of MVC/SVC, though I'm not as familiar with its syntax, as I still don't use h265 that much overall (no need with my current hardware players). Can't remember the name right now, but will look it up later when I have time at my workstation.

The 3D fad has come and gone, but 3D has not. 3D films are still being profitably shown, though not as many. 3D TVs are no longer built for the mass lowest-common-denominator market, but are still being built (though not advertised) for high end Consumer, and for Commercial markets. BD3D is diminishing in production, sad to say. 3D cams are once again a rarity.

Your idea of "real" 3D is probably still S3D, though just what I would call start-to-finish 2-cam productions. And yes, that is almost always better than converted S3D.
If you'd ever worked with either multi-cam imaging, Integral imaging or phase-coherent imaging (including Holograms), you'd know that nothing beats that kind of 3D and it is tons easier to focus on and naturally pay attention to. And without glasses, too. Some day...

Scott

[Bernix]

Hi Cornucopia, hello_hello,
thank you for time you spend answering my question. Now everything seems clear to me.
You mentioned there is less 3D videos and TVs. I see bright future of this, some reborn due to technology. I have to say, i never before have experience with 3d and must say, that trully 3d movie is worth to watch.
With technology that is cheaper and cheaper and more advanced year after year, there will not be choice for studios to make real 3d movies.


Yes I had on my mind spatial instead of temporal. This is right.


To problematics about 3d user watching TVs i think it will moved to different device more. 3d Tv i think is not best way to implement it. And technology still improving so i believe 3d has a great feature. Even first try probably wasn't so succesful.


Thank you again!!!
Wishing you all best
Bernix

[sneaker]

SEI is metadata, and the kind you mentioned is really designed to describe 2D-frame-compatible formats, not true BD3D "Frame Packing" hdmi format, nor Service-compatible formats such as MVC.

x264 --frame-packing 5 should give similar compression benefits as MVC (or given how good x264 is compared to other encoders, - probably better than all existing MVC encoders, at least for lower bitrates). While in addition to just the SEI there is also a slight optimization the main benefit is that since left and right view simply alternate x264 can make good use of the similarities between the views. So in principle it isn't really much different from MVC except it doesn't provide a backward-compatible pure 2D stream. Problem is player support ...

[hello_hello]

@hello_hello, you have it reversed: since human (and most other hunting animals) have eyes spaced apart horizontally but not vertically, the difference between their perspectives (IOW, the horizontal difference) is the determining factor in depth perception, Thus, formats which give the best horizontal resolution, even at the compromising of the vertical, will give the best 3D perspective.

Except.... a full HD 2D image would be 1920 pixels wide. When you halve it, you still have 960 pixels worth of resolution.
The same 2D image would be at absolute best, 1080 pixels high, but lets only count the actual picture, so for a 2.35:1 image you're down to around 800 pixels, and when you halve that you have 400 pixels, which is a lower vertical resolution than an NTSC DVD.

I can't say I'm sold on the "eyes spaced apart horizontally" argument. Normally, both eyes receive a full resolution picture and your brain performs some depth perception magic. I'm not sure why horizontal resolution would be more important for that to work. Similar horizontal and vertical resolution would probably be more ideal, assuming that's what each eye sees in the real world. 960x800 is closer to that than 1920x400. Even 960x1080 is closer to even than 1920x540.

And because our eyes are spaced apart horizontally, you could argue they're probably more sensitive to vertical resolution because they can work together more to resolve the horizontal stuff.

Anyway, I gave it a spin (Spline36Resize). A 720p image downscaled to 640x720 and back to 1280x720, and also downscaled to 1280x360 and back. Often there wasn't a lot in it. One part of a frame looked better when the width was downscaled and it looked better in another part when the height was downscaled. If I had to pick a loser though, it'd be the height downscaling. It lost here due to the pool rope, but overall it did more damage than the width downscaling. Chances are I could have hunted for a different frame with more fine vertical detail (lines and edges of objects in a vertical direction) and the Top Over Bottom version would've looked better, but I suspect the horizontal edges here would be more typical.

1. Original
2. Side By Side
3. Top Over Bottom

[Bernix]

Hi,
right now, deleted cca 10 row of nonsences. It results that each method has its pro and cons. Both have to be resized. Thats GPU thing on VR or Comps. And there is more common SBS for some reason. Probably because eye are left right in orientation. It reminds me pictures from 19/20 century. They were actualy SBS too
I don't believe one method is better than other and i think they are equal at the end.



Will subjectively try both method and let you know.

Edit - just comes to my mind, if it was invented in times of interlaced content, probably this can also favour SBS, but even not sure how it can works with interlaced content. 30 fields per second to left and 30 fields to right eye, but it is far behind my imagination...

Bernix

[Cornucopia]

When you are evaluating those variations, to be fully thorough in your comparisons, you should be primarily viewing them IN Stereo3D, not just in 2D. And, of course, you should be evaluating dual-perspective images, since that's the whole point of the exercise.
Those clips are also roundtrip encodings, so they'll have two artifact generations down from the original, not just one (as would be the case with normal S3D-formatted files being played back live to screen).

Back when interlaced was king, so was SD-quality. Many more issues, and many former formats. Interlaced S3D in SD quality was C*R*A*P! But I still had a collection.

Scott

[Bernix]

Hi,
just small notice. HTC vive and Oculus rift (mainstream VR not just for watching videos but also work, play and anything) they both have resolution 1080x1200 per eye. There can be also problem that favour SBS. I think both company knows better than we all why 0,9 ratio for eye. Yes it seems to be close to 16/9 for both eyes, but TAB image for eye has to be stretched to fit this width, while SBS fit it without any problem. Of course to get 16/9 or 2.35/1 has to be stretched when viewing, but SBS seems to me be better here.



Bernix