Capturing live 3D video from a camcorder using HDMi and 4G

[boris_blackmilk]

Hello all,

I'm pretty certain this hasn't been done in the consumer space yet, nor has anyone been crazy enough to try it. I am trying to, with a mind for budget (not only cost, but power consumption and weight) capture a live 3D video stream from a 3D camcorder on an unmanned aerial vehicle, and stream it over 4G to a ground station.

I am reaching a roadblock at a reference to the FPGA implementation to capture and process such a stream, and package it up for transmission on a 4G mobile internet connection. I'll quote my design idea, then some references.

For the camcorder, I was looking at the 3D Hero2 HD, but I've heard they're pretty baulky, with sync problems and problems with cameras not activating over the link cable, giving only one viewpoint. So they're cheap? I don't want to program my headaches due to non-sync'ing in my implementation. I do get migraines with shutterglasses, but that was on a twenty year old SGI system - worked great when it worked, wasn't all that reliable though, which could've actually caused the headache.

So, I went and started looking at Sony cameras, theres a HDRTD20V camcorder that weighs 420g, and is meant for home users capturing home videos - which should imply there is no HDCP in the system. Then again, I could be wrong, but God help me if I am.

On the ground station side, a Samsung XE303C12 ChromeBook with the Mali T-604 GPU, which has stereoscopic 3D support, and an Acer passive 3D LCD monitor, along with a BladeRF running an LTE microcell to provide the link. Now, no one tell me about the legal implications of blasting a 20MHz 4G signal out across town - I'm a radio amateur, this is my life.

I've asked Samsung and ARM about details on the Mali T-604, waiting to hear back from them. As you can see, pretty much everything on this project is custom coded. Which brings me, at last, to the capture device: an Atlys Spartan 6 board, or a NeTV.

The end objective, apart from not spending ludicrous amounts of my hard earned cash, is to have a system that is as light as possible, that uses the least amount of power as possible - allowing me to load up more batteries, and not drain them out as quickly on video equipment, keeping my 3kg lift capacity UAV in the air longer.

I would prefer the NeTV, due to it having the HDMi input, wifi, and a Linux system to work with - I want to keep this as open source as I can get. It still has an FPGA that needs to be programmed, and I understand its "intended" use - strip off the HDCP and "overlay" other information on a live 3D stream, but could it be used to capture 3D video off a camcorder, package it up (VLC server or something?) and stream it over the wifi to a 4G modem, down to the base station, for display in 3D?

I should start looking at the NeTV FPGA implementation, it would capture the HDMi and process it. At the point where its in a digital format able to be streamed is where I start coding my own implementation suitable for live streaming over 4G. As mentioned, my only encounter with 3D was that SGI system, where it caused an almost immediate migraine - I was hoping A) that it was the fault of really old gear, and B) passive 3D wouldn't do that to me. However, I don't have the ChromeBook or the passive 3D monitor yet, I'm waiting to buy them, and designing the system and software in between.

Technically, I don't see why this is not possible - I mean, we are talking some grey areas if the camcorder outputs a HDCP stream (which would suck), and the activation of my own 4G cell, and I intend on breaking no laws in doing this - this might be a case of curiosity killed the cat, but I could also write a great engineering report out of it...

[Cornucopia]

I would very much doubt there would be HDCP on "original material" (unless a cam had some hidden "anti-bootleg-screening" circuit), so I think you're OK there.

However, part of the deal with 3D over HDMI (which expects it to be v1.4-compliant or better/newer) is the handshake between devices. When they talk to each other, the receiving device has to announce to the sending device that it has an EDID that is 3D-capable (in one or more of the available format choices), then the 2 devices negotiate which choice is most appropriate, and only then does the sending device transmit a 3D signal.
So, when designing the wireless transmitter & receiver, you'll need to "spoof" a 3D-compatible sink EDID on the incoming HDMI stream to the xmitr, and also spoof a 3D-compatible source EDID on the outgoing HDMI stream of the receiver. Otherwise, negotiation for 3D capability will fail and then, even if you were able to surmount all other obstacles, you would only be getting a 2D HDMI signal.

Also understand that, since HDMI is UNCOMPRESSED video, a common 3D HDMI signal such as 1080p24 FramePacking format would be putting out 6.75Gbps! And, unless I'm mistaken (plus I have no knowledge of NeTV specs, etc), 4G communications tops out at 1Gbps. So unless you planned on having 7 or 8 4G connections going in parallel (with some kind of TDM or other way of managing high bandwidth spread spectrum), you won't have to bandwidth to pull this off.

Scott

[aedipuss]

just a curiosity question. do consumer 3d cams record/display blu-ray type 3d or lower quality 1080p side-by-side/over-under type video?

[Cornucopia]

My Fuji W3 (which generates 720p24 video using Dual-mux MJPEG AVIs), from what I can tell, does a realtime high-quality resize and outputs 1080p24 FramePacking 3D to my 3DTV and it looks marvelous (assuming I shot it well enough and there's enough light to avoid noise).

Other cams use a variety of systems: MVC, AVCHD 2.0 (presumeably also MVC), Side-by-side AVC, Dual Files, etc., and different framerates. Most are 1080 (the Fuji is one of the few that is 720, but I think that is because of it not having an updated version recently).

Scott

[boris_blackmilk]

Hello Scott,

Thanks for your reply - of course, I did forget that HDMi is an uncompressed stream, oops...

This is probably more a question for another forum, but the second you might be able to answer - could an 800MHz ARM processor, along with the FPGA in the NeTV, compress an uncompressed HDMi stream? Or, failing that route, does the W3 (which looks a fine camera, and cheap too!) stream over USB in the multi-AVI format? I can imagine, it wouldn't be exactly 720p over USB2.0 but I can live with that, as long as its 3D.

Thanks again!

[Cornucopia]

Hmm...I'll have to get back to you.

Scott

[boris_blackmilk]

Many thanks - I'll await your reply! =)

[Cornucopia]

Had to check manuals, etc...

The Fuji W3 (and probably ALL other 3D cams) does NOT stream anything over USB (whether 3D or 2D). It might allow file transfer that way, but that's it. The ONLY method on this cam that streams live 3D is HDMI (when all the stars are aligned). On other cams, even the HDMI is hit or miss. The pro cams will stream live 3D over 2x SDI ports. That's about it.

After further reading, NeTV won't work - period. It uses chromakey overlay. Since you will be using full-spectrum of color, there is no "safe" key color, plus the overlay ADDS to the bandwidth problem (basically 3-4 HD signals instead of 2). Nothing is compressed with it's system.

You mentioned at the beginning doing this in the "consumer space". However, when dealing with HD, then also with 3D, then also with aerial & remote work, you are clearly NO LONGER in the consumer realm. Those areas require PRO solutions (with pro technology/knowledge & resources/funding).
Sorry,

Scott