raw, BT.656, VLC player

[addi II]

Hi Friends!
Im a newby in video and I have got question, please somybody explain
I have got SDRAM buferized 480x640 50Hz pictures from FPA. How to decode this raw data( maybe in BT.656 standart?) to stream buy UDP with RTSP which can understsnd VLC player?


Thanks in advance!

[jagabo]

docs here: https://www.itu.int/rec/R-REC-BT.656/en

[addi II]

Thanks for replay!

As I understand, - BT.656 defines Stream payload format under UDP/TCP ?
And itsnot depends in which format video comperssed, - M-JPEG, MPEG etc ?
But rest the main question - does VLC player recognize stream under BT.656 ?

Thanks in advance

[pandy]

BT.656 define physical interface and data organization on dedicated video broadcast hardware - like printer port on PC or OSI model layer 1 .
From software perspective this is like YUV422 data format https://en.wikipedia.org/wiki/Chroma_subsampling#4:2:2 so efficiently you have set of data with X,Y and three main components Y, Cb, Cr samples with particular bitdepth (in case ot BT.656 10 bit or rather 8.2 format).
There is few variation on this - packed, planar with more than 8 bith bit depth even more tricky. You should have clear specification from your FPA manufacturer how RAW data are organized. By RAW i mean NO compression - pure digital samples.

As RAW data are frequently headerless then you need be sure how they are organized and how to decode them (where first pixel start and how to offset other components).
VLC should support most if not all "YUV" formats but it need to be guided explicitly where is you X,Y 0,0 and what kind of organization for Y, Cb, Cr is (offsets to 0,0).

[jagabo]

does VLC player recognize stream under BT.656 ?

I've never seen any indication it does. But I don't know for sure.

[addi II]

Thanks for detailed replay!
Yes, I have RAW data - only luma(Y) of 640x480 pixels.
Im going to convert this data to RGB with meaning that R= G= B = Y
After that its neccessaty to convert it to YCbCr
And after that Im going to stream YCbCr to VLC under RTP(UDP)
But Im not sure is it right and does VLC recognize YCbCr
Maybe I must to convert before stream to Motion-JPEG ?

[jagabo]

Certainly VLC knows how to display RGB24, RGB32, YUY2, YV12, and probably YV24 and Y8. But you also need to communicate to VLC what color format you are sending and the frame dimensions and frame rate.

How is the data getting from the camera to the computer? Is a system installed video capture driver involved? If so, you can just open the device via the driver in VLC. If you are using some custom program to read the camera data it's a bit harder. If your video is in a file you can probably use AviSynth with RawSource() to read it, then use any player/editor that supports AviSynth to play it.

Luma only is essentially Y8. So you may not need to do any conversion.

http://fourcc.org/yuv.php

If you have some of your raw video in a file can you upload a sample?

[pandy]

Thanks for detailed replay!
Yes, I have RAW data - only luma(Y) of 640x480 pixels.
Im going to convert this data to RGB with meaning that R= G= B = Y
After that its neccessaty to convert it to YCbCr
And after that Im going to stream YCbCr to VLC under RTP(UDP)
But Im not sure is it right and does VLC recognize YCbCr
Maybe I must to convert before stream to Motion-JPEG ?

No, send only Y8 format as jagabo already wrote, to avoid issues encapsulate your RAW data within some container, even crude YUV4MPEG2 https://wiki.multimedia.cx/index.php?title=YUV4MPEG2 or piped_pbm https://en.wikipedia.org/wiki/Netpbm_format - ffmpeg sources may help to create required code.

[addi II]

Certainly VLC knows how to display RGB24, RGB32, YUY2, YV12, and probably YV24 and Y8. But you also need to communicate to VLC what color format you are sending and the frame dimensions and frame rate.

How is the data getting from the camera to the computer? Is a system installed video capture driver involved? If so, you can just open the device via the driver in VLC. If you are using some custom program to read the camera data it's a bit harder. If your video is in a file you can probably use AviSynth with RawSource() to read it, then use any player/editor that supports AviSynth to play it.

Luma only is essentially Y8. So you may not need to do any conversion.

http://fourcc.org/yuv.php

If you have some of your raw video in a file can you upload a sample?

Thank you for ypur support!

I have FPA and ditialized data from it - 14 bit data of pixels Luma
I read it row by row, where row is 480 and column is 640.

Thus Ive got 307200 14-bit pixels data of Luma
I planned to do necessary conversion to some sdandart format to be able to recognize by VLC plaer without any driver just within UDP connection for strem video
To do that Ive got custom board with W5300 chip on it

unfortunally I have not got raw data directly
Now I just store it to SDRAM which is standed on board to thuther conversion

Also I thoght that to stream data though RTP I must to conver my pixels data to MPEG TS, but now it is not cleat yet

[addi II]

P.S. images rate is 10 per sec

[jagabo]

I don't know anything about RTP. Maybe you've seen these:

RTP Protocol, RFC 3550: https://www.ietf.org/rfc/rfc3550.txt
C implementation: http://www.linphone.org/technical-corner/ortp/overview
Wikipedia: https://en.wikipedia.org/wiki/Real-time_Transport_Protocol

[pandy]

Not sure - how good you are with SW (i mean are you doing regular programming or rather electronics and software is not best part of your skills).

If you feel comfortable with software then i would suggest to keep 14 bit data, convert them to one of those two ffmpeg supported formats

IO... gray16be               1            16
IO... gray16le               1            16

or

IO... gray14be               1            14
IO... gray14le               1            14

They are 16 or 14 bit gray - this will increase required bandwidth twice but allow you to keep high dynamic data for further processing.
Bitrate required to transfer 640x480 Y16 10 fps is around 50 Mbps, this fit in Fast Ethernet 100Mbps W5300 capabilities.
To encapsulate RAW video you may think about other than MPEG-TS container (AVI or better MKV should be easy to use).
You may think about implementing lossless video codec, APNG, FFV1, seem to support gray16 and provide lossless compression so may solve some of your issues (compression is optional). Recommend to check ffmpeg source what you can use without too much reinventing wheel.

[addi II]

Yes I dont have enouth skills to develop top level SW to decode my RAW data throuth Ethernet of PC.
Cause I devided to use ready player wich can stream under UDP(RTP) and prepare neccessary data on bottom layer on FPGA to do that.
About ffmpeg - I didnt understand - is it player which can decode Y16 and get it under stream through UDP(RTP)?
I download that but cant launch.

[pandy]

Yes I dont have enouth skills to develop top level SW to decode my RAW data throuth Ethernet of PC.
Cause I devided to use ready player wich can stream under UDP(RTP) and prepare neccessary data on bottom layer on FPGA to do that.
About ffmpeg - I didnt understand - is it player which can decode Y16 and get it under stream through UDP(RTP)?
I download that but cant launch.

ffmpeg can accept Y16 at input also over network - issue is how to stream data from your HW to network interface - you need to read RAW data, encapsulate
them inside some simple container and use some tool (lib? perhaps some code for W5300) to output data over network.

I advised ffmpeg as sources are available thus you can check how encapsulate data...

If you are not SW guy then i see lot problems in front of you... perhaps someone is able to help you with software?

[addi II]

Thanks for this decision
Maybe you know - does ffmpeg supported by Qt?
I know that VLC player does, and cause of that I decided to orientate on VLC

[pandy]

Thanks for this decision
Maybe you know - does ffmpeg supported by Qt?
I know that VLC player does, and cause of that I decided to orientate on VLC

IMHO VLC share some core (libav) with ffmpeg thus they are largely overlapping. Perhaps you can try to use gstreamer as your framework? Seem qt is accepted by gstreamer https://gstreamer.freedesktop.org/bindings/qt.html .

[addi II]

Thanks, but building ffmpeg its more difficult descision
Still looking to the VLC player side
I will try to convert Y16 to Motion JPEG and trying with SDP file stream it under RTP