Programming a GUI to Display YUV data in C#

[gfxcat]

I'm trying to write an app in C# to display YUV streams/frames.

My usual input is raw YUV 420 format (IMC4).

Usually I need to convert this into other rarely used YUV formats like NV12, P208, YU12, etc.

I'd like to be able to compare 2 YUV frames and display any pixel mismatches, etc.

I know how to handle the YUV data at the byte level but want to be able to provide a GUI interface for all this.

Where do I start? What do I need to do to display YUV? Do I need to convert them to BMP's to display them?

I've not worked with DirectShow, is this what I'd need to use?

I'm currently developing in C#.

My main wants are:
- Display YUV frames/streams w/o converting to BMP
- Some methods to do good 4:2:2 -> 4:2:0 conversions or vice-versa.
(I'd be able to do this only via averaging or duplicating but I'm sure there are Graphics libraries with better methods to interpolate the pixel values when upscaling, etc)

Thanks for any help.

[jagabo]

You need to convert them to RGB bitmaps to display them on the desktop. If you use Video overlay (via Directshow) you can leave them in YUV format but overlay usually accepts a limited number of layouts, like YUY2, UYVY, etc.

[JohnnyMalaria]

Since you want to compare YUV frames, why not display each of the three components as separate grey scale bitmaps? It will be much more appropriate than converting to RGB.

[jagabo]

Since you want to compare YUV frames, why not display each of the three components as separate grey scale bitmaps?

That's a good idea -- but sometimes there's no substitute for seeing in color.

[JohnnyMalaria]

Perhaps both would be best?

The problem with RGB is that you can't tell whether the error is in the Y, U or V. The subsampling of the chroma components necessitates interpolation or duplication in the RGB domain. But perhaps only the resultant RGB is important.

[gfxcat]

The problem with RGB is that you can't tell whether the error is in the Y, U or V. The subsampling of the chroma components necessitates interpolation or duplication in the RGB domain. But perhaps only the resultant RGB is important.

Right, I don't want to lose any YUV data which would be bound to happen when converting to RGB.

The purpose is to compare output from two different methods that should yield the same YUV binary data from a given raw YUV input.

Which is why I'd like to be able to display the YUV frame with as little conversion as possible.

I understand if I need to convert from one YUV format to another but I'd imagine Directshow has a trusted algorithm for this as opposed to me just averaging neighboring pixels or throwing out bits (depending on what YUV format I'm converting to or from)

So DirectShowNet library would allow me to do this?

Thanks.

[vhelp]

Hi gfxcat,

A few years ago I wrote some code to process RGB images into raw YUV images and which
were easily decoded back to RGB via AVIsynth scripts. However, its been a while since I
last dabled in that tool's development. But, here are my memories on all this, etc..

One possible start or pointer in this endeavor of comparing Frame A vs. Frame B can be made
by taking the color channels Y or U or V and comparin them separately. To do this you would
have to know that format structure first -- how it is laid out from inside the stream and bring
it into a manageable matrix of your own but still representative of that format.

So, for example, if you have a UYVY or YV12 raw image source, then you know you have the following:

Example 1:

color space: YUV
color sampling: 4:2:2
container format: UYVY
layout type: packed -- Ex: UYVY UYVY UYVY UYVY ..


Once weaved into a stream would look like this in a finished file:

file: UYVYUYVYUYVYUYVY

And, to compare the contents, would go something (crudely) like this:

IF the matrix of fA = [UYVY][UYVY][UYVY][UYVY][UYVY] ..
and the matrix of fB = [UYVY][UYVY][UYVY][UYVY][UYVY] ..
THEN compare fA -to- fB, byte by byte or color channel -to- color channel as in
if fA.U=fB.U then do-your-stuff, etc. etc.
if fA.Y=fB.Y then do-your-stuff, etc. etc.
if fA.V=fB.V then do-your-stuff, etc. etc.


Example 2:

color space: YUV
color sampling: 4:2:0
container format: YV12
layout type: planar -- Ex: see below

YYYY YYYY YYYY YYYY
UU UU UU UU
VV VV VV VV

Once weaved into a stream would look like this in a finished file:

file: YYYYYYYYYYYYYYYYUUUUUUUUVVVVVVVV

And, to compar the contents, would go something (crudely) like this:

if (YYYYYYYYYYYYYYYY) = (YYYYYYYYYYYYYYYY) then do-your-stuff
if (UUUUUUUU) = (UUUUUUUU) then do-your-stuff
if (VVVVVVVV) = (VVVVVVVV) then do-your-stuff

But, you should get the gist of all this once you've had some time to digest it all and
begin to put things into perspective and so on..

When comparing the color channels (ie, R/G/B 's or Y/U/V 's) you might have to decide on
how you evaluate the results as being Equal or Close-to-Equal or within Threshold Range
as being Equal, etc. Its easy to say if white=white then it is true. But its another when
you are dealing with millions of colors in a single image, especially when the channels are
all merged (multiplied as in [R*G*B] ) together to become the actual image of which
represents the actual detail or art on the canvas, much like to faces being compared.
They both might have the same features and size noses, but their shades/hues/contrast
etc. will be different or way off, and so on.

-vhelp 4559

[vhelp]

Regarding the points made by the last few posters, I'd like to add..

The problem with measuring from YUV images and then in the RGB domain for comparitable
differences is that when you compare:

RGB -- remember, that there are errors during the YUV->RGB conversion and also that this has
the tendency to 'induce' additional image realities. So, imho, it is not a good idea to meaure
(during processing) an image if it came from YUV color space and you want to compare it but
while inside RGB color space.

In the posters case, he wants to measure the actual (raw) YUV images. So, the best way to
do this is to measure them by components, Y; U; and V; domain.

Also, remember that in YUV color space, the Y channel carries the greatest or complete original
image detail, while the U and V components .half or .quarter from the originals detail because
none of the Y's are being sub-sampled (eliminated) nor averaged like the U and V are. But
all this prob doesn't matter because after all you are measureing from a raw source and not
from a raw_source->new_source as in YUV->RGB conversion. So his comparison will be most
accurate.

-vhelp 4560

[jagabo]

I understand if I need to convert from one YUV format to another but I'd imagine Directshow has a trusted algorithm for this as opposed to me just averaging neighboring pixels or throwing out bits (depending on what YUV format I'm converting to or from)

So DirectShowNet library would allow me to do this?

I belive Directshow does all colorspace conversions using installed filters. You'll be at the mercy of whatever colorspace converters you have installed. Conversion of say YV12 to YUY2 will do different things depending on what converter is being used. One may interpolate the U and V channels. Another may duplicate. And there's good reasons for each method. If you convert YV12 and YUY2 back and forth several times you want to duplicate rather than interpolate to avoid generational losses. If your converting for display you may want to interpolate to get smoother color trasitions.

You'll find the following site useful:

http://www.fourcc.org/