MPEG I-frames to JPEG

[bluethunder888]

Hello,

I'm currently working on a project where I need to extract the I-frames of an mpeg file to jpeg that'll undergo some operations then reconstruct the original video file(with the new I-frames).

The approach I took was to parse the video file and whenever I find a picture header that indicates an I-frame I store the macroblocks of this frame till the end of its slices. so I end up with a byte array containing the bitstream of those macroblocks(huffman encoded DCT coefficients?).

What I need to know is how can I from this byte array construct a jpeg file of this frame? What I know is that an I-frame is almost identical to the jpeg so I'm guessing what I lack here is markers and headers of the jpeg, am I correct?(any clear document about a typical jpeg structure?)

Please, I really need help with this as it is crucial to the system i'm working on and any help would be highly appreciated

P.S. : if its of any importance I'm using C# in my implementation

Thanks in advance

[jman98]

Your post is rather advanced and off the top of my head I can only think of a handful of members here who might be able to do something this complex. I am not one of them. If you don't get help here you might try posting to the forums at doom9.org as they have some really strong technical people there who might find your problem interesting and want to help.

[poisondeathray]

1) why jpeg ? jpeg is lossy, why not something lossless like png or tif ?

2) what do you plan to do with the other P, B frames if this is long GOP MPEG2 ? You cannot reconstruct the video from I-frames only. Or is this part of your project and you intend to modify them based on the new I-frame ?

[bluethunder888]

Your post is rather advanced and off the top of my head I can only think of a handful of members here who might be able to do something this complex. I am not one of them. If you don't get help here you might try posting to the forums at doom9.org as they have some really strong technical people there who might find your problem interesting and want to help.

Thanks man

1) why jpeg ? jpeg is lossy, why not something lossless like png or tif ?

2) what do you plan to do with the other P, B frames if this is long GOP MPEG2 ? You cannot reconstruct the video from I-frames only. Or is this part of your project and you intend to modify them based on the new I-frame ?

thanks for replying,
1) jpeg because it is similar to the structure of the I-frame so I think this will make it easier for me to get back the modified macroblocks after being done with the processing
2) P,B frames will go in the reconstructed file unmodified (just copy them from the original stream)

To give you more insight about what I'm doing so that it might make it easier for you to provide help. Basically, what I need to do with the I-frames is embedding of binary data inside its bitstream then reconstructing the original file...the embedding scheme is based on i-frames only. so here came the thought of extracting them to jpeg's and embed the data inside the DCT coefficients. Then reconstruct the macroblocks with the new coefficients and finally put them back to the mpeg file

Appreciate your help

[jman98]

To give you more insight about what I'm doing so that it might make it easier for you to provide help. Basically, what I need to do with the I-frames is embedding of binary data inside its bitstream then reconstructing the original file...the embedding scheme is based on i-frames only. so here came the thought of extracting them to jpeg's and embed the data inside the DCT coefficients. Then reconstruct the macroblocks with the new coefficients and finally put them back to the mpeg file

Appreciate your help

I am troubled by your explanation for wanting to do this because off the top of my head I can think of some very evil reasons why someone might want to embed binary data in the bitstream and there is no way I'd help such people.

[bluethunder888]

hahahaha...your reaction to me is rather amusing because I've encountered it many times

I intend no evil in doing this...this is for my graduation project which is a complete multimedia steganographic system...nothing more and nothing less...I swear it!

I guess I should've explained my purpose because it does look suspicious at the first glance...please accept my apologies...

[hank315]

IMHO, using the huffman encoded MPEG2 MacroBlocks in JPEG is not possible, AFAIK JPEG always uses a zigzag scan while MPEG2 often uses the alternate scan method so the run length encoding is incompatible.
But why the difficult way, if you want to embed extra data, you can do that on the fly, it's called padding...

[Cornucopia]

These 2 old threads might help you a little:

https://forum.videohelp.com/threads/125628-An-efficient-JPEG-to-MPEG-1-transcoding-algo...979#post568979

and

https://forum.videohelp.com/threads/123139-JPEG-and-MPEG-1-I-frame-exact-difference

Scott

[El Heggunte]

this is for my graduation project which is a complete multimedia steganographic system...

and

P.S. : if its of any importance I'm using C# in my implementation

make me think the school you're frequenting does not want you to be a seriously-skilled professional.

[bluethunder888]

IMHO, using the huffman encoded MPEG2 MacroBlocks in JPEG is not possible, AFAIK JPEG always uses a zigzag scan while MPEG2 often uses the alternate scan method so the run length encoding is incompatible.
But why the difficult way, if you want to embed extra data, you can do that on the fly, it's called padding...

Yes,I just came across the different run length encoding between the two which will make it even more difficult. Padding however is not feasible because the algorithm has to retain the file size constant....thanks for your reply

These 2 old threads might help you a little:

https://forum.videohelp.com/threads/125628-An-efficient-JPEG-to-MPEG-1-transcoding-algo...979#post568979

and

https://forum.videohelp.com/threads/123139-JPEG-and-MPEG-1-I-frame-exact-difference

Scott

Thanks for the links I'll check'em out right away

this is for my graduation project which is a complete multimedia steganographic system...

and

P.S. : if its of any importance I'm using C# in my implementation

make me think the school you're frequenting does not want you to be a seriously-skilled professional.

haha, ahh well....

[bluethunder888]

It seems like the whole jpeg thing is not applicable...However, is there a way to decode the macroblock to its original YCbCr structure?

[hank315]

Yes, do an inverse quantization and an inverse DCT, this will give you the 16x16 luminance and chrominance planes.
You can look in mpeg2dec how to implement it.

How many bits do you want to insert per I-frame?

[bluethunder888]

thanks for your reply. Well, the amount of bits per I-frame will vary because the embedding should be bases on PVD (Pixel value differencing) which will mainly limit the embedding to the edges area in the image.

EDIT : does the number of bits/I-frame will affect the process? I mean will a specific number of bits dictate another (easier scheme)...I can really trade off PVD for easier and quicker implementation right now (deadline approaching)

[hank315]

Well, the amount of bits per I-frame will vary because the embedding should be bases on PVD (Pixel value differencing) which will mainly limit the embedding to the edges area in the image.

I don't understand, I thought you can manipulate every pixel in the image but I'm not an steganography expert.

You have to realize that you have to first decode the MB, then apply your PVD stuff and then encode it again. Apart from the fact the whole process is lossy, you will not get the original file size.
If you want this to work you have to apply the PVD on the AC/DCT coeffs itself. The AC coef might be a candidate both for luminance and chrominance.
Just have a look how mpeg2dec decodes the AC coeff.