JPEG and MPEG-1 I-frame; exact difference?

[Emiel Schiphorst]

On a high level, I know the steps towards MPEG-1 compressed video (also DCT, quantization, RLE etc.). I read I-frames undergo 'JPEG-like' compression.

Questions I would like to be answered:

1. Could it then be possible to construct an I-frame straight from a JPEG? I suppose the EXIF info is not in the I-frame bytes. And of course the quantization values and compression quality should be the same.

2. Does anyone know the exact difference between a jpeg and an I-frame only?

3. Does anyone know a tool/way to EXTRACT (NOT to SET) the used quantization values from an MPEG-1 stream?

I would like to make Sony MPEG EX, a proprietary but compliant MPEG-1. I searched, but could not find a video-editing tool that can take into account it's specifics.

I studied it's file structure and did not take into account the layer-2 audio yet. That will follow later. For the video I found 3 IPP GOP's per physical block. At 320x240 the size of this block is 32Kb, at 160x112 this is 8Kb.
(This physical block size takes into account the formatting of the Sony Memory Stick.)

The 2 P-frames are in fact both dummy (empty) frames (really!), only referencing the I-frame, so no changes at all! This is NOT about MJEG or Motion-JPEG! I was able to manually isolate I-frames, replace them, fix pointers and padding and even display the altered MPEG EX on Sony hardware (PHD-A55 Cyberframe or digital camera) to prove my findings!

Thanks in advance for reading!

[FOO]

How did you study the file structure ?
Do you have ISO 13818-2 ?
I think the quatization matrix is in the MPEG file itself

Edit: I mean ISO 11172-2

[Emiel Schiphorst]

Good questions! I'm glad someone is responding. I feel like a calling in the desert ...

I was just looking, comparing and manipulating bits.
No, serious (one level up): with a HEX-editor.
It is hard (took some months), but it can be done.
For one element, I even had to go to the bitvalue.

The 6 sources were very small MPEG EX's (3-4 seconds), 4 without sound, 2 with sound. The ones with sound were taken with a DSC-S85, the ones without with a U20. Off the 4 without sound there were 2 with size 320x240 and 2 with size 160x112.

Comparing between these outputs learn a lot. Also using a simple tool like VideoMach showed no difference between first, second and third frame. I already guessed the quantization matrix should be in the MPEG file itself. I'm just curious about its values (in relation to the input JPEG's). They must be somewhere in the header (before the first I-frame bytes). However, this part is constant bewteen files.

I could isolate an MPEG EX I-frame, replace it with another, adjust the padding and fix a pointer and display on Sony hardware.

Some years ago someone was busy with the same:
http://www.mav-magazine.com/Aug1999/MPEG/
I'm much further (found pointers, could replace I-frames) but I'm still not there.

I know there is ISO, but I don't have it (or need it uptil now). These MPEG EX files are Sony proprietary, but compliant MPEG-1. I need to stick to exact byte locations for pointers and padding.

[FOO]

Try VOBedit. if it works on MPEG1 it will help lots

[SaSi]

One of the original questions was whether JPEG and MPEG-1 I-frames are identical. They are not. The "technology" behind creating a JPEG picture and MPEG-1 I-Frame may be similar but the encoding of the two is quite different.

I have briefly studied the MPEG-1 video standard and only glanced at the JPEG one, so I can't really explain the differences - only suggest that there are quite a few differences in the structure of a JPEG file and the construct of an MPEG-1 I-Frame.

The Quantization matrix in the MPEG-1 stream is found in the stream (of course) unless the default MPEG values are used. I have not read the MPEG-1 video specs (only MPEG-2) and based on the document compatibility information I am (guessing?) that the quantization matrices are to be found in the Sequence Header - as two 64 byte blocks after byte 12. If the matrices are built in, then the Sequence header is longer than 11 bytes. Bits 1 and 0 of byte 11 are set if each matrix follows. There are also some marker bits to take out from there.

Your project looks interesting, however I doubt you can go much further without reference to the 11172-2 document (-2 also for audio and -1 if you want to decode a system stream).

[FOO]

Well I know for a fact that they both do a Discrete cosine transform
to start with. Have no idea about quantization or entropy coding.

[Emiel Schiphorst]

FOO,

Thanks for the addition! I also know both use quantization matrices.
I have to look up the document(s) describing JPEG (and Exif). Later I will give the URL's.

SaSi,

I guess I was incomplete in my information. You may not have read all my posts. But the following might interest you.

In forum Editing, cutting and joining, post 'Make MPEG EX; use which frameserve, script or programming?' I wrote:

'But I first need to know exactly what the relation is between Sony captured JPEG and an MPEG-1 I-frame.

There is probably a possibility to analyze this relation. Sony says they can produce JPEG's with 5 seconds of audio ... That is impossible, I know, but ... What results when you use this feature is a JPEG and a still MPEG EX MPEG-1 (with sound). The still MPEG EX is much smaller when displayed than the JPEG. Due to the Memory Stick architecture, a Sony display can show the JPEG and let hear the layer-2 audio from the MPEG-1 in parallel. Nice! Now, the big question is: is the MPEG-1 I-frame based on the JPEG bytes (a recompression) or is it done before that?'

When I have time next weekend, I will compare both MPEG (I know where to look in the bytes) and JPEG (I have a marker description) and look for similarities in the process as well as in the data. Perhaps a dumb question, but: where can I find this 11172-2 document? Nice to have, but never mind, I can conclude a lot by only looking at the binary/hex data. In the meanwhile I have become experienced in that.

Let's see!

[Emiel Schiphorst]

SaSi and FOO,

This is to keep you posted on my research progress. You may, or may not already know certain details ... And let me know if you are interested in this information.

I found the sequence header in the MPEG EX and the load_intra_quantizer_matrix equals 0, meaning NO QM! So the default QM specified by the MPEG standard is used (know that QM).

I was able to find and manually extract the QM values from a JPEG. I just wonder about their ordering. It it always standard zig-zag for JPEG or could it also be something different (lineair).

Also, I found 2 possible additional applications:
- Improve Sony proprietary MPEG-1's by making real, instead of dummy
P-frames (the camera and/or card are the bottlenecks)
(if the camera processor can't handle the bitrate, perhaps afterwards?)
- Taking edited movie with you on your camera(-card) and show on TV:
no need to show by computer or other peripheral, or buy extra
equipment/cables

I haven't come yet to the comparison based on the 'JPEG with audio'.

As a summary, the most significant information (for me ... at this point in time):

Subject A: The difference between MPEG and JPEG encoding
An important source: http://wwwbrauer.in.tum.de/seminare/web/WS0001/vortrag07/proseminar.pdf
Generally MPEG uses the same coding techniques as JPEG.
However MPEG features:
Additional: block based motion compensated prediction
(irrelevant for me: dummy P-frames)
Only 1 color space (4:2:0, YCbCr) (conversion necessary/possible?)
Only 1 sample precision (8bits) (conversion necessary/possible?)
Only 1 scanning mode (sequential) (conversion necessary/possible?)

Subject B: Entropy Encoding
The JPEG algorithm outputs the DCT block’s elements using an entropy
encoding mechanism that combines the principles of RLE and Huffman encoding. Output of the entropy encoder: a sequence of 3 tokens, repeated until the block is complete. The three tokens are:
1. run length (= number of consecutive zeros that precede the current
non-zero element in the DCT output matrix)
2. bit count (= number of bits used to encode the amplitude value that follows, as determined by the Huffman encoding scheme)
3. amplitude (= of the DCT coefficient)

I will keep researching/bridging (try to convert if necessary/possible) these differences. Theory is nice, but first I need to understand and then
look at bytes and bits for confirmation.

By the way: I also remember having read/seen the first block in MPEG being monochrome and some kind of average of the whole I-frame ... (everything in the frame is related to that block).

[FOO]

Keep up the good work.
You are obviosly a serious maniac.
I am studying this for a different reason from yours.
I want to know (and fix) the problems with editing mpeg2 streams
which is 1 level above the actual picture encoding.

[Roderz]

*** BANG ***

MY HEAD JUST EXPLODED -- after reading all that!

Good luck.

[SaSi]

Only 1 color space (4:2:0, YCbCr) (conversion necessary/possible?)
Only 1 sample precision (8bits) (conversion necessary/possible?)
Only 1 scanning mode (sequential) (conversion necessary/possible?)

MPEG-2 allows - in addition to 4:2:0 - 4:2:2 and 4:4:4 color spaces.
Sample precision can be 8,9,10 or 11 bits, although MP@ML is only allowing 8,9 and 10.
Scanning mode (if you mean how the macroblock is converted to a series), is zig-zag and alternative. Zig-Zag is fine for progressive frames while the "alternative" - they couldn't find a better name for this seamingly "random" scanning order - is better for interlaced.

[Emiel Schiphorst]

Sasi,

I just found some very interesting 'old' information, of which I already had some sense:

https://www.videohelp.com/forum/viewtopic.php?p=660264#660264