Quantize Matrix

[patrickm]

sometimes i wonder if maybe we're missing something here. people here have put a lot of effort into producing the best quality encodes in a number of ways, but how come no one ever touches the actual encoding algorithm? i'm a mechanical engineer, but i also have a background with some electrical engineering in it, and i do know a few things about JPEG/MPEG encoding. but my question is to the real digital imaging pro's out there- can the 'natural image' matrix TMPG uses be shown to be pretty close to optimal, or is it just something that no one bothers to check out? i'm assuming that you could improve quality a bit, but it would have to be done on a movie-by-movie (or at the extreme a scene-by-scene) level.

so i'm looking for a good way to go about this. i was thinking of taking several still shots from a scene or movie clip and JPG-ing them with an image editor using different matrices, then comparing the differences (i don't have the time to encode scenes 2 dozen times and subtract their differences). has anyone done this, or at least know of an imaging program that lets you rewrite the matrix easily?

i have written some Matlab scripts to compare still images taken from MPEG/AVI frames, so the compression is really my problem here. i may just end up doing many small encodes, then comparing I, B, and P frames, and also with different encoders, but i was looking for some help as i start. any suggestions would be appreciated...

[tacosalad]

TMPGEnc lets you edit the matrix directly, and to name your own matrices.

There are several subtle tradeoffs of a customized quantization matrix. For example, reducing the values may leave too many occupied coefficients after quantization, so that the encoder simply raises the quantization scale to compensate, or starts discarding coefficients.

Nevertheless, there are probably useful traits that can be exploited, especially with a television signal. For example, vertically arranged pixels are discrete (from separate scan lines) and hence might exhibit some useful trait in terms of energy distribution. Are high-frequency harmonics likely to result? Can they be discarded, or should they be emphasized? A histogram might help decide.

If you're going to mess with quantization matrices, I suggest looking for a reference that illustrates the basis patterns as vectors or gives their discrete values, such as Sayood, Bhaskaran, or Konstantinides. Most online sources illustrate the basis patterns visually as grayscales, which isn't helpful for this type of analysis since gray has no particular sign or value.

[patrickm]

oh, actually what i meant was that what i do know is about spatial frequency response, fourier analysis, etc, but what i didn't know was that if this (or very similar) matrix was created through years of trial and error through JPEG image processing, or basically just someone's best guess. i know TMPG lets you edit the matrix directly, but i was looking for a still image editor too so i could process groups of representative stills via matlab for histograms and comparisons (it's not as easy to compare encoded video streams)

but of particular interest was disregarding some of the high-frequency noise, since i have an ATI card and they seem quite susceptible to this, as you can see from the captures the macroblocks switche back and forth from predominantly vertical to horizontal very rapidly, and i was hoping by reducing either horiz, vert, or possibly both high spatial frequency components i could arrive at a more stable pattern temporally. i think this might also be useful for VHS captures, where even most high horizontal frequencies are probably not very precise.

but thanks a lot for the references, i'll try and check them out.

[tacosalad]

It sounds sensible to study JPEG if possible, but I didn't know of any that would do that.

The apparent switching between horizontal and vertical high frequencies might be an artifact of whether a particular image is a B or P frame. If you load the mpeg into VirtualDub, it'll tell whether each frame is B, P, or I. This might be a worthwhile test, just to diagnose the source of those particular artifacts.

I've messed with the quantization matrix, and I agree that it's not easy to detect what impact it has on motion video. Lacking good analysis tools, I set it aside as too awkward.

I don't know the origin of all the matrices, but I'd wager that TMPGEnc's "CG/Animation" matrix is a simple, off-the-cuff attempt to cope with the unpredictable distribution of cartoon type images.