Codec Explanation, which is best for this scenario ...

[andrewbutkus]

Hi,

I would like to use a codec which utilizes the previous 10 frames for its current frame (ie it uses the previous 10 frames to create its current frame - and by doing this, reducing the overall size of the outputted movie).

So rather than frame 11 being say 40kb it is at least 50% smaller as it uses the previous 10 frames to create the remaining information.

Can anyone point me in the right direction?

[AlanHK]

Every video codec does that, except "uncompressed". Even lossless ones.

[andrewbutkus]

I thought that was the case until i extracted each individual frame into a jpg, and the quality loss was incredibly high (compared to the original image i inputted), appearing that all the video codec did was compress each frame to be more lossy to reduce its size, and not take into account previous frames.

[jagabo]

There are many intra-frame only video codecs (MJPEG, DV, HuffYUV, etc). They compress each and every frame without any reference to others.

Most inter-frame codecs only refer to one or two nearby frames (MPEG 2 for example). h.264 can refer to as many as 16 different frames (backward and forward). Try x264 with high reference frame settings. Be aware that many players (Blu-ray players, PS3, for example) cannot handle 16 reference frames.

[AlanHK]

I thought that was the case until i extracted each individual frame into a jpg, and the quality loss was incredibly high (compared to the original image i inputted), appearing that all the video codec did was compress each frame to be more lossy to reduce its size, and not take into account previous frames.

When you "extracted" each frame, you decompressed the video stream, not just that one one frame. And that uses the information in the "GOP" = group of pictures.

Also, JPEG is itself a lossy format. You should use a lossless format like PNG or BMP to see what is really there.

Google or look up Wikipedia for a full technical discussion. E.g.:

http://en.wikipedia.org/wiki/Video_compression
Video data contains spatial and temporal redundancy. Similarities can thus be encoded by merely registering differences within a frame (spatial), and/or between frames (temporal). Spatial encoding is performed by taking advantage of the fact that the human eye is unable to distinguish small differences in color as easily as it can perceive changes in brightness, so that very similar areas of color can be "averaged out" in a similar way to jpeg images (JPEG image compression FAQ, part 1/2). With temporal compression only the changes from one frame to the next are encoded as often a large number of the pixels will be the same on a series of frames.

[AlanHK]

There are many intra-frame only video codecs (MJPEG, DV, HuffYUV, etc). They compress each and every frame without any reference to others.

Most inter-frame codecs only refer to one or two nearby frames (MPEG 2 for example). h.264 can refer to as many as 16 different frames (backward and forward). Try x.264 with high reference frame settings. Be aware that many players (Blu-ray players, PS3, for example) cannot handle 16 reference frames.

Okay, I stand corrected.
Though I think the "intra-frame" codecs are used much less than those that compress temporally as well.

[andrewbutkus]

Hey,

thanks alot for the quick replies, - i thought about using x.264, but it seems all the opensource implementations (ffmpeg, mencoder) are really buggy, ie, they crash an awful lot when using this codec with fairly standard command line arguments, are there any cheap commercial alternatives?

Andy

[jagabo]

I don't think I've never seen x264 crash. And, in my experience, when an encoding task crashes it's usually because of the source decoder screwing up, not the encoding codec.

[andrewbutkus]

Ok, ill try and get a copy of the latest stable trunk and compile it myself to make sure its not doing something weird with the configuration options to whomever created this binary.

Thanks =) i go home happy tonight.

Andy

[lordsmurf]

A little less dogma is in order here...

JPEG is not necessarily "lossy" -- it depends entirely on the settings.
Both the original JPEG and JPEG 2000 have lossless options.

MPEG-2 also does not have to use temporal compression -- I-frame only is intra-frame only.

I've seen x264 crash. In my experience, the GUIs tend to be the ones at fault.

[poisondeathray]

I've seen x264 crash too , but it's usually from something else, like a system , decoder issue or GUI issue as mentioned above

In the rare cases that is that particular build of x264's fault , it's usually fixed within hours .

There is a very active team of developers and testers , and new builds are pushed out almost daily, so things can get temporarily broken. No paid product has as good support or testing and feedback as x264 . Try even getting return emails from retail products, let alone working solutions in such a short time frame

[andrewbutkus]

tried x264, it seems to crash whenever i try and use just the default parameters and pass in a video file which plays fine under windows media player, not sure what else to say - it doesnt output anything useful when it crashes, and all the warning messages in the console window seem ok to me =( - anyone got a revision number for mencoder which is known to be ok?

[poisondeathray]

can you post the log file or any error messages

what kind of video? use mediainfo

I don't use mencoder, so someone else will have to help you with that part

[andrewbutkus]

C:\clouds>"C:\Documents and Settings\butkusaj\Desktop\jpeg saver\Bin\mencoder.exe" "mf://*.jpg" -ovc x264 -x264encopts bitrate=1000 -o movie.avi

MEncoder dev-SVN-r27419-4.2.2 (C) 2000-2008 MPlayer Team
CPU: Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz (Family: 6, Model: 15, Step
ping: 2)
CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
Compiled with runtime CPU detection.
success: format: 16 data: 0x0 - 0x0
MF file format detected.
[mf] search expr: *.jpg
[mf] number of files: 1 (4)
[demux_mf] file type was not set! trying 'type=jpg'...
VIDEO: [IJPG] 0x0 24bpp 25.000 fps 0.0 kbps ( 0.0 kbyte/s)
[V] filefmt:16 fourcc:0x47504A49 size:0x0 fps:25.000 ftime:=0.0400
[ass] auto-open
Warning: -ass implies -keep-pts, which may cause "badly interleaved" files.
[ass] Init
================================================== ========================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffmjpeg] vfm: ffmpeg (FFmpeg MJPEG decoder)
================================================== ========================
VDec: vo config request - 1024 x 768 (preferred colorspace: Planar YV12)
VDec: using Planar YV12 as output csp (no 3)
Movie-Aspect is 1.33:1 - prescaling to correct movie aspect.
Opening video filter: [scale]
[swscaler @ 00DF5724]using unscaled yuv420p -> yuv420p special converter
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64


And then just crashes, asks if i want to debug it or not, choose yes, and it gives me a dissasembly with lots of jmp/jne instructions

[poisondeathray]

So you're not using x264.exe or a GUI for it , you 're using it through mencoder.exe ?

You're encoding some jpeg images ?

What OS?

The way I would do it is string them together in avisynth, then use x264 to encode . I'm not familiar with mencoder syntax

[andrewbutkus]

Hey,

i tried downloading mencoder from a reputable source, it seems i didnt have it compiled correctly/optimised for my processor - seems to be doing what it should now without crashing,

results seem not too bad at this stage,

thanks for the help

Andy

[andrewbutkus]

Ok a new question =D

I see that x264 uses bframes so it searches for similar frames forwards and backwards. I just want to search for similar frames backwards, is there an option to do this, or is the x264 algorithm hard coded to search in both directions?

Andy

[jagabo]

B frames are Bi-directional by definition. You could disable B frames and only use I and P (forward Predicted) frames. You won't get as much compression.

[poisondeathray]

If you provide more information on the content you have (it looks like the source is jpeg image sequence) , or provide a sample, and clarify what your goals are, people can give you better suggestions on settings or how to approach this

[andrewbutkus]

I have movie1 and movie2
each contains the same sequence of jpg images except movie 2 has 1 extra,

I want movie 1 stream and movie 2 stream to be as much the same as possible, except for the additional 1 frame at the end.

(so i would expect something like this:


|-movie 1 - image sequence-----------------------------|
|-movie 2 - image sequence-----------------------------|---single image at end---|

Im wanting that single image at end to pick up some delta's from the image sequence so its size on disk is less than a standard jpeg when extracted away from the movie2 (as it will hopefully use p frames to delta against).

I cant use bframes, as it changes the avi stream so movie1 and movie2 image sequences dont match in their encoded form - this i believe is due to the bframes using the single image at end as a reference frame changing the content drastically in movie 2 (basically changing the stream i dont want changed and using the single image at end as a reference point to compress the overall stream).

Ideally i just want the single image at end to do its delta against previous frames, and that way maintaining the integrity of the pre stream so movie1 and movie2 nearly match.

This may seem like a strange goal, but it has real purpose.

[poisondeathray]

Still quite cryptic...

What is the "real purpose?"

Why can't you change the frametype distribution? Why does it have to stay the same?

You can specify exact frametypes and %quantizer (e.g. you might want 1.2x on a specific frame) per frame with a qpfile , but overriding what x264 analyses for frametype placement and bitrate distribution usually leads to worse quality

Does the appended single image last 1 frame duration , or is it repeated?

How are the images related ? (are they part of 1 related scene or completely random images?) , this matters becase the larger the difference the higher the probablilty an I-frame is placed (or even IDR frame)

If that single appended image is unrelated to previous frames, you almost have to place an I-frame (not enough content similarity to warrant a P or B frame)

If you specify that single image to be an I-frame, you can keep just append it to video1, and the video will be exactly the same except for that 1 frame

[andrewbutkus]

The purpose is to have a file which size on disk is less than a jpg when movie1 stream is remove from movie2's.

> Why can't you change the frametype distribution? Why does it have to stay the same?

It has to stay the same because i want to be able to remove movie1 from movie 2 leaving just the differences (and the differences hopefully being less than the size of a jpg).

>usually leads to worse quality

This is something subjective, depending on how much loss, this may be ok

>Does the appended single image last 1 frame duration , or is it repeated?

Its a single image lasting 1 second (the movie is set to 1fps)

> How are the images related ? (are they part of 1 related scene or completely random images?) , this matters becase the larger the difference the higher the probablilty an I-frame is placed (or even IDR frame)

My test data varies, i have a camera taking pictures of the same object from different angles.
My test data also includes scenary which varies greatly.

[poisondeathray]

It has to stay the same because i want to be able to remove movie1 from movie 2 leaving just the differences (and the differences hopefully being less than the size of a jpg).

clarify what you mean by "leaving just the differences" . Are you digitally subtracting, or using some metric like PSNR? between video 1 and 2 ?

clarify what you mean by "remove". Do you mean trim or cut at the frame boundry ? and comparing or comparing the entire sequence

still very cryptic....

[andrewbutkus]

"Leaving the differences" was just an example of what i expect to achieve from the filesize.

Infact movie1 and movie2 will be delta'd and patched at the byte level. - which is why i would like the original sequence of images to be as exact as possible to ensure that whatever is delta'd at the byte level and is the same, wont end up in the patch file, (therefore reducing the size compared to a jpg)

Andy

[poisondeathray]

EDIT: ok I think I understand , you want to reduce the patch size ? This is your main goal ?


Does it matter that the video uses a different colorspace (YV12) than the original JPEG images (usually use RGB) ?

You can get a large reduction in the main movie, from INTERframe compression (compared to individual JPEG's), a lot of saving too from chroma subsampling (4:2:0) , while RGB is not subsampled

For the individual appended image, I think it would be better to use Intra only , that way you can just append it on to identical video1 . So video1+endjpg = video2 . So you can reduce the patch size, the differences will only be from the end image. I suspect the discrepancies will actually be larger at the byte level if you don't, and the patch will be bigger to fix the differences.

Intra only x264 compression is better than jpeg compression (e.g. for still images) , there are quite a few scholarly articles on this subject if you search . But those look at comparing to encoding from a master (i.e. master=>jpeg image vs. master=>x264 image) , not what you are doing (jpeg itself vs. jpeg=>encoded to x264) . You're looking at different generations . How much quality loss is acceptable for the appended image ?

[andrewbutkus]

bit by bit accurate is ideal, however if there are small differences that is ok, as long as the majority of the bytes are identical, movie2 can be reconstructed from movie1, however if there are massive differences, obviously that increases the patch file as it has to describe these differences.

The patch will contain, any small differences in say the movie header to describe the extra image frame, and the single image at the end, the idea is that the patch file will be less than a jpg image as at least a small proportion of the image will be contained in the original sequence.

However, what im seeing is the bframes option modifying the original sequence of images stream in movie2 so dramatically, that the patch file is pretty huge.

Andy

[andrewbutkus]

Nope, colorspace isnt a problem, the image doesnt have to be exactly the same, just similar to the naked eye

[poisondeathray]

I've edited my post above, before seeing your 2 replies....

If the patch size is the most important, (even more than making the whole movie more efficiently compressed) , then you can disable b-frames. But even adding 1 end frame with b-frames disabled will can still change the frame type distribution and bitrate distribution depending on the content. The "safest" way to do this is to append as described above (so video2 = video1 + appended I-frame)

[andrewbutkus]

My understanding: If its just appended and compressed as intra, that would mean i wouldnt get any saving from previous images (and any possible similarities) as all compression is done on the inputted image itself and not on previous frames. Is this correct?

I've been down a similar route with other intra based codecs and unfortunately i wasnt too impressed with the quality level of the intra frame versus a bog standard jpeg image compressed at a lower %.

>How much quality loss is acceptable for the appended image ?

This is the million dollar question, unfortunately i dont have the answer. For me, its a case of, is the image quality and the filesize smaller than that of an equivalent jpeg picture quality and file size. - of which is subjective from person to person =( ... If it beats jpeg, by utilising additional source images to up its quality and reduce its filesize, so much the better.

Andy

[poisondeathray]

Yes, intra wouldn't rely on info from other frames

The problem is , even adding 1 frame , can alter the frametype distribution and bitrate distribution . (It will depend on content) . So your patch size can balloon up. On some content that end frame might be coded as a P-frame, but that can effect small changes to say frames 5-6 previous. And, x264 is more complex than that, it uses prediction on macroblock basis, not just frames. This will be unpredictable

I'll give it some more thought, but I think the best way to go is still Intra only, and you're right the bitrate savings will be miniscule for 1 frame anyways (not worth it). However, the bitrate savings for a sequence is huge when using x264 vs. jpeg still images