Actual still images in videos

[cornemuse]

I am curious as to how videos are constructed. As I understand it, dvd's are a series of jpeg images (24 or whatever frames/second).
I assume all the other vid formats are similar.

Now, I could certainly be wrong about this! Clue me in!

Another thread asked about time lapse/compressing many days video into 10 minutes.

I wonder if one could take a series of 'still camera' photograph jpeg images and combine them to create their own video. (Not gif anis, way too lo rez)

Create to mpeg, ani, etc.

Possible?

-c-

[smrpix]

I am curious as to how videos are constructed. As I understand it, dvd's are a series of jpeg images (24 or whatever frames/second).

Basically yes, but compressed formats such as DVD are a series of images that rely on information in adjoining images to be completely reconstructed.

I wonder if one could take a series of 'still camera' photograph jpeg images and combine them to create their own video.

Yes, almost every time lapse you see these days is made in precisely this way. Most NLEs have an option for importing sequential stills as motion footage.

[pandy]

In some cases it can be single I frame repeated automatically over time... This is more efficient approach.

[Cornucopia]

Just understand that DVD/BD does NOT incorporate JPEG natively (a JPEG source would be first converted to video / MPEG video I-frames), so you'd have to follow the usual EDIT->ENCODE->AUTHOR route just like any other video source going to DVD/BD.

Scott

[pandy]

Well, actually i refer to something like this:
http://www.pixeltools.com/tech_tip_repeat_frame.html

Btw tmpgenc can be used to create such video syntax (there is hidden option in timeline where manually type of video frame can be marked individually - it can be bit consuming to mark many of frames in such way but there is script file support and such script can be generated way easier and then imported to timeline editor).

[Cornucopia]

Yes, I've used that (as well as TMPGEnc's and DVDMaestro's) tool quite a bit to do just that. But what I was saying to the OP was that it still needs to be encoded from JPEG first. That's what I was referring to with the "MPEG I-frame" comment. I'll revise it...

Scott

[SameSelf]

Thinking about modern codecs as a series of jpeg's, analogous to flip book animation, really only applies to uncompressed or Intra-only codecs. To get a more thorough understanding of what most modern codecs do, you should read up about I, P, and B frames. Wiki is always a good place to start:

https://en.wikipedia.org/wiki/Video_compression_picture_types

The best way to think about it is a Gedankenexperiment using an edge case: a single white pixel moving across a black background. We could store the information for every pixel in the series of images. For a 1920x1080 image, that is 2 MP per YUV channel. At 8 bits per pixel and 30 fps that is nearly 500 Mbps per channel or 1,500 Mbps for the entire image, which is a really fat bitrate and what uncompressed 4:4:4 HD video feels like.

But what if we quarter the U and V channels to 960x540, like what is done for 4:2:0 video? That reduces the bitrate for U and V to 125 Mbps each, and we now have reduced the total bitrate to 750 Mbps, which is what uncompressed 4:2:0 HD video feels like.

But what if we break the image into 16x16 chunks in Y space and 8x8 chunks in U and V space? Each Y chunk would need 2,048 bits and each UV chunk would need 512 bits. But, after analyzing each macroblock, we notice that the pixels are all the same in each chunk (with the exception of one chunk where a white pixel exists) because the entire image is black. So rather than storing 16x16 pixels worth of information in each Y chunk, why not just store one pixel's worth in each chunk? The pixels are all the same after all. So forgetting the single white pixel for just a second, we can reduce the amount of information stored for Y by 1/256 which is only 8,100 pixels per image or 2 Mbps. Wow, we went from 500 Mbps to 2 Mbps just like that! Similarly, U and V compress by a factor of 64 to ~4 Mbps. So, now we have a total of just under 6 Mbps for the YUV image. That is a 128:1 compression ratio. However, not all video compresses this easily. It is also why you get blocky artifacts. But, we now have what is an HD 4:2:0 I-frame.

If we stopped here we would have Intra-only compression which still resembles a flip book. What about P- and B-frames? Well, going back to our single white pixel, let's say in the next frame the white pixel moves 3 pixels left and 4 pixels up. We can take the previous I-frame and create a predictive, "P", frame that stores only the changes from the previous frame. In this example, I really only need to store 3 pixels left and 4 pixels up, something like this: (-3,4). Everything else from the previous frame is the same, so I have no need to store that information again. In our example, this would be a radical reduction from the storage requirement of an I-frame. Typical video offers about 2:1 compression ratio for I- vs P-frames. But to illustrate with an analogy, if I told you I am 3 years older and 4 inches taller than Irene, you would have no idea how old or tall I am unless you know how old and tall Irene is. Thus Irene's age and height is our I-frame, and my age and height is our P-frame. The P-frame cannot exist or be reconstructed without the I-frame, and our video is starting to look like a flip book where 14 out of every 15 pages has been ripped out and replaced with some funky math telling you where to redraw the white pixel from the preceding page. You would have a hard time visually interpreting that as you flip through it.

B-frames are special in that they use both previous and following frames, that is they are bidirectional, to store motion information. Think of P-frames as extrapolating from a previous frame while B-frames interpolate between two or more frames. The upshot is B-frames are even smaller than P-frames. It is like saying my age and height are midway between Pablo and Pasquale where Pablo and Pasquale are P-Frames from the Irene I-frame. My height and age are now a B-frame. If you have just Pablo's height and age, you won't be able to tell me what my height and age are. You need Pasquale's too.

If all that seems endlessly complicated, don't worry. It is. But that is why decoders exist, so we don't have to think about it. It is also why people fight over which encoder is best because there are lots of different strategies for storing information in P- and B-frames to reduce the bitrate as much as possible while minimizing compression artifacts.

But if you can wrap your head around all that, then it should be easy thinking about your other edge case, still pics in video. Even for Ken Burns type effects, the additional information needed for P- and B-frames beyond the initial I-frame is minimal.

Happy encoding!

[pandy]

Thinking about modern codecs as a series of jpeg's, analogous to flip book animation, really only applies to uncompressed or Intra-only codecs. To get a more thorough understanding of what most modern codecs do, you should read up about I, P, and B frames.

For still image P and B particularly are inefficient way of coding - for such special case I frame can be repeated so no data except codec syntax to repeat buffer content is sent - this is most efficient way to transmit such still video - encode one I frame and repeat...

[racer-x]

Another thing you can do with still images, is animate them in a way that makes it look like video. You'll need a multitrack video editor that supports key-frame animation. Here is an example I posted in another thread. It was comprised of two still images and was done all with free tools...

Spider_Hole2.mp4 (3.4 MB)

[SameSelf]

For still image P and B particularly are inefficient way of coding - for such special case I frame can be repeated so no data except codec syntax to repeat buffer content is sent - this is most efficient way to transmit such still video - encode one I frame and repeat...

That makes sense. Thanks for the clarification. What about Ken Burns style effects? That is what I do with all my still image videos.

[poisondeathray]

For still image P and B particularly are inefficient way of coding - for such special case I frame can be repeated so no data except codec syntax to repeat buffer content is sent - this is most efficient way to transmit such still video - encode one I frame and repeat...

That makes sense. Thanks for the clarification. What about Ken Burns style effects? That is what I do with all my still image videos.

It sounds like he's taking about like VFR with still images. 1 image can be displayed for a long period of time using timecodes. For example, if you have a 30FPS video, and displayed a static image for 10 seconds, that would normally require 300 frames to encode. But using VFR , you can encode 1 frame and display it for the same 10 seconds. In addition to saving bitrate, there would be no I-B-P temporal quality fluctuations.

For something like a ken burns style animation, that won't work, unless you have completely static sections in at least part of the animation, but those completely static sections could benefit from VFR

[pandy]

It sounds like he's taking about like VFR with still images. 1 image can be displayed for a long period of time using timecodes. For example, if you have a 30FPS video, and displayed a static image for 10 seconds, that would normally require 300 frames to encode. But using VFR , you can encode 1 frame and display it for the same 10 seconds. In addition to saving bitrate, there would be no I-B-P temporal quality fluctuations.

For something like a ken burns style animation, that won't work, unless you have completely static sections in at least part of the animation, but those completely static sections could benefit from VFR

I'm talking about force decoder to repeat last decoded I frame (null frame syntax in MPEG codec) - this is CFR - idea similar to RFF flag for pulldown - use syntax instead sent data.

This approach can be used only for static picture, animation is normal movie and as such normal methodology should be applied.

[racer-x]

The method of creating one I-frame and have it repeat is not well supported. DVD menus can support it, DVD-Lab used this method to generate slideshow by converting each image into a menu. This limited the number of images you could use. Outside of a DVD, it has little support.

[pandy]

The method of creating one I-frame and have it repeat is not well supported. DVD menus can support it, DVD-Lab used this method to generate slideshow by converting each image into a menu. This limited the number of images you could use. Outside of a DVD, it has little support.

It is supported by every MPEG compliant decoder, side to this such way of saving bandwidth is popular in TV broadcast - you may provide radio or information services with static picture at very low bitrate (very high quality, 720x576 static picture at less than few tens kbps).