VideoHelp Forum
+ Reply to Thread
Results 1 to 23 of 23
Thread
  1. I do a lot of time lapse photography - up to 5 5MP cameras running 13h a day. This generates a lot of files and consumes disk space, so I got the idea to compress the individual images into a high quality video to take the benefit of frame-to-frame compression and later, if needed to "re-compress" the frames back to the individual images. I use ffmpeg/avconf, and for example, to compress/restore I may use this:

    ffmpeg -r 25 -vcodec mjpeg -i inputfile-%06d.jpg -c:v libx265 -preset slower -crf 20 outputfile.mp4

    ffmpeg -i outputfile.mp4 -qscale:v 2 whatever_%06d.jpg


    It works fine, but I wonder, if there is a better way. My main concern is that CFR aims at constant perceived quality but it is a different story when one watches a movie as compared to the images to be extracted. In other words, what would be the best way to compress images to a video when the intention is to restore later the images?
    Quote Quote  
  2. Originally Posted by agatek View Post
    It works fine, but I wonder, if there is a better way. My main concern is that CFR aims at constant perceived quality but it is a different story when one watches a movie as compared to the images to be extracted. In other words, what would be the best way to compress images to a video when the intention is to restore later the images?
    You may explicitly control level range (limited or not level) - also it will be wise to add -q:v (range 2 - 4 perhaps) to control output jpg quality, of course it will e better to use png as it is lossless (but quality improvements may be not perceivable - purely up to you).
    Quote Quote  
  3. Dinosaur Supervisor KarMa's Avatar
    Join Date
    Jul 2015
    Location
    US
    Search Comp PM
    If all you want to do is compress your jpeg files, you can use wxpackjpg (which is a gui for packJPG) which will losslessly recompress jpeg files by about 20-30%. Large pictures directly off a camera seem to compress well. With this program you can view the compressed jpegs and also decompress them. I would suggest you let it be a part of your right click menu, so you can just right click on a folder you want compressed. Instead of selecting all the files and dragging them over to the program. This program won't delete the originals unless you click the overwrite box. I have used packJPG and now this GUI frontend wxpackjpg for years without any issues. If these files are of any importance I would click the verify tab to make sure the output file is the same as the input file, but it adds time to the whole process.

    For a JPEG--->Video route, I'd probably not use x265 and instead stick with x264. As x265 will tend to blur stuff, and probably not best if you want to extract these videos back to pictures. Maybe x264 with a CRF of 15 or below, on one of the slow presets.

    Edit: Should point out that with packjpg, when they get compressed they can't be viewed anymore with any normal jpeg decoder but requires wxpackjpg to view it, or needs to be decoded back to jpeg with wxpackjpg (or commandline packjpg). With packjpg, it's really just putting your images in a type of .zip file specially designed only for jpegs. The guy who created this compressor also has a lossless .MP3 compressor, giving around 15% compression.
    Last edited by KarMa; 11th Jan 2017 at 13:40.
    Quote Quote  
  4. Thank you for your responses. I will take a look at packjpg, sounds very interesting although 20-30% is not that encouraging, at least for this specific project.

    I've been trying both x264 and x265 still playing around with the basic settings. I didn't notice any significant blurriness but will recheck it more carefully again. The cameras I use are the Raspberry Pi native cameras and while they produce reasonably decent images they are surely nothing like top notch. What I eventually do, I scale down the 5MP images to 2MP (1080) for the final time-lapse video so I have some small margin for limited distortion issues / artifacts. What is more critical is to avoid heavy compression in lights or shadows - some of the cameras are IR and the images are later channel swapped with contrast etc. adjusted.

    Talking about the x265 I am getting completely unexpected results in terms of file sizes using different presets. For example, veryfast vs slow gives me much smaller files (the same CRF value).
    ffmpeg version 2.8.10-0ubuntu0.16.04.1 / Lavc56.60.100 libx265
    Last edited by agatek; 11th Jan 2017 at 21:09.
    Quote Quote  
  5. Dinosaur Supervisor KarMa's Avatar
    Join Date
    Jul 2015
    Location
    US
    Search Comp PM
    Originally Posted by agatek View Post
    Talking about the x265 I am getting completely unexpected results in terms of file sizes using different presets. For example, veryfast vs slow gives me much smaller files.
    If you are finding slow to give smaller sizes per CRF, that's what ideally should happen. With slower settings using more complex and bitrate efficient compression settings. You use more CPU power for a smaller size.


    You might also consider using FFV1 for lossless video compression. It can use keyframes to help with redundant video information compression. It seems to support GOPs of up to 500 frames. I'm just not sure how much compression could be had when going from a bunch of JPEGs --> Lossless video. But in your case with timelapse, it's certainly worth a try. And potentially saving all the image quality.
    Last edited by KarMa; 11th Jan 2017 at 21:27.
    Quote Quote  
  6. I mean, the slow setting gives much bigger files than the veryfast one.

    I just tried FFV1: -c:v ffv1 -level 3 and the resulting video is like 3x the size of all the jpeg images.
    Quote Quote  
  7. Dinosaur Supervisor KarMa's Avatar
    Join Date
    Jul 2015
    Location
    US
    Search Comp PM
    Originally Posted by agatek View Post
    I just tried FFV1: -c:v ffv1 -level 3 and the resulting video is like 3x the size of all the jpeg images.
    Maybe something like
    -c:v ffv1 -level 3 -coder 1 -context 1 -g 400

    Also have to worry about staying in the same color space as the original JPEGs. You can use Mediainfo to look at the color space of the original JPEGs, and color space of your created h264 or FFV1 videos to see if they are the same.
    Last edited by KarMa; 11th Jan 2017 at 21:51.
    Quote Quote  
  8. Nope. Helped a bit but still the sizes is more than doubled.
    Quote Quote  
  9. Originally Posted by agatek View Post
    the slow setting gives much bigger files than the veryfast one.
    I don't really use x265 but from what I've read that is not unusual. Like x264 before it, the slower presets change not only settings that increase compression, but also settings that give better quality at the cost of less compression. I think x265 just hasn't reached a good balance with that yet. Even with x264 veryfast usually gives smaller file sizes than faster, fast, and sometimes even medium. Slow, slower, and veryslow usually give slightly smaller files than veryfast. Sometimes placebo gives larger files than veryslow. So encodings at veryfast and medium may turn out about the same size but the latter will have significantly better image quality.

    Originally Posted by agatek View Post
    I've been trying both x264 and x265 still playing around with the basic settings. I didn't notice any significant blurriness
    You won't see it at high contrast edges and detail. It will be in the low contrast detail like film grain, the fuzziness of a sweater, the grain of a stucco wall, things like that.
    Quote Quote  
  10. Dinosaur Supervisor KarMa's Avatar
    Join Date
    Jul 2015
    Location
    US
    Search Comp PM
    Originally Posted by agatek View Post
    Nope. Helped a bit but still the sizes is more than doubled.
    Yeah I did my own time lapse test by taking ~60 similar JPEG photos with my camera and trying to compress it with FFV1. With similar results to you. You almost always get an inflated file size from lossy to lossless but I figured this might be an exception.
    Last edited by KarMa; 11th Jan 2017 at 23:22.
    Quote Quote  
  11. You are not going to save space with lossless intra codec (FFV1 does not use anything GOP)
    Quote Quote  
  12. Originally Posted by shekh View Post
    You are not going to save space with lossless intra codec (FFV1 does not use anything GOP)

    FFV1 use GOP and it intra and inter (not same as MPEG).
    Quote Quote  
  13. Originally Posted by pandy View Post
    Originally Posted by shekh View Post
    You are not going to save space with lossless intra codec (FFV1 does not use anything GOP)

    FFV1 use GOP and it intra and inter (not same as MPEG).
    What am I missing?

    https://raw.githubusercontent.com/FFmpeg/FFV1/master/ffv1.md

    > The FFV1 video codec is a simple and efficient lossless intra-frame only codec.
    Quote Quote  
  14. Originally Posted by pandy View Post
    Originally Posted by shekh View Post
    You are not going to save space with lossless intra codec (FFV1 does not use anything GOP)

    FFV1 use GOP and it intra and inter (not same as MPEG).
    But with real world video the noise is different in every frame so inter frame references don't help with lossless compression.
    Quote Quote  
  15. You don't think it be like this but it do.

    About the topic: I wonder if turning off or weakening mbtree/cutree could be helpful. They take movement of objects into account for the bit distribution. But at the end of the day you still want high bitrate anyways.
    Quote Quote  
  16. Ok, experimental part.

    This is the whole image for overview and some idea...
    Image
    [Attachment 40288 - Click to enlarge]


    Below is the same section of that image compressed and later decompressed (fixed, as in my first post). All using the default, medium presets. 999 images

    1) Original image
    Image
    [Attachment 40289 - Click to enlarge]


    2) x264@CRF18, file size ca 220k
    Image
    [Attachment 40290 - Click to enlarge]


    3) x265@CRF18, file size ca 77k
    Image
    [Attachment 40291 - Click to enlarge]


    4) x265@CRF18, tune=grain, file size ca 120k
    Image
    [Attachment 40292 - Click to enlarge]


    5) x265@CRF14 file size ca 210k
    Image
    [Attachment 40293 - Click to enlarge]


    If you as blind as me, you will need to blow it out a few times to see the differences but generally they are pretty subtle. X265 blurs a bit the image at the proclaimed quality similar to that of x264 (by doubling the size) but then x264 has some other drawbacks - it seem to lose more details in darker areas. Generally looks (my subjective view), for the 4 options tested, x265 with the -tune=grain give most optimal results, allowing small file size and good average reproduction of the details.
    Quote Quote  
  17. Dinosaur Supervisor KarMa's Avatar
    Join Date
    Jul 2015
    Location
    US
    Search Comp PM
    Originally Posted by jagabo View Post
    But with real world video the noise is different in every frame so inter frame references don't help with lossless compression.
    Yeah with my test of 60 frames, even though there was only one key frame, all the nonkeyframes were about the same bitrate as the keyframe.

    Originally Posted by shekh View Post
    > The FFV1 video codec is a simple and efficient lossless intra-frame only codec.
    It supports GOPs.

    https://trac.ffmpeg.org/wiki/Encode/FFV1

    Originally Posted by agatek View Post
    X265 blurs a bit the image at the proclaimed quality similar to that of x264 (by doubling the size)
    CRF between x264 and x265 are not 1:1 the same, nor considered to be close to 1:1.
    Last edited by KarMa; 12th Jan 2017 at 11:18.
    Quote Quote  
  18. Originally Posted by KarMa View Post
    Originally Posted by agatek View Post
    X265 blurs a bit the image at the proclaimed quality similar to that of x264 (by doubling the size)
    CRF between x264 and x265 are not 1:1 the same, nor considered to be close to 1:1.
    General opinion is, x265 allows similar perceived video quality to x264 with half size of the file (bit rate) of x264. It may vary of course depending on the video, but this is what I tried to examine above with different CRFs. Some visible loss in JPEG quality is there, ultimately, these JPEGs are going to be back in a video and i think this loss should be even less visible.

    Getting back to my initial post. What would be the best way to fix the quality? CRF vary (to some extend) quality from frame to frame. Fix "-q"? at a specific number?
    Quote Quote  
  19. Originally Posted by agatek View Post
    Getting back to my initial post. What would be the best way to fix the quality? CRF vary (to some extend) quality from frame to frame. Fix "-q"? at a specific number?
    --qp mode uses a fixed quantizer, like JPEG.

    But I think you're making a mistake going to video, then back to JPEG then eventually back to video.
    Quote Quote  
  20. Originally Posted by sneaker View Post
    You don't think it be like this but it do.

    About the topic: I wonder if turning off or weakening mbtree/cutree could be helpful. They take movement of objects into account for the bit distribution. But at the end of the day you still want high bitrate anyways.
    I want the same metric (is it the right word?) quality of the resulting, decompressed JPEGs and to have this quality similar to the original images. Motion-wise, I think time-lapse could be more demanding comparing to a standard video camera footage. These are properly made static images, meaning an object can suddenly appear within a frame far away from its previous location. No nicely stretched, gradual, smooth, blurred movements to provide some level of frame to frame similarity.
    Quote Quote  
  21. Originally Posted by jagabo View Post
    Originally Posted by agatek View Post
    Getting back to my initial post. What would be the best way to fix the quality? CRF vary (to some extend) quality from frame to frame. Fix "-q"? at a specific number?
    --qp mode uses a fixed quantizer, like JPEG.

    But I think you're making a mistake going to video, then back to JPEG then eventually back to video.
    Why? I mean, what counts at the end of the day is the perceived quality. Would it be THAT different?
    Quote Quote  
  22. Originally Posted by agatek View Post
    But I think you're making a mistake going to video, then back to JPEG then eventually back to video.
    Why? I mean, what counts at the end of the day is the perceived quality. Would it be THAT different?
    You won't know for sure until it's too late. Just to save a little disk space? Disks are cheap.
    Quote Quote  
  23. @jagabo you've got a point and I was thinking that way, but then I also think it is not about to have the best affordable, but optimal or practicable. If I would be concerned about all the factors possible to go wrong, then it would be, I reckon, like 250TB a year and not cheap any longer. - I should not compress that much the original jpegs, I should at least have basic redundancy for the storage, the redundancy should be in a different geographic location. I do things like that for really sensitive files, but for some average, massive and automatic time-lapse shooting, is more like having reasonable feed preserved so later it can be examined, and put together for my own (and my family/friends) enjoyment.
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!