Stuttering clouds in 4k@30fps... What can I do about it?

[slaindevil]

Hello guys,

I recorded some video with my P3P in 4k@30fps and I came across this issue.

The shadows of moving clouds are stuttering. They do not move smoothly.
The whole movement in the videos (panning, cars, trees in the wind) is smooth
except for those shadows.

Can someone explain to me, how this is possible so I can avoid it in the future?
Here is a little video file showing those stutter clouds:
https://www.youtube.com/watch?v=wX2aZ5a4Tb4

And do I have any chance to smooth out this stuttering with a video editing
software somehow?

Thanks in advanced for any help!

Greetings,
slain

[poisondeathray]

What did the original look like? Did it have the issue but to a smaller extent before Youtube ?

The problem is accentuated by YT's compression. You're seeing I,B,P frame fluctuations. The quality makes a large jump between a new I-frame. Part of the problem is YT uses a low bitrate, another is it uses distributed encoding (segments are processed separately so the motion estimation algorithms used between segments will produce as good results as if it was processed as 1 segment) . Foliage, lots of detail is difficult to compress using what YT currently uses

It is possible to smooth it out (temporal smoothing filters), but at the cost of losing details and possible "ghosting" issues

What recording setting options do you have ? Recording at higher bitrate, higher quality should reduce it if the effect is present in the source

[KarMa]

It seems like all the frames are there, just has a lot of duplicate frames in between. But not sure. Uploading the original here would be best.

[poisondeathray]

^ yes definitely there are duplicates at the beginning. Sometimes equipment takes a bit to "warm up" and the first few frames can be glitched or lower in quality. But I think he's referring to the middle and later parts where the drone is actually moving with steady motion and there aren't any duplicates.

[slaindevil]

I got multiple videos with this issue. So I can upload more examples if needed. All of them are 4k@30fps.

Those duplicate frames, could this be a problem with the writing speed of the memory card?

Since my internet connection isn't the fastest, I uploaded a shorter version of the YouTube video from my first post.
Its from the original video cutted with Avidemux without encoding, just copy.

Edit:
The stuttering is also present in the original video files (like you can see in the attached file).
4k@30fps with ~60MBit/s is already the highest possible quality I can use.
I could only choose smaller resolutions with a higher framerate.

[poisondeathray]

Yes, that's the reason - poor compression efficiency from the camera. There is a relatively short GOP (8 frame) ,with no b-frames. The "popping" occurs on the keyframe as the quality refreshes on the I-frame

It's not really "worth" UHD IMO, since the quality is so low from the bad compression. You would need to record at a much higher bitrate for decent quality UHD - but you're already maxed out on that camera. If you can record at a lower resolution (not higher framerate), at the same bitrate, that should give you better results. Some cameras have an "ideal" setting. But in camera resizing is usually poor - usually you get bad aliasing artifacts an other issues, but definitely experiment with some of the other shooting modes an settings. It might be for this camera, shooting at a lower resolution might give better results

So one suggestion is you can use a temporal smoother and downscale for a decent 1080p. On this type of footage, you're less prone to the side effects from temporal smoothing (ghosting), because the motion is low. But if you shot higher motion content, you'd there would be more artifacts from temporal filtering. E.g if a car or bike was zipping down the path, that would be prone to blending/ghosting. So the type of footage and context is important to consider. Or you might choose to filter selectively certain segments that were prone, or even more selectively through masks/compositing

Here is an example using avisynth's ttempsmooth , but you can use virtualdubs' temporal smoother and there are many filters for various commercial programs. It you list what software you are using , I can probably point you in the right direction. Or if you want, avisynth and vdub are free. This used "medium" settings, but the stronger the settings, the smoother the results, but the higher risk of artifacts and detail loss.

EDIT: and added an A/B compare original/filtered crop of the left side

[Cornucopia]

I'd check to see what the shutter speed was set to, also. If it's very short (and ~independent of video framerate), 1/1000sec for example, it could be creating very sharply defined time-slices that emphasize the stutter, while a standard/expected shutter speed of 180angle (aka 1/60sec for 30FPS) would naturally motion blur things that move faster.
Also note that the image shows the distant clouds (which may be moving fast, but wouldn't seem so by relative distance), but the local clouds above which are moving fast (and thus cast fast-moving shadows) are not visible - they would be stuttering also if they were visible. Also note, the cam is moving forward in a relatively slow fashion relative to visible ground, so the ground motion is both slow and forward, whereas the clouds are fast and moving left->right - the wide aspect ratio emphasizes this difference, too.

Reminds me of those days of playing a "1D" L<-->R video game on a personal device, and then playing the same game at the same "speed" on a big screen device, and it now appears to be moving way faster. just because those pixels are now much bigger percentage of your visual arc.

None of this is surprising. I agree with pdr's assessment.

Did you use a UHS-1 (or better yet, UHS-3) rated card? Maybe you can up the bitrate for use with a faster card (that one was 60Mbps, which is equivalent to ~15Mbps for 1080p30 - I regularly shoot 25-45/50/60Mbps for my 1080 & 720 AVC stuff, so yours does seem a good deal lower). Maybe they'll have a firmware update to raise the bitrate ceiling (or a hacked firmware could appear doing same thing).

Scott

[slaindevil]

I recorded the video files mostly with an ND2, ND4 or ND8 filters on (depending on the weather and day time). The shutter speed was always something around 1/30 to 1/200s.

For recording I used a SanDisk Ultra microSDXC 64GB Class 10.
Since I currently don't have the drone with me I did some googling and it seems, that those 60MBit/s are the maximum it can do. If I choose a lower resolution, I will get a lower bitrate. As soon as I got my drone back I will do some experiments with framerate / resolution and the resulting bitrate just to be sure.

I chose 4k, since I wanted to be able to resize the material down to 1080 to get a better sharpness. So maybe I will get a better result with 2,7k the next time



@pdr: Thanks a lot for the effort. I am very happy with the result of your test file, even if I haven't tried it yet by myself and on my other videos (with more movement in the frames).
I tried to open my file in VirtualDub with the FFMpeg plugin, but during the recompression it crashes when I try to apply the temporal smoother. I will give avisynth a try.

I am mainly working with Adobe Premiere. I also downloaded a demo of Magix Video deluxe, because a friend told me it would be a good piece of software, but I think I will stick to Premiere.

I would be happy to hear about a solution which I can use within Premiere.



What bothers me the most, is the "fear" to encounter the problem again and that I cannot find anyone on the usual drone forums who encountered the same problem. I would have imagined, that a lot of people do have this problem as well.

So just to make this clear to myself.... A higher framerate and/or bitrate (mainly bitrate) would solve this issue?

[poisondeathray]

I recorded the video files mostly with an ND2, ND4 or ND8 filters on (depending on the weather and day time). The shutter speed was always something around 1/30 to 1/200s.

For recording I used a SanDisk Ultra microSDXC 64GB Class 10.
Since I currently don't have the drone with me I did some googling and it seems, that those 60MBit/s are the maximum it can do. If I choose a lower resolution, I will get a lower bitrate. As soon as I got my drone back I will do some experiments with framerate / resolution and the resulting bitrate just to be sure.

I chose 4k, since I wanted to be able to resize the material down to 1080 to get a better sharpness. So maybe I will get a better result with 2,7k the next time

It's not just the bitrate - it's the bitrate for that type of compression used. You can get high quality images at 60Mb/s if it used higher compression. Portable devices tend to use "cheap" compression , because they don't have enough power, or will overheat, or drop frames otherwise (you need stronger processing, or more expensive chip for encoding)

@pdr: Thanks a lot for the effort. I am very happy with the result of your test file, even if I haven't tried it yet by myself and on my other videos (with more movement in the frames).
I tried to open my file in VirtualDub with the FFMpeg plugin, but during the recompression it crashes when I try to apply the temporal smoother. I will give avisynth a try.

Not sure why, maybe UHD footage is too much for 32bit vdub/plugins

I am mainly working with Adobe Premiere. I also downloaded a demo of Magix Video deluxe, because a friend told me it would be a good piece of software, but I think I will stick to Premiere.

I would be happy to hear about a solution which I can use within Premiere.

Ideally, you would want to get a better recording than "fix it in post" . But for plugins - You're looking for temporal smoothing, or deflicker plugins. So in PP/AE common ones would be RevisionFX Deflicker, Digital Anarchy Flicker Free, Granite Bay Deflicker, Boris FX Flicker Fixer

What bothers me the most, is the "fear" to encounter the problem again and that I cannot find anyone on the usual drone forums who encountered the same problem. I would have imagined, that a lot of people do have this problem as well.

So just to make this clear to myself.... A higher framerate and/or bitrate (mainly bitrate) would solve this issue?

If nobody else is complaining about it on various drone forums it might be that you have a defective unit ?

In addition to Cornucopia 's comments, try resetting the camera before your next tests (look in the manual for the procedure).

Yes - Either a higher bitrate , or better compression (more efficient compression) would improve the issue. But it's definitely related to the compression used - it's conclusive in the file you uploaded. But I don't know if that is "normal" for your camera model

[slaindevil]

Not sure why, maybe UHD footage is too much for 32bit vdub/plugins

Ideally, you would want to get a better recording than "fix it in post" . But for plugins - You're looking for temporal smoothing, or deflicker plugins. So in PP/AE common ones would be RevisionFX Deflicker, Digital Anarchy Flicker Free, Granite Bay Deflicker, Boris FX Flicker Fixer

Thanks, Ill have a look into those plugins...

If nobody else is complaining about it on various drone forums it might be that you have a defective unit ?

In addition to Cornucopia 's comments, try resetting the camera before your next tests (look in the manual for the procedure).

And I will have a look into this as well. Maybe its defective, but I don't think so.

Thanks!

[poisondeathray]

Is it only the clouds in this specific shooting day ? In other scenes, other locations, shots on other days, do the clips look ok ? (do other clips not from this shooting day look ok ?)

The shadows of moving clouds are stuttering. They do not move smoothly.
The whole movement in the videos (panning, cars, trees in the wind) is smooth
except for those shadows.

I looked at some other videos from the dji phantom 3 pro, but couldn't find any native samples, only processed/re-encoded. They do have some compression issues too, but not as bad as your clip

[slaindevil]

It is visible in videos from different locations on different days. I uploaded another two files. One is from the same video as my first sample and one is from a new video.
Both have more movement in them. Cars, motorcycles, wind, the drone is moving...

I do have two more examples where it is also visible, but not as much as in the previous video(s).

[slaindevil]

So, today was good weather for testing and I tried several different settings.

What I found out is:
It is stuttering in 4k @30fps and in 24fps as well.
It stops stuttering in 2,7k, no matter if 30fps or 24fps.

I tested this with the MicroSD card, which was included in the drone bundle by DJI.

I attached both 2,7k videos I took. To me it looks like they are smooth and do not stutter.