x264 deblock setting for very soft material (Video8/VHS)

[ktmf]

Hi all,

I'm trying to fit as much home-video Video8 and VHS tapes on as few Blu-ray discs as possible, at an for me acceptable quality.

I was wondering: VHS and Video8 have a fairly low resolution. When captured at 576i the material is rather soft. I can't use a lower resolution, as I want to author spec-compliant Blu-ray discs, so I can hand these discs out to several family members without a manual. However, when I encode this material with x264, I see quite a lot of blocking artifacts at the default CRF of 23. I first lowered the CRF to 18, but that leads me to needing about 4 single layer BD-R's to store everything. As I'm going to hand these out, I rather have everything fit on two BD-Rs instead.

So, I was wondering, what if I try again at a CRF of 23, but with a much higher deblock setting? As the material is already quite soft, I would imagine this is hardly noticeable. Has someone tried this before? I couln't find much on this topic on the web though various search engines.

[hello_hello]

The picture may be soft but it's probably noisy and still hard to encode (especially VHS tapes). You'd probably be better of denoising and cleaning up the source as much as possible.

I'm not a deblocking expert as I generally use the x264 tuning defaults and CRF18. The tunings adjust the deblocking automatically. For the moment, here's an explanation as to how the deblocking works.

How To Use Mpeg4 AVC Deblocking Effectively

[godai]

im not expert by any means, but i suggest use firewire, i just test it this year and its get results

how i kn ow?
what i use its check frame by frame, or pic by pic in virtualdub and compare with usb composite recordings, you get more detail and sharper video. its not so big difference but its makes it better.

[ktmf]

I did some tests with various deblock settings, and I think I'll stick with CRF 18 after all. I don't like the results.

CRF 23 smears the noise while CRF 18 mostly retains it. My biggest concern is the things various deinterlacers and upscaling filters are going to do. These artifacts are magnified when viewed on a HD screen. I could of course deinterlace and upscale myself, but I doubt that will decrease the storage needed. Storing at native resolution in high quality probably beats storing in upscaled form with the same bitrate.

One of my results is attached. As you can see, CRF 23 introduces quite a lot of artifacts over CRF 18. Increasing deblocking strength helps a little, and if I really had to stick with CRF 23, I'd choose deblock 6:6. I don't mind the softness, but I don't like the artifacts of CRF 23 deblock 0:0. High deblocking gets rid of that without softening the material too much. By the way, the attachment is 5 150x150 crops magnified 200% (300x300) at different x264 settings.

The picture may be soft but it's probably noisy and still hard to encode (especially VHS tapes). You'd probably be better of denoising and cleaning up the source as much as possible.

Well, I just found out I'd rather keep the noise

im not expert by any means, but i suggest use firewire, i just test it this year and its get results

how i kn ow?
what i use its check frame by frame, or pic by pic in virtualdub and compare with usb composite recordings, you get more detail and sharper video. its not so big difference but its makes it better.

I highly doubt that. The horizontal resolution of Video8 is already quite low (~250 TV lines per picture height) and my capture card is quite capable of processing that. I connected my current HD cam through composite (it still has a composite out in mini-jack form) and its sharpness far surpasses anything that I capture from VHS or Video8.

Moreover, I don't have equipment with Firewire out and I do not see the point in buying such equipment

[hello_hello]

I'm generally not a fan of upscaling (animation being the exception) but I'm definitely a fan of de-interlacing before encoding,although I always de-interlace to 50fps/59.94fps because I don't need to worry about Bluray compatibility. If you're an Avisynth user I'd still consider de-interlacing with QTGMC and encoding as progressive. I think for SD you have to use --fake-interlaced in the x264 command line to make it strictly Bluray compatible. Or of you upscale to 720p, 50fps/59.94fps is Bluray compatible.
http://www.x264bluray.com/home/720p-encoding
QTGMC will also denoise and help with existing blocking while retaining a fair amount of detail.

I posted them in another thread a while ago, so there's some de-interlacing examples attached if you want to have a look. The original clip is also included. It's a kind of de-interlacing torture test. The examples were both de-interlaced to 50fps rather than 25fps (as that's how a player would de-interlace). The QTGMC example has extra noise filtering enabled. The Yadif example is the sort of de-interlacing you'd expect from a player.
QTGMC has settings for restoring noise after de-interlacing, even a lossless mode, but if the clip is noisy I prefer to let QTGMC have at it, otherwise it can retain the problems you're trying to fix.

Both the de-interlacing samples were followed with GradFun3() from the DitherTools package. If you're not an Avisynth user that won't mean much, but it takes an 8 bit clip, converts it to 16 bit, smooths out the gradients, then dithers it back to 8 bit for encoding. It can help prevent the encoder causing banding, and possibly some blocking too.

If you're not an Avisynth user you could upload a sample of your video for someone else to de-interlace with QTGMC for you, and then decide whether or not to give yourself a crash course in using Avisynth.

Deinterlacing Examples.zip (44.1MB)

[ktmf]

I'm generally not a fan of upscaling (animation being the exception) but I'm definitely a fan of de-interlacing before encoding,although I always de-interlace to 50fps/59.94fps because I don't need to worry about Bluray compatibility. If you're an Avisynth user I'd still consider de-interlacing with QTGMC and encoding as progressive. I think for SD you have to use --fake-interlaced in the x264 command line to make it strictly Bluray compatible. Or of you upscale to 720p, 50fps/59.94fps is Bluray compatible.
http://www.x264bluray.com/home/720p-encoding
QTGMC will also denoise and help with existing blocking while retaining a fair amount of detail.

The samples look rather impressive indeed. However, I don't see this working in my case. My material is much, much softer.

I have 576i material that I want on Blu-ray in high quality, and as much material on a single layer disc as possible. Currently, I can fit about 10 hours on one disc. If I would deinterlace, I would have to upscale to 720p to have 50fps. Deinterlacing makes the already soft material even softer. I highly doubt x264 would be equally efficient on upscaled material.

[hello_hello]

QTGMC doesn't soften as such, and it has an option to increase sharpness. If you're removing noise you can lose some fine detail (if there is any) but that's par for the course with denoising. The point I was making though, is if the video has to be de-interlaced, QTGMC is generally the best way to do it regardless of the source quality. A player's de-interlacing won't be of the same quality, which is what you were concerned about, and of the two samples I uploaded, the only difference was the de-interlacing method used. Same encoder settings and CRF value, but the QTGMC encode was 50% smaller than the Yadif encode because the video was cleaner and easier to compress.

Then there's interlaced encoding being a fair bit less efficient than progressive encoding. I couldn't tell you how much as I never encode as interlaced, but I assume you are. If not, it wouldn't be Bluray compatible and I don't know if a player would assume it's interlaced, or not deinterlace on playback as it should.
http://www.chaneru.com/Roku/HLS/X264_Settings.htm#tff

Doubling the frame rate (50fps compared to 25fps) doesn't actually increase the file size by all that much for a given CRF value. For a SD video deinterlaced with QTGMC, I think it's usually around 5%, or 10% tops.

And while it's an increase, 720p isn't a massive leap up from 576i. I assume it's all 4:3, so before adding borders it'd be 960x720 instead of 720x576, and the borders take almost no bitrate to encode. I haven't seen your source so I'm just pontificating theory, but if you upload a sample I could deinterlace, denoise, upscale and encode it for you to give you an idea of what the btirate will be for a given CRF value so you can compare it to what you're doing at the moment. It's up to you, and of course if you're not an Avisynth user, you'll need to become one.

[ktmf]

Then there's interlaced encoding being a fair bit less efficient than progressive encoding. I couldn't tell you how much as I never encode as interlaced, but I assume you are. If not, it wouldn't be Bluray compatible and I don't know if a player would assume it's interlaced, or not deinterlace on playback as it should.
http://www.chaneru.com/Roku/HLS/X264_Settings.htm#tff

I question the claim that interlaced encoding is less efficient. I can't find any backup for it. To me it seems, x264 actually has more possibilities/strategies to encode. Only downside is that weightp is not implemented.

edit: perhaps progressive coding is more efficient when comparing at the same framerate, but not when comparing 25 interlaced frames per second (50 fields per second) to 50 progressive frames per second. I'd say the latter is a fairer comparison, as I value the fluidity of having 50 frames/fields per second.

And while it's an increase, 720p isn't a massive leap up from 576i. I assume it's all 4:3, so before adding borders it'd be 960x720 instead of 720x576, and the borders take almost no bitrate to encode. I haven't seen your source so I'm just pontificating theory, but if you upload a sample I could deinterlace, denoise, upscale and encode it for you to give you an idea of what the btirate will be for a given CRF value so you can compare it to what you're doing at the moment. It's up to you, and of course if you're not an Avisynth user, you'll need to become one.

Even when ignoring the black bars, the number of luma samples triples when moving from 576i @ 50 fields per second to 960*720 @ 50 frames per second. So yes, it is a massive leap in my eyes. The material is from Video8 and VHS, so it's inherent resolution is more like 320x288 per field (assuming 250 TVL/PH). So, actually, I'm oversampling by a factor 7 with respect to the original material.

As I am not an experienced Avisynth user, so I did some tests with ffmpeg's yadif, which I am familiar with, just to test what bitrates do when deinterlacing and upscaling. A result is attached. I'm not impressed. It seems to me keeping it at 576i has a higher quality/bitrate tradeoff than converting to 720p as you suggested. I had to change crf from 18 to 19 to keep the bitrate in the same range. While it is hard to compare material at such crfs with lack of any detail in the source material, I still see differences, most notable a bit of blurring of details.

So, I'm not convinced deinterlacing+upsampling is better, on the contrary.

[hello_hello]

I question the claim that interlaced encoding is less efficient. I can't find any backup for it. To me it seems, x264 actually has more possibilities/strategies to encode. Only downside is that weightp is not implemented.

I was simply quoting the Wiki. I never encode as interlaced.

As I am not an experienced Avisynth user, so I did some tests with ffmpeg's yadif, which I am familiar with, just to test what bitrates do when deinterlacing and upscaling. A result is attached. I'm not impressed. It seems to me keeping it at 576i has a higher quality/bitrate tradeoff than converting to 720p as you suggested. I had to change crf from 18 to 19 to keep the bitrate in the same range. While it is hard to compare material at such crfs with lack of any detail in the source material, I still see differences, most notable a bit of blurring of details.

So, I'm not convinced deinterlacing+upsampling is better, on the contrary.

As a general rule, you can use higher CRF values for higher resolutions without it seeming to reduce the quality, assuming in the end it'll all be viewed on a 1080p display. Think of the Xvid AVIs you've no doubt seen that look okay on a small screen or CRT, but upscaled to 1080p they look horrid because the encoder artefacts are upscaled too. I often encode 1080p at CRF20, but I almost never use it for SD.
Have a look at the screenshots here. That thread is a whole other argument, but they're 1080p's worth of a 4k frame, and each was taken from a frame with a fair bit of movement, but the x264 screenshots were encoded at CRF 24. With a clean UHD source it looks pretty good.

The extra blurriness in your 720p encode could be the result of soft(ish) upscaling. What upscaling method (resizer) did you use? How much did you enlarge those screenshots? Enough that you couldn't duplicate it under normal viewing conditions without a magnifying glass and your nose touching the screen?

How do you know the 567i encode is how it should look, and what appears to be detail isn't de-interlacing artefacts? Have you compared them to a de-interlaced version of the source using the same de-interlacing method to see which one is most similar?

Anyway, just to see if I might be talking complete crap, I used the source I uploaded previously, cropped it as before (688 x 572) and anamorphically encoded as interlaced. With the same encoder settings as before (aside from enabling interlaced encoding), the output was 6569 kb/s.
I de-interlaced with QTGMC's default settings (no extra denoising), and even though it should display at close to 16:9, I squished it to 960x720 and added borders to the sides. The resulting bitrate at CRF18 was 7251 kb/s. I ran a second encode at CRF19 and the bitrate was 6267 kb/s.

De-interlacing to 50fps with Yadif and encoding at CRF19, the bitrate was 10.4 Mb/s.

So my theory does hold up, and it was just a theory, given you don't want to share a sample of your source, but as long as you clean up a low quality video before upscaling, and Yadif de-interlacing on it's own apparently doesn't fall into that category for my tests, it's possible to upscale without having to increase the bitrate, or without increasing it much. Obviously it'll be quite source dependant though.

For the 720p screenshots below, I de-interlaced while cropping the source to 4:3 instead of squishing it. At CRF19 the QTGMC encode came in at 6280 kb/s, while for Yadif it was 10.2 Mb/s, so it didn't change the bitrate too much. I can upload the encodes if you want, but here's screenshots from each, upscaled to 1080p by MPC-HC.

1. The mpeg2 source, de-interlaced with ffdshow/Yadif.



2. The anamorphic interlaced encode, 688 x 572, CRF18, de-interlaced with ffdshow/Yadif, 6569 kb/s.
I tried to discover the cause of the chroma bleeding in some frames without any luck. I tried different renderers, different de-interlacing methods, hardware decoding/de-interlacing and software decoding/de-interlacing. In the end I attributed it to interlaced encoding artefacts that don't de-interlace well, but that could very well be a load of bollocks.



3. 720 upscale, CRF19, Yadif de-interlacing, 10.2 Mb/s.



3. 720 upscale, CRF19, QTGMC de-interlacing, 6280 kb/s (for CRF18 it was 7131 kb/s).