NVENC is better than X264 with the same CRF / CQP ?

[DunnoNo]

I did some tests yesterday and didn't expect that. Yeah NVENC vids look way crisper and retain more detail. Yeah that twice the size and bit rate but even CQP 23 on NVENC looks better than CQP / CRF 18 on X264 which makes the file size about the same.

Or it's a known thing?

[jagabo]

RF/CRF/QP/CQP mean different things in different encoders. So comparing them is meaningless. It's like saying the volume knob on your amp goes up to 100 but someone else's amp only goes up to 10. Those numbers are arbitraty, they don't determine how loud (or good) the sound is.

You should compare bitrate for bitrate.

[DunnoNo]

Ok I just did test. NVENC CQP 24 vs X264 CRF 18. the bitrate is about 30k. NVENC still looks better. It's about the same file size and NVENC renders very quickly.

[KarMa]

the bitrate is about 30k

At those bitrates for 1080p, just about any H.264 encoder should be transparent for most content. Making hardware encoders an easy choice when you get to free up your CPU to do other things. I'm guessing when you use x264, you are using the ultra fast settings?

[DunnoNo]

the bitrate is about 30k

At those bitrates for 1080p, just about any H.264 encoder should be transparent for most content. Making hardware encoders an easy choice when you get to free up your CPU to do other things. I'm guessing when you use x264, you are using the ultra fast settings?

Nope medium preset. When I wait for x264 to render it doesn't stop me from doing other things. It's not that taxing on my rig

[jagabo]

Your claims are contrary to many many comparisons performed by others. I recommend you post some samples of your source and encoded videos.

[DunnoNo]

Hell I used to believe the same thing.

Have a look. PNG format

The weapons sight and the brick pattern on the building wall scream the difference.

[DunnoNo]

Source

[KarMa]

I don't really like to compare single frames as a representation of the entire video, but even the single frames makes x264 look better. Mostly by retaining more of the high frequency detail, such as the purple coat coving the wrist and the orange glove to the stone tile detail to the top right. High frequency content is going to be the first to go with lossy compression, and x264 manages to retain more.

[poisondeathray]

I don't really like to compare single frames as a representation of the entire video, but even the single frames makes x264 look better. Mostly by retaining more of the high frequency detail, such as the purple coat coving the wrist and the orange glove to the stone tile detail to the top right. High frequency content is going to be the first to go with lossy compression, and x264 manages to retain more.

^this




Did you look at any other frames ? e.g. this might be an I-frame vs. B-frame comparison

Maybe post samples of the actual videos

Some parts of the frame look better, some worse.

Look at the entire picture. KarMa pointed out the glove, and arm cloth textures and tile details differences - but street pavement textures look significantly worse with nvenc as the details are blurred away .

Easy to blur details away with any encoder - for example , you can denoise or apply a blur. But retaining details is much more difficult

[DunnoNo]

I don't really like to compare single frames as a representation of the entire video, but even the single frames makes x264 look better. Mostly by retaining more of the high frequency detail, such as the purple coat coving the wrist and the orange glove to the stone tile detail to the top right. High frequency content is going to be the first to go with lossy compression, and x264 manages to retain more.

You're right but still the sight and the brick pattern still retained more detail on NVENC

[DunnoNo]

I don't really like to compare single frames as a representation of the entire video, but even the single frames makes x264 look better. Mostly by retaining more of the high frequency detail, such as the purple coat coving the wrist and the orange glove to the stone tile detail to the top right. High frequency content is going to be the first to go with lossy compression, and x264 manages to retain more.

^this




Did you look at any other frames ? e.g. this might be an I-frame vs. B-frame comparison

Maybe post samples of the actual videos

Some parts of the frame look better, some worse.

Look at the entire picture. KarMa pointed out the glove, and arm cloth textures and tile details differences - but street pavement textures look significantly worse with nvenc as the details are blurred away .

Easy to blur details away with any encoder - for example , you can denoise or apply a blur. But retaining details is much more difficult

Yeah Some parts of the frame do look better, some worse.

yep street pavement does look better on X264

I look into other frames

[ToyzRUsKid]

I'm able to confirm what the OP is positing. I am experiencing a higher quality (smoother and sharper) stream using NVENC over x264. The paradigm shift comes with the new encoder on NVIDIA's RTX cards. I fully realize my observations are anecdotal and may be unique to my hardware, but I'm a believer that NVENC is about to surprise the content creation community.

My use case is different than that of the OP. I stream live content using OBS Studio. My encoding settings are as follows:
NVENC, High Quality, 8000 bitrate, 1080p60
x264, Medium preset, 8000 bitrate, 1080p60

What I'm observing:
NVENC stream is smoother, has crisper detail (text is more defined) as well as something I would not have anticipated. When my network connection was experiencing stability issues, I tested both encoding methods. While using NVENC my bitrate fluctuated within a much tighter window (3k-7k bitrate) and x264 varied from 200-12k. I have no educated explanation for that other than NVENC may utilize a bigger buffer, possibly taking advantage of VRAM. Another interesting observation I would like to investigate more was how NVENC handled frame drops (also related to network instability). I saw NVENC produce a stream that would produce a less jarring response to frame drops. It appeared to stitch the sequence together creating a somewhat stuttering effect rather than jarring skips and freezes.

Nvidia has focused the promotion of the new encoder chip to single pc setup streamers (for which it will be a major opportunity for improving stream quality), but I'm experiencing the improvement running a 2 pc setup.

We are still in the midst of fully exploring this new RTX encoder, but it should be taken seriously as viable if not better option than x264 with modern hardware. I am currently trying to setup so that I can create some good side-by-side comparison footage.

Here is my setup (I have recently upgraded to current gen hardware, so that must be taken that into account):

Streaming Rig:
i9 9900K@5GHz
Asus ROG Strix Z390-E
Corsair Vengeance LPX 32GB DDR4@3200MHz
Asus ROG Strix OC RTX 2070
Corsair H115i Pro

Game Rig:
i7 9700K@5GHz
Asus ROG Strix Z390-I
Corsair Vengeance LPX 32GB DDR4@3200MHz
Asus ROG Strix OC GTX 1080 Ti
Corsair H100i Pro

[DunnoNo]

I'm able to confirm what the OP is positing. I am experiencing a higher quality (smoother and sharper) stream using NVENC over x264. The paradigm shift comes with the new encoder on NVIDIA's RTX cards. I fully realize my observations are anecdotal and may be unique to my hardware, but I'm a believer that NVENC is about to surprise the content creation community.

My use case is different than that of the OP. I stream live content using OBS Studio. My encoding settings are as follows:
NVENC, High Quality, 8000 bitrate, 1080p60
x264, Medium preset, 8000 bitrate, 1080p60

What I'm observing:
NVENC stream is smoother, has crisper detail (text is more defined) as well as something I would not have anticipated. When my network connection was experiencing stability issues, I tested both encoding methods. While using NVENC my bitrate fluctuated within a much tighter window (3k-7k bitrate) and x264 varied from 200-12k. I have no educated explanation for that other than NVENC may utilize a bigger buffer, possibly taking advantage of VRAM. Another interesting observation I would like to investigate more was how NVENC handled frame drops (also related to network instability). I saw NVENC produce a stream that would produce a less jarring response to frame drops. It appeared to stitch the sequence together creating a somewhat stuttering effect rather than jarring skips and freezes.

Nvidia has focused the promotion of the new encoder chip to single pc setup streamers (for which it will be a major opportunity for improving stream quality), but I'm experiencing the improvement running a 2 pc setup.

We are still in the midst of fully exploring this new RTX encoder, but it should be taken seriously as viable if not better option than x264 with modern hardware. I am currently trying to setup so that I can create some good side-by-side comparison footage.

Here is my setup (I have recently upgraded to current gen hardware, so that must be taken that into account):

Streaming Rig:
i9 9900K@5GHz
Asus ROG Strix Z390-E
Corsair Vengeance LPX 32GB DDR4@3200MHz
Asus ROG Strix OC RTX 2070
Corsair H115i Pro

Game Rig:
i7 9700K@5GHz
Asus ROG Strix Z390-I
Corsair Vengeance LPX 32GB DDR4@3200MHz
Asus ROG Strix OC GTX 1080 Ti
Corsair H100i Pro

On 1070 here. However Nvidia claims 15% of bit rate savings on Turing. Since I already see that in some areas NVENC on Pascal does better like I mentioned (the sights, the brick pattern) It's not a stretch to assume with those 15% it can compete and show better results

check this out https://pp.userapi.com/c844724/v844724142/17f6d0/951OECvcOlQ.jpg

[poisondeathray]

Post some sources, tests, samples. If you can, variety of situations, content types and bitrates. That's how you find the truth

People should have a healthy skepticism of marketing slides, especially ones that use PSNR - there is a low correlation coefficient with user perception of "similarity" to the source, and it's easily tricked. Of all the common used metrics, it's probably the worst. But if RTX can compete with "x264 medium", that's a huge improvement

Apparently NVENC HEVC has b-frames enabled now too finally. It's great that they are continuously improving. AMD/ATI should pay attention

[jagabo]

I think what's happening here is that the hardware encoders are geared toward retaining sharp edges at the cost of smoothing away grain, noise, and small-low-contrast details. Whereas x264 works harder to retain such details at the cost of moving edges. Since the sample here is noiseless, grainless computer animation it favors NVEnc.

[DunnoNo]

Post some sources, tests, samples. If you can, variety of situations, content types and bitrates. That's how you find the truth

People should have a healthy skepticism of marketing slides, especially ones that use PSNR - there is a low correlation coefficient with user perception of "similarity" to the source, and it's easily tricked. Of all the common used metrics, it's probably the worst. But if RTX can compete with "x264 medium", that's a huge improvement

Apparently NVENC HEVC has b-frames enabled now too finally. It's great that they are continuously improving. AMD/ATI should pay attention

I'm not telling people to buy into marketing bullshit . I simply showed what they claimed.

Hellooooo! NVENC enabled b frames a while ago. Dunno when but when I started using i in 2017 they already had them



btw Any screenshot comparison software you guys know?

[poisondeathray]

Hellooooo! NVENC enabled b frames a while ago. Dunno when but when I started using i in 2017 they already had them

Not for HEVC . Very recent, maybe months. RTX did not even support it on release , it was only enabled later in drivers

[KarMa]

I'm able to confirm what the OP is positing. I am experiencing a higher quality (smoother and sharper) stream using NVENC over x264. The paradigm shift comes with the new encoder on NVIDIA's RTX cards. I fully realize my observations are anecdotal and may be unique to my hardware, but I'm a believer that NVENC is about to surprise the content creation community.

My use case is different than that of the OP. I stream live content using OBS Studio. My encoding settings are as follows:
NVENC, High Quality, 8000 bitrate, 1080p60
x264, Medium preset, 8000 bitrate, 1080p60

Maybe you can post some samples. Maybe record a video game benchmark with the same bitrate between the two encodings. Or convert a video sample with FFMPEGs NVENC builtin support.

[cajon]

I've had great success with the latest handbrake using NVENC H.265 Constant Quality 27 on 1080p video's...

I do need to caveat a few things:
1) If the video is grainy, it does too much smoothing, so I increase up to 23 (but no more, depending on how grainy the original video is)
2) I also do a weak denoise on everything to keep video size down some, but never more unless I just don't care about the video
3) If the video has a lot of dark scenes, I have to encode with x265 10bit or there is banding/blocking all over the place (on my HDR tv)

Anyone know when Handbrake will start supporting NVENC H.265 10bit?

Thanks!