Encoding with higher resolution doesn't preserve quality.

[poisondeathray]

What also would be interesting is to see how different scaling algorithms fare. eg. lanczos vs. bilinear etc...One is sharper and one is smoother. The sharper one is usually percieved by humans as "better", but creates more noise artifacts. How does the metric interpret this?

[Xpenguin17]

But denoising it would cause deviation from the original. Isn't the purpose of this tool supposed to measure closeness in "quality" to the original? Isn't that diametrically opposed to striving to obtain "transparency"? And how realistic is denoising everything?

The true original never contains noise. Denoised samples better resemble original quality. Also, noise unnecessarily wastes entropy at transparent bitrates, and fucks up the loss consistency at lower bitrates. It also complicates quality comparison like your test. If one codec successfully destroys the noise from the original then how do you measure faithful reconstructions of REAL details with other codecs?

What do you think grain is? It's essentially noise. To the algorithm or computer doing the work calculations, it can't distinguish if a dot is supposed to be noise (created from compression artifacts) or grain (originally present in the original). But here clearly the tool is wrong... and by a lot (and I can show you many more examples). Which encode do you think more resembles the original? It should be pretty obvious here, as I chose a pretty clear cut example.

Grain is a rigor mortis symptom -- the result of the organic film slowly turning into shit.

The other reason these metrics fail, is the type of weighting used. Human perception of quality usually percieves sharpness and clarity as "higher quality" than smoothness and lack of detail, where the metric has difficulty distinguishing "types" of noise. It tends to "penalize" sharper pictures more than lack detail, which is usually the opposite from " human perception of quality model". Also humans have higher weighting for certain parts of the frame which can deviate from the weighting used by the measurement tool (e.g you focus on faces & people usually rather than background stuff because thats usually considered more important). But this is where the subjective & objective world collides and there is lots of room for error

Penalize sharper pictures? What do you mean? If the original was sharp and the processed blurred, it would be just as penalized as a blurry original and sharp output.

[poisondeathray]

The true original never contains noise. Denoised samples better resemble original quality.

Grain is a rigor mortis symptom -- the result of the organic film slowly turning into shit.

What? "never contains noise"... You can't be serious?!

Have you ever seen dpx or studio masters? Have you seen original film prints? Chalk full of grain. Hollywood often even goes so far as to adding more grain in post prior to distribution. Even your average blu-ray have lots of grain. Ever shot anything at low light? Original film grain is also a function of silver particles in the film. Whether you like it or not is a completely different matter. It is there in the true original 1st generation capture.

All I am asking of this tool is to simply measure the differences between the original and another sample. That's all. And it fails. "Original" here is defined as what you started out with, not what it looked like 2 or 3 generations ago , not what you think it should look like. This is how MSU defines it. This is how the universities and testing gurus define it. I mean that's what we are testing right? How well an encoding procedure can reproduce what you started with. This is what this tool is supposed to measure isn't it?

Fact: any post processing will lead to degration of quality and deviation from the original. And you cannot preferentially reduce unwanted noise without eliminating wanted detail, unless you do it by hand, and even that's not perfect.

Also, noise unnecessarily wastes entropy at transparent bitrates

I agree noise impairs compression. But that is a different topic entirely. The whole point is to see how well an encoding procedure resembles what you started out with. By using denoising and filters you add confounding variables. You also add subjectivity as to what you think it should look like. How do you know what the original unprocessed master looks like? or should look like? How far should you denoise? The only way to keep is semi-objective is to test it as is, otherwise you are testing for similarity to another encode which is your "new original"

how do you measure faithful reconstructions of REAL details with other codecs?

EXACTLY. Now you see what is the crux of the matter here. You simply can't with the current crop of "objective tests". None of them can distinguish between wanted "detail" and unwanted "noise." A dot is a dot to them. That's why you have use your eyes...(enter broken record....)

Penalize sharper pictures? What do you mean? If the original was sharp and the processed blurred, it would be just as penalized as a blurry original and sharp output.

I was linking the usefulness of this tool in gauging the perception of quality. There is a good correlation with SSIM and perceived quality, but it breaks down, and this also limits it's usefulness. Sharper pictures are generally percieved to be of higher quaility than blurred pictures . But noise artifacts from sharper pictures tend to lower SSIM much more than blurry lack of detail. Go back and actually look at your test encodes and you should see what I mean.

By weighting I was referring to deviations in the main part of a frame (percieved as important) vs. those perceived as less important. This metric spits out a single number, but deviations in parts of the frame: in important parts like faces, etc.. are usually more important than things like background etc... The metric doesn't distinguish this. e.g. you might have your hollywood star with unrecognizable face due to artifacts, but in another encode, perfect visibility and detail of the actor, but some distortions in the clouds, and the metric would give you the same "score". Similarly, when you look at the total average SSIM, this isn't very useful if there are entire sections that are horrible.

Anyways, I suggest we get back on topic... apologies for the excursion

[lordsmurf]

Redoing footage to suit a display is backwards. Displays can change.

[Xpenguin17]

The true original never contains noise. Denoised samples better resemble original quality.

Grain is a rigor mortis symptom -- the result of the organic film slowly turning into shit.

What? "never contains noise"... You can't be serious?!

Have you ever seen dpx or studio masters? Have you seen original film prints? Chalk full of grain. Hollywood often even goes so far as to adding more grain in post prior to distribution. Even your average blu-ray have lots of grain. Ever shot anything at low light? Original film grain is also a function of silver particles in the film. Whether you like it or not is a completely different matter. It is there in the true original 1st generation capture.

Well, first generation film prints should contain minimal noise, and modern equipment alleviates the low-light issue. I've never seen studio masters so I wouldn't know.

All I am asking of this tool is to simply measure the differences between the original and another sample. That's all. And it fails. "Original" here is defined as what you started out with, not what it looked like 2 or 3 generations ago , not what you think it should look like. This is how MSU defines it. This is how the universities and testing gurus define it. I mean that's what we are testing right? How well an encoding procedure can reproduce what you started with. This is what this tool is supposed to measure isn't it?

Original can be anything. Program surely doesn't give a goddamn. If I switch 'em around and set the processed as the original, obviously the better quality frames will receive the lower scores. The application compares the difference and reports, nothing else. The cleaner original qualifies fine and makes comparison easier.

Fact: any post processing will lead to degration of quality and deviation from the original. And you cannot preferentially reduce unwanted noise without eliminating wanted detail, unless you do it by hand, and even that's not perfect.

Modern tools allow you to select an area of pure noise so it knows exactly what to remove and what to keep.

How far should you denoise?

Far enough to stop the movement of the noise. Utter removal is impossible.

EXACTLY. Now you see what is the crux of the matter here. You simply can't with the current crop of "objective tests". None of them can distinguish between wanted "detail" and unwanted "noise." A dot is a dot to them. That's why you have use your eyes...(enter broken record....)

Hence why you remove that sumbitch, so it doesn't interfere with the testing.

I was linking the usefulness of this tool in gauging the perception of quality. There is a good correlation with SSIM and perceived quality, but it breaks down, and this also limits it's usefulness. Sharper pictures are generally percieved to be of higher quaility than blurred pictures . But noise artifacts from sharper pictures tend to lower SSIM much more than blurry lack of detail. Go back and actually look at your test encodes and you should see what I mean.

I know objective tests are unreliable, I'm just saying they do better if you help 'em out. But yo, I never reviewed the actual SSIM algorithm so hell if I know how any of it works.

By weighting I was referring to deviations in the main part of a frame (percieved as important) vs. those perceived as less important. This metric spits out a single number, but deviations in parts of the frame: in important parts like faces, etc.. are usually more important than things like background etc... The metric doesn't distinguish this. e.g. you might have your hollywood star with unrecognizable face due to artifacts, but in another encode, perfect visibility and detail of the actor, but some distortions in the clouds, and the metric would give you the same "score". Similarly, when you look at the total average SSIM, this isn't very useful if there are entire sections that are horrible.

True, human eyes perceive objects instead of detail, not to mention deviations in peripheral vision n' all...

And now with video compression becoming less and less linear, SSIM will have to be replaced with a more effective metric, just like PSNR had to go when the transition from MPEG-1/2 happened and loss was less consistent with the whole frame.

Now with mb-tree, I got high average SSIMs where many scenes looked like dogshit. Lowering bitrate by half only lowered SSIM by 0.0100 -- anything that moves was trashed but backgrounds were retained perfectly.

Anyways, I suggest we get back on topic... apologies for the excursion

OP got what he needed, old topic was already over.

[jagabo]

Well, first generation film prints should contain minimal noise

Certain types of film and processing are often chosen to enhance film grain. Look at the movie 300 for an example. High ASA film (used in low light conditions or high speed photography) has large grain. Film grain noise is inevitable under those conditions.

[poisondeathray]

Original can be anything. Program surely doesn't give a goddamn. If I switch 'em around and set the processed as the original, obviously the better quality frames will receive the lower scores. The application compares the difference and reports, nothing else. The cleaner original qualifies fine and makes comparison easier.

Exactly. "original" is defined as whatever you put in the 1st box. It shouldn't matter if it has grain, noise, playboy bunnies, or whatever. It should just do a simple comparison right? But it fails to do so properly and is unreliable.

Modern tools allow you to select an area of pure noise so it knows exactly what to remove and what to keep.

"exactly what to remove and what to keep."

Hmm. I'm quite familiar with many denoisers and tools, with avisynth scripts, using masks, photoshop, AE. You must be referring to a super uber secret one that allows you do this perfectly... because I don't know of any that can do this exactly perfect.

Hence why you remove that sumbitch, so it doesn't interfere with the testing.

No, even you said above the "original" is whatever you put in box#1. It shouldn't interfere at all. Just measure if it's different from the entry in box#1. I just want to know if it's closer or not to the "original in box#1", and this metric clearly fails in some situations. We need something better for an "objective" measure, otherwise "your eyes" are still the gold standard.

Now with mb-tree, I got high average SSIMs where many scenes looked like dogshit.

This is what I was getting at. Everyone seems impressed by mbtree when they look at the single number spit out (the overall average ssim) but fail to look at the scenes that were severely damaged. It's not very balanced or tweaked properly yet. IMO, there are too many problems with it or you have to use it restricted to certain types of content. I don't think it should have been commited to the main branch.

[Xpenguin17]

"exactly what to remove and what to keep."

Hmm. I'm quite familiar with many denoisers and tools, with avisynth scripts, using masks, photoshop, AE. You must be referring to a super uber secret one that allows you do this perfectly... because I don't know of any that can do this exactly perfect.

HUZZAH!

No, even you said above the "original" is whatever you put in box#1. It shouldn't interfere at all. Just measure if it's different from the entry in box#1. I just want to know if it's closer or not to the "original in box#1", and this metric clearly fails in some situations. We need something better for an "objective" measure, otherwise "your eyes" are still the gold standard.

Yeah and if it retains details but destroys very high-frequency noise (such as blocking) then it will drop a couple points even though the processed looks better. Or what if it completely erases noise and retains SOME details, how do you compare it to other codecs that just smear the noise, making it worse but faithfully reconstructs edges in static scenes, and so on? Then sometimes you get those trailing chroma artifacts and start to wonder whether they even existed on the original or not.

Why complicate shit? Clean up the source god damn it.

This is what I was getting at. Everyone seems impressed by mbtree when they look at the single number spit out (the overall average ssim) but fail to look at the scenes that were severely damaged. It's not very balanced or tweaked properly yet. IMO, there are too many problems with it or you have to use it restricted to certain types of content. I don't think it should have been commited to the main branch.

Have they already implemented B-pyramid-compatibility? Unchecking B-pyramid resulted in worse quality, yet the header of the encode reports b-pyramid to be off.

[poisondeathray]

Hmm. I'm quite familiar with many denoisers and tools, with avisynth scripts, using masks, photoshop, AE. You must be referring to a super uber secret one that allows you do this perfectly... because I don't know of any that can do this exactly perfect.

HUZZAH!

Yes I use neat video too, it's a great filter, but it's far from perfect. It tends to oversmooth details leaving you a "plastic doll" look. Like all other denoisers it cannot distinguish wanted detail from unwanted noise. The author even says this. This is a fundamental fact of all denoisers

exactly what to remove and what to keep.

OK here are some examples. Tell me if you think the neat video performs well in terms of "exactly what to remove and what to keep." The original and neat sample was taken from the downloadable samples on the neat video page. Better yet, if you think you can do a better job with neat or provide better settings than what the author did, please post it

Ignore the color corrected differences; just compare the details in the rocks, on the lizard's skin etc.. especially in the darker shaded areas, like the lizard's left leg


original


left = neat video ; right = custom filters

Have they already implemented B-pyramid-compatibility? Unchecking B-pyramid resulted in worse quality, yet the header of the encode reports b-pyramid to be off

No b-pyramid doesn't work with mb-tree currently. If you try to use it, look at your log file, it will give you a warning and automatically disable it.

[creamyhorror]

Tangentially, wasn't there an incredible grain-retaining cleanup of that lizard clip posted here? (Or maybe on Doom9, hmm.) It used Neat Video as a step. It was really very impressive.

About mbtree, I have to say I was a bit worried when I saw DS's sample anime clip and how all moving areas looked terrible, but the problem hasn't been very apparent to me in clips with more sane bitrates (that anime clip was 67kbit@720p). Maybe I need to do actual comparisons, from what you guys have said.

[Xpenguin17]

You can do better than that. Study:

Mine doesn't have the ringing above rocks and has slightly more detail. Your customized denoise result has chroma noise everywhere and blurrier backgrounds. Some regions contain more detail than NV and some less but I conclude that NV outperforms it by a margin, plus without all the hassle of adjusting settings of 10 fuckin filters.

And I know it's possible to do better, 'cuz I only bought NV 2 months ago and haven't tried out everything yet.

The key thing is not removing noise though (you can't) just halting it from running all over the place. Setting Y channel noise removal to 0% and letting the temporal filtration do its shit will work for most purposes and perfectly preserve details. You got no frame of reference to remove noise in the spatial domain because you're dealing with a 2D picture. So you just gotta live with that.

No b-pyramid doesn't work with mb-tree currently. If you try to use it, look at your log file, it will give you a warning and automatically disable it.

I compared 2 mb-tree encodes with the exact same settings, one with B-pyramid checked. The one with B-pyramid on was higher quality, but it said in the header that it's off, even though I know it was on when it was encoded. 'Sup with that?

And creamyhorror, are you on crack, bro? 720p at 67 kbps is about 8000:1 compression ratio. That is not friggin' possible unless the video was a still picture with no movement.

[creamyhorror]

And creamyhorror, are you on crack, bro? 720p at 67 kbps is about 8000:1 compression ratio. That is not friggin' possible unless the video was a still picture with no movement.

Mostly a still picture

[lordsmurf]

And it looks mostly like crap when in motion. Sorry.

[Xpenguin17]

And creamyhorror, are you on crack, bro? 720p at 67 kbps is about 8000:1 compression ratio. That is not friggin' possible unless the video was a still picture with no movement.

Mostly a still picture

That's a 480p video... but impressive nonetheless. I wonder if a DVD quality south park episode can finally be tolerable at the same filesize as those notorious trashy .RM rips (36 megs.)

[creamyhorror]

That's a 480p video... but impressive nonetheless. I wonder if a DVD quality south park episode can finally be tolerable at the same filesize as those notorious trashy .RM rips (36 megs.)

Oops, true, I should've downloaded the clip and checked. At 36MB, South Park is probably going to look just fine. It already would have, anyway, before mbtree.

And it looks mostly like crap when in motion. Sorry.

At 67kbps, yes...it's meant to show the effect of mbtree. At normal bitrates, things are quite different.

(For those who aren't clear on what's happening, x264 is actually determining the level of movement in sub-areas of the image and allocating a lower number of bits accordingly. No other encoder has this capability yet, AFAIK.)

[creamyhorror]

Here's a 1Mbps CGI/3D clip at 1080p:

Big Buck Bunny @ 1mbps (MP4)

(Fades are poor; we're waiting for the weight-p patch to fix it)

[poisondeathray]

Mine doesn't have the ringing above rocks and has slightly more detail. Your customized denoise result has chroma noise everywhere and blurrier backgrounds. Some regions contain more detail than NV and some less but I conclude that NV outperforms it by a margin, plus without all the hassle of adjusting settings of 10 fuckin filters.

You did a better job than the neat video posted demo overall. You were able to keep more detail with the lizard, at the expense of having a "noiser" background and frame edges. Much better control over ringing above the rocks. But a lot of the important details are missing compared to the custom filtered encode.

The point of this exercise wasn't to see "who can do better;", it was to illustrate you cannot keep detail and eliminate noise st the same time perfectly and your example clearly supports this.

I did mine using masks (different areas got different treatment), something automated tools like SSIM cannot do i.e. process different parts of a single frame (this is why we went down this discussion in the first place). Similarly , neat video by itself (just like most denoisers that don't use masking) is limited to processing the entire frame. So either you lose to much detail or keep too much noise. You can't differentially process without human intervention and using things like masks, and even then it's far from perfect.

Have you played the original file? This is a depth of field shot. The foreground subject(s) are supposed to be in focus, and the background elements out of focus. So you shouldn't be leaving all that sharp crud around the "7" logo or the horizontal lines under the logo. Why are the top left corner and side outer edges of the frame so ratty? The background looks like wrinkled paper. Since the background should be out of focus, why is there all this sharp background noise in your example? It looks sharper than the foreground lizard and should be the other way around. By the same token, I may have oversmoothed the background my example, because there should be a tiny amount of grain or noise in the background, but certainly not all the crud that you have there.

Yes you have more detail in some parts , but it's distributed awkwardly where it's not supposed to be - in the less important background. The areas of high detail should be with the lizard and rocks ie. the important parts, and you are missing that compared to the custom filtered encode, especially on the lizard's yellow under neck & belly, it's left side, and the dark areas of the rocks. It's all smooth there. Neat video is notorious for eroding important details like that.

The detail that you did manage to keep is distributed in a splotchy fashion over the lizard, and the rock underneath the lizard that is covered by shade (There is some detail in some regions and all of a sudden there is a smooth patch). Part of the reason might be you never corrected for levels first. Instead of just blurring them the details away, there is information in those darker areas, you just have to bring them out first

Yes there is more chroma noise in my filtered version, but there is also more detail in those sections. This illustrates the point of this exercise: you cannot filter the noise perfectly and keep the detail. In my opinion, having detail there but a tiny bit of noise is better than the plastic doll look that has been smoothed out.

And yes I do have some ringing around the top of the rocks - thanks for bringing that up - but I can add another filter or edge clean that. But again, that's not the point of this exercise. The whole point is I have to add yet another filter or do something manually because automated tools cannot diffentiate between wanted detail and unwanted noise all by themselves.

[poisondeathray]

I compared 2 mb-tree encodes with the exact same settings, one with B-pyramid checked. The one with B-pyramid on was higher quality, but it said in the header that it's off, even though I know it was on when it was encoded. 'Sup with that?

That's what I mean. It is automatically disabled. It wasn't on. Look at the log file. They were both off and should have given you similar encodes. The difference between them would be attributed to non-determinism

How did you determine the one with b-pyramid was "higher quality" ?

[Xpenguin17]

An important point I left out is having an ideal sample of the noise. A program is like a savant with shit for brains by default; it does what you tell it to do. If you select a noise sample that contains some details, then you can expect to end up with plastic.

The reason the background looks rag-textured is because I didn't select a noise sample from the background, but during the shadow scene where I can select a larger area. Not to mention that I set spatial removal to 0%. I've removed temporal noise only.

Whether the scuzz around the '7' should remain is irrelevant. It's a different breed of scuzz and NV only removes the breed you introduced it to with the noise sample, remember?

Moreover, my background is not noisy -- it looks like it in the screenshot but the noise is really smooth and you barely notice it move.

Remember what I said earlier about locking the noise instead of removing? It looks something like this:

Noise.
Denoise.

The point of this exercise wasn't to see "who can do better;"

Don't worry, no e-penis growth intended. Doom9's the place for that.

How did you determine the one with b-pyramid was "higher quality" ?

It looked better, but I dunno anymore. I'll have to redo it.

Here's a 1Mbps CGI/3D clip at 1080p:

DAYUM! At this rate, it's possible to fit a 90-minute animated movie in a CD. 700MB 1080p movie... that is ******* crazy. I didn't think even H.265 would be able to reproduce high-quality frames above 720p at CD bitrates... NEVER thought H.264 would somehow achieve this with 1080p content.

Too bad this trippy shit didn't exist in 2003. It would've been extremely useful.

[poisondeathray]

Remember what I said earlier about locking the noise instead of removing? It looks something like this:

Noise.
Denoise.

Yes I know how neat works. And of course it does a good job on a black background. Try putting that noise over a video with detail. Like a lizard.

You cannot "exactly" remove noise and keep detail, as you claim. If you tried to get rid of the background noise, you would have ended up looking like the neat demo and lose even more detail in the lizard, right? Neat does a good job, but tends to oversmooth. You could improve your results by using masks, because noise isn't always evenly distributed, and sometimes you have different types of noise in the same frame.

[Xpenguin17]

Remember what I said earlier about locking the noise instead of removing? It looks something like this:

Noise.
Denoise.

Yes I know how neat works. And of course it does a good job on a black background. Try putting that noise over a video with detail. Like a lizard.

That's the work of a simple temporal filter, not Neat video. NV does way better. I was illustrating the idea of halting noise instead of removing it, because a program doesn't know what's "under" the layer of noise in a 2D image. Noise is part of the video, as long as it's kept in control.

You cannot "exactly" remove noise and keep detail, as you claim. If you tried to get rid of the background noise, you would have ended up looking like the neat demo and lose even more detail in the lizard, right? Neat does a good job, but tends to oversmooth. You could improve your results by using masks, because noise isn't always evenly distributed, and sometimes you have different types of noise in the same frame.

If you have an accurate noise profile and the noise isn't too intense (which describes most content sourced from professional film equipment) then you won't lose any detail.

But whatever, I concede. Perfect removal and retention of details is impossible.

I tried again with NV, and this is the best I could do: