Downsizing, jagged edges around very bright colors, Oh yes....

[The.Yield]

Hello all. I am a die hard user of PMPs or Portable Media Players.

Sony PSP, Cowon A2, etc (And my new A3 is on the way!)

At any rate -- I am very, very well versed in the process and principles of conversion and downsizing.

But there has been one nagging artifact that I've always seen over the years.

That artifact rears it's ugly head when a video I've downsized has some scenes that contain very bright colors (like Pixar films etc).

If you ever done something like this you may know what I'm talking about.

The edges of the brightly colored object will become aliased, or jagged like a stair step, in basic terms.

The entirety of the rest of the film will be sharp as a pin, but this bright RGB 255ish colors will become aliased, always.

I've never tried to pin down the cause. Simply passing it off as a limitation of these small, portable screens has always been much easier, lol.

So now I'm trying to track down the root cause of this artifact without doing a ton of test renders lol.

My suspicion is that it has something to do with the conversion of color depth from YUV to RGB... But I don't know that for a fact, and really that's just a guess. These portable screens usually have 16.X mil color so that may have something to do with it as well...

Has anyone already figured this one out so that I don't have to, lol?

Thanks for reading.

[guns1inger]

I would suspect it is more likely a problem with resizing interlaced or badly deinterlaced footage.

Perhaps you could describe the process you use to prepare the video for your player, and someone might be able to find the flaw and advise on how to improve it.

[The.Yield]

Happens with purely progressive material (I tend to avoid all things interlaced).

Basically I find the best source I can find, be it a 1080p apple trailer or a 720p/1080p rip from a HD DVD, BluRay...

And then run that video through a single/series of highly tuned passes with the lovely MediaCoder software (great stuff btw). This results in the final product.

But I've gotten this aliased 255ish color business from many tools I've tried over the years. Seems like if you're slamming video down onto a portable of some kind... It's gonna' have aliased super-bright colors, lol.

I can't be the only one who has ever seen this! I've personally seen the same artifact on Ipods to PSPs and everything in between. Video's run through god-knows-what software common folk use to butcher there transcodings.

I'm doing some marathon setting change runs now to see what I can find out. I haven't done any testing on this issue in a very long time; and I forget the particulars (Does the artifact appear on a PC's screen when playing the resulting video, or only on the device... That kinda stuff I forget)

[Squash]

Don't know if there's a connection or not, but your post reminded me of this thread that went up a few weeks ago:
https://forum.videohelp.com/topic341545.html

[guns1inger]

Many portables have a limited colour palette - often 64000 colours and sometimes less. This leads to banding, regardless of the quality of the source. Banding also comes from low bitrates (another curse of portables). Until portables have full colour ranges and can handle higher bitrates, you will never get great quality.

[thecoalman]

Porobably due to scaling, either the software is doing a poor job if you're doing it there or the player is. There's different methods for scaling video/images and one that simply removes every X line will produce stair stepping.

----------

Having said that I just purchased my mother one of those digital picture frames for Christmas and it produces slight stair stepping but it's not something your average person would notice. Not sure if its due to scaling or the screen itself.

[The.Yield]

Here is an example taken from a final quality PSP conversion from a 1080p HD DVD rip.

You can see that the aliasing is present even when viewed on a PC monitor. Namely in the 'W' but some of the other letters show it as well.

I'm going to do a whole bunch of different resizing methods but, this particular one is done with the Area algorithm. Which yields awesome results across the board usually, you can tell how sharp the boy on the moon and the cloud line are razor sharp as can be had from 480x268 downsizing. Also, the Area Algorithm gives by far the best results on large areas of flat color such as sky and most parts of Japanese animation. Virtually no stair stepping of color panes at all (Though Sincr algorithm is still the king of sharpness in my book).

And since this is present even on a PC monitor I'd say it isn't the color limits of the portables that is causing the issue.

[jagabo]

Oversharp video and YV12 (4:2:0 subsampling) encoding. The same problem Squash pointed out. Sharper isn't always better.

[The.Yield]

Oversharp video and YV12 (4:2:0 subsampling) encoding. The same problem Squash pointed out. Sharper isn't always better.

Just had a chance to go over that thread and indeed, seems like the case. Sadly I have no idea how to control the color space MediaCoder uses =/

I'll have to do some research, as I don't really wish to change tools (I <3 MediaCoder)

[jagabo]

I have no idea how to control the color space MediaCoder uses =/

Even if you can force 4:2:2 or 4:4:4 encoding I doubt many portables would play it.

[The.Yield]

Well the Cowon A3 has an 800x480 screen @ 16.7mil color. Running a TI DaVinci chip...

[guns1inger]

If MediaCoder is the cause (and if it's the only converter you have used then it would seem to be the problem) then you have two possible choices.

1. Change converters. Trial a few others until you find one that gives you the results you want.

2. Create your own avisynth scripts, then use MediaCoder for encoding only. You can then tweak and test the scripts until the video looks right before encoding.

[The.Yield]

Ya, MediaCoder is just a GUI. It's really up to the codec I choose that will determine whether or not I can use 4:2:2/4:4:4 color space.

I know x264 doesn't support it ala HP... But it does support "lossless" encoding which may help the problem (gotta' run some tests).

Xvid has a color pre-filter that is supposed to help with this very problem (found that one out in my research).

I need to do smore' digging to see what codec support the use of these colorspaces.

[jagabo]

Ya, MediaCoder is just a GUI. It's really up to the codec I choose that will determine whether or not I can use 4:2:2/4:4:4 color space.

This isn't entirely true. If the resizing is in RGB and RGB is sent to the encoder, then, yes, the chroma subsampling will be determined by the encoder. But many programs will resize in YUV and feed either YUY2 (or similar 4:2:2 format) or YV12. In this case, the damage is done before the encoder gets it. 4:2:2 won't completely eliminate the problem but you may find it acceptable. Also keep in mind, half the problem is caused by the player software -- as can be seen in the image in this post:

https://forum.videohelp.com/topic341545.html#1784363

[The.Yield]

Interesting stuff. Never thought that the resizing messed with the media's colorspace... I might have to ask on the MediaCoder forums about what exactly there software does to the source material before the codec gets it's turn.

I've also found that this aliasing problem goes away more and more with increased resolution. 480x272 on Sony PSP material shows more 255ish aliasing then 800x480 Cowon A3 material. Just as one would assume.

Also I've found that the downsizing algorithm makes a smidge' of difference as well (which I think was mentioned earlier by someone). Even at similar "sharpness levels", one can yield less 255ish aliasing than another.

[jagabo]

Never thought that the resizing messed with the media's colorspace...

Many video editing programs do all their filtering in RGB. TMPGEnc and VirtualDub for example. So an incoming YV12 frame will be converted to RGB, filtered, and then sent to the compressor as RGB, or converted back to YV12 and sent to the compressor -- depending on what the compressor accepts as input.

I've also found that this aliasing problem goes away more and more with increased resolution.

In terms of playback, since you are seeing the videos at the same size on the player, the defects in the small frame are magnified by upsizing the video.

In terms of what's in the file, downsizing can increase sharpness (just as upsizing can reduce sharpness). The sharper the color details the more the YV12 color artifacting will show up.

Also I've found that the downsizing algorithm makes a smidge' of difference as well

The sharper the resizing algorithm, the more the YV12 defects will show up. A simple blinear filter will give the least sharp results and the least YV12 artifacting. A Bicubic filter will be much sharper and display more YV12 defects.