How exactly does upconversion work?

[Acid_Reign]

I'm a bit curious as to how the whole "upconversion" process works in hardware devices—particularly DVD players that advertise near-HD derivations of SD source material. I know the basics (deinterlace and/or resize) but I want to know the technical specifics. I don't understand how a pixel-based image can be blown up and gain quality; it sounds too much like a marketing gimmick to be completely true. If anyone can explain this, or provide a link to a definitive article, I'd be most appreciative.

[manono]

Hi-

Here's some information:

In fact some, by no means all but some, of these new generation "up-scaling" DVD players produce a significantly better image than the previous generation of "progressive scan" players. Why this is so is due to several factors. First the scaler in the player may be better than the scaler in the TV. The closer the player can get the data to the native resolution of the TV the less work the TV's scaler has to do. Second, engineering continues to advance. Other factors than scaling are likely to be better in a good "up-scaling" player than in the previous generation players. But the most important reasons why folks get good results from some of these new players is that the data stays entirely in the digital domain...The bottom line is that despite the best efforts of the marketing guys to pull a fast one here, many of the better up-scaling players DO INDEED produce a significantly better image on many HDTV-ready TVs. Some of that is due to the digital connection, but some is also due to the combination of de-interlacing and scaling technologies working well to produce a signal the TV happens to be optimized to display. Combine that with other improvements naturally occurring with each product cycle and you get a better player.

http://www.avsforum.com/avs-vb/showthread.php?t=477740

How it's done is also explained, if you read the entire very long post.

[SingSing]

Upscaling involving adding pixels, the process after that is filtering that soften the picture, and sharpening that add artifact, then noise reduction to remove some of the noise. Go to bestbuy and starred at the upscaled movie, and you will see these for yourself.

Upscaling DVD player not always necessary do a better job that HDTV. Some of the upscaling Samsung DVD player can make your DVD look like VCD on HDTV.

[jagabo]

http://mathworld.wolfram.com/BicubicSpline.html
http://www.cs.uic.edu/~kenyon/Papers/Comparison.of.Interpolating.Methods.Parker.Kenyon.Troxel.pdf

[yoda313]

So would upscaling in effect be the opposite of compression? Since the basics of compression is removing similar colors from a picture to reduce the overall size of the video is this the opposite function then? Does upscaling use a reverse process to fill in the missing detail? Is that what upscaling does?

[jagabo]

So would upscaling in effect be the opposite of compression? Since the basics of compression is removing similar colors from a picture to reduce the overall size of the video is this the opposite function then?

No, upscaling is the opposite of downscaling. Compression leaves the frame the same size but looks to remove redundancy (both spacially and temporally) to make a smaller file.

Does upscaling use a reverse process to fill in the missing detail? Is that what upscaling does?

That's what it attempts to do. The reality is that upscalers can't really know what was removed from an image that has been downscaled. About the best they do is enlarge the frames and apply a little sharpening.

Say I have a high resolution image. I downscale this to a single pixel. How could you restore the original image from that single pixel? Of course you can't. Fortunately most video has not been downscaled so drastically! You can use pixels from the downscaled image to make some predictions of what was lost. The result won't be exactly like the original high resolution image though.

[Alan69]

So would upscaling in effect be the opposite of compression? Since the basics of compression is removing similar colors from a picture to reduce the overall size of the video is this the opposite function then?

No, upscaling is the opposite of downscaling. Compression leaves the frame the same size but looks to remove redundancy (both spacially and temporally) to make a smaller file.

?? I think he meant in a general sense, and yes it's quite similar. In upconversion, you're making the picture higher resolution, then adding in info that isn't really there in the smaller file. With decompressing a file you're making the file bigger, then adding back in info that also isn't really there in the compressed file.

Bias of what your eyes see is what makes it able to 'improve' the picture. Your eyes see edges extremely well, other detail less so. Straight edges in a lower resolution tend to be straight edges in a higher resolution too, so you can add in the 'missing' detail and make them look better. So if the upscaling is tweaked to make edges look good without doing much harm to other detail, it'll appear as an overall improvement in the picture. Just think about it from usage end, you already know you notice jagged edges pretty quick. Usually would take you several more times viewing the same amount of degradation in any other aspect of video. Few pixels out of place on an edge and it looks terrible, same percentage of randomly or even grouped changes elsewhere in video will hardly get noticed.

Edge detection was very important to spot both predators and prey in our past, your whole vision system is heavily geared towards it for object detection against similar backgrounds.

[edDV]

The upscale process for progressive to progressive relies on interpolation of missing pixels in H and V. Different strategies can be used to "guess" what info belongs using reference from an x y rectangle of surrounding pixels and similar pixels from past frames and if delay can be tolerated, from "future" frames. These guesses can then be applied to additional image processing to derive statistical "detail". The success of all this depends on a quality low noise source. If you add noise or compression artifacts to the source (e.g. home recording vs. commercially mastered DVD) the guessing can get bizarre and the picture quality goes down.

Interlace and telecine video processing is more complicated.

Bottom line there is a competition between the upscale processor built into your HDTV and the processor in the DVD player (or HD cable/sat box). In the case of the DVD processor, it usually only knows how to upscale to 1280x720p or 1920x1080i. The HDTV processor then needs to finish the job to match native display resolution (typ. 1024x768, 1366x768 or 1920x1080 in today's HDTVs). In the case of 1080i inputs the TV still needs to do the IVTC or deinterlace before scaling.

Bottom bottom line, test 480i, 480p, 720p and 1080i DVD player outputs and choose the one that works best with your particular HDTV. Evaluate with both progressive and interlace DVDs.

If the HDTV processor is superior, the 480i/480p outputs get best results.

[Marvingj]

If a scaler is converting a lower resolution to a higher one, the process is called upconversion. The opposite process is called, unsurprisingly, downconversion.

The main job of a video-processing system is to up- or downconvert an incoming signal to a different resolution. This process for a scaler working with your HDTV has two main benefits:

* Video signals are matched to the best display resolution for a particular model of HDTV, regardless of the signal's original resolution.
Standard-definition video signals are upconverted to a higher resolution to look closer to HDTV than standard-definition. Don't buy anyone's marketing spiel or sales pitch telling you that his or her scaler can make any TV source into HDTV. Good scalers can produce something very pleasing to the eye, which looks close to HDTV. But you can't create something from nothing - real HDTV signals can have six or more times as many pixels as standard-definition. In other words, in an upconverted video stream, five out of every six pixels could be "made up" by the scaler. Even the best scaler can't create something as good as the original recording! The simplest video processors - which actually pre-date HDTV - are those devices known as line doublers. High-end home theaters have used line doublers for years, in conjunction with fancy (and expensive) front-projection systems.

You may also hear line doublers sometimes called deinterlacers.

The job of a line doubler is pretty simple - it scales the video by converting an analog (480i) video signal into a progressive scan (480p). By doing this simple trick (effectively doubling the scan lines in a CRT TV - hence the name), a line doubler greatly smoothes out the picture and reduces the subtle flicker that you normally get from an interlaced picture. You also hear about devices call line quadruplers, which not only deinterlace video, but also interpolate in between the lines to create a picture that's the equivalent of 960p (twice the resolution of 480p). Line quadruplers are really high-end devices for use with the most expensive CRT front-projection HDTVs. Fixed-pixel HDTV displays - plasmas, LCDs, DLPs, and LCoS - have created a need for something beyond just a simple line doubler (not that a line doubler is all that simple!). Many fixed-pixel displays actually have rather funky native resolutions that require all incoming video signals to be scaled.

This demand has led to the development of scalers that convert any incoming video signal (well, any standard incoming video signal) into a specified output resolution. If your HDTV needs a scaler to operate (and most need at least a deinterlacer/line doubler), then it already has one built in.

Internal scalers do have some limitations, however. Sometimes an external scaler offers advantages. Here are three scenarios that may call for an external scaler:

* Some HDTVs have internal scalers that accept signals of only certain resolutions. For example, some HDTVs may accept only 480i, 480p, and 1080i inputs. If your external HDTV tuner puts out only a 720p signal (admittedly rare), you're out of luck.

* Some HDTVs (CRT tube-based TVs, either direct-view or projection systems, usually) contain an internal scanner that deinterlaces analog video sources (meaning it converts 480i to 480p) but doesn't upconvert analog signals to a higher resolution. There's nothing wrong with this lack of upconversion - 480p probably looks better than analog's ever looked! - but the picture doesn't use the full capacity of your HDTV.

* You may just be a person who wants the best! The scalers built into most HDTVs are good enough for most owners. But, if you demand those last few percentage points of picture perfection from your system, you may want a fancier external scaler.

[edDV]

...
* Some HDTVs (CRT tube-based TVs, either direct-view or projection systems, usually) contain an internal scanner that deinterlaces analog video sources (meaning it converts 480i to 480p) but doesn't upconvert analog signals to a higher resolution. There's nothing wrong with this lack of upconversion - 480p probably looks better than analog's ever looked! - but the picture doesn't use the full capacity of your HDTV.

While there are some CRT HDTV sets that can display 1280x720p and even a few very high end CRTS that can reach 1440x1080, most consumer CRT sets have progressive scan capabilities maxing around 540p (half 1080) to 576p. Such sets are limited to 1080i input for HDTV where sequential odd/even fields of 540 lines are scanned. Such sets aren't capable of displaying 720p inputs unless a 720p->540p downscaler is used.

Such TV sets usually display progressive DVD at native 480 or 576 lines. This is actually a very good thing for 4:3 aspect ratio HDTV sets when displaying 16:9 material. The 480 lines displayed are usually closely matched to the optimal dot pitch characteristic of the tube. In other words 16:9 480p is all the tube can deliver. Wide HD Ready CRT sets also display native 480p although the CRT may be capable of a bit higher scan (e.g. 540p).

480i inputs usually go through an internal upscale to 1080 lines (540 lines per field). If you feed the set 1080i from an external scaler, you get the same 540 lines per field displayed. CRT sets in this class that have "Cinema" (aka IVTC) processing remove telecine but also scale to ~540p and frame repeat 3:2 to 59.94Hz refresh.

Fixed pixel flat panel displays don't have this flexibility for 480i/1080i. All input resolutions must be scaled to the panels fixed pixel native resolution. All 480i/1080i must first be deinterlaced or inverse telecined to progressive before scaling.

...
* You may just be a person who wants the best! The scalers built into most HDTVs are good enough for most owners. But, if you demand those last few percentage points of picture perfection from your system, you may want a fancier external scaler.

Most flat panel HDTV sets only accept 480i/480p/576i/576p/720p/1080i and maybe 1080p over HDMI or analog component. Most flat panels have native resolutions that differ from those choices. An external scaler can only output 720p or 1080i (sometimes 1080p). So the HDTV still needs to scale again to native resolution (e.g. 1024x768, 1366x768).

One exception that may or may not deliver superior results, many 1366x768 LCD-TV sets allow 1366x768 resolution input on the VGA "computer/game" port. This puts all the inverse telecine, deinterlace and scaling load on the computer's display card or the game system scaler. Mileage will vary.

[SingSing]

The video has to be decompressed before it can be upscaled.

Bspline is used to do second or higher order curve fitting.

Interoplation is used to fill in intermidiate points when changing a signal to double or higher order, in frequency.

Bspline and interpolation are based on the same basis idea : assume the world is make of sine wave.

[TBoneit]

Correct me if I'm wrong, But if I hook up My cheapo Philips via hdmi and I upscale with it. Will the resulting image from a WS movie now fill the screen from side to side vs being in a 4:3 image on a 16:9 screen? Plus a 4:3 movie will still be 4:3 ?

To do this would require me to move things and swap my HD DVR to component and the Philips to HDMI, Only one HDMI on the TV.

It seems to me that as I have a 32" LCD the component input should most likely look the same for TV? And if it will allow WS movies to play full screen then it will be worth the work.

If this will allow me to fill the screen from DVDs then it will be worth it. My other option is Computer, a WinXp MCE 2005 to VGA to watch. I already do this but to be honest I prefer using a DVD player. I mainly use the TV on the computer as a monitor and not to watch video. It seems that it displays dark for video. And I could adjust the VGA input brightness for video and then it would be to bright for normal computer use.

So bottom line is if upscaling will fill the screen with same quality video it will be worth it for me and the main benfit of uscaling as I see it.

Cheers

[jagabo]

Correct me if I'm wrong, But if I hook up My cheapo Philips via hdmi and I upscale with it. Will the resulting image from a WS movie now fill the screen from side to side vs being in a 4:3 image on a 16:9 screen? Plus a 4:3 movie will still be 4:3 ?

Those are aspect ratio issues, not upscaling issues.

[TBoneit]

Well sort of I guess. The issue is that the Philips only outputs 4:3 via component cable, 480p I'm guessing since I have set tp progressive in setup. Hopefully upscaling via hdmi would let it output 16:9 at 1080i(p)? Thus filling the screen. If that is how it works then it would be worth the work of rewiring / buying a hdmi switch.

[edDV]

Well sort of I guess. The issue is that the Philips only outputs 4:3 via component cable, 480p I'm guessing since I have set tp progressive in setup. Hopefully upscaling via hdmi would let it output 16:9 at 1080i(p)? Thus filling the screen. If that is how it works then it would be worth the work of rewiring / buying a hdmi switch.

Screen filling has all to do with the way the DVD is authored (flagged 16:9 wide), the material on the DVD (2.35 to 1 aspect will still have letterbox), the DVD player settings (should be set to 16:9 wide, progressive) and the HDTV aspect ratio controls. If the movie is really 16:9 and all the settings are correct, then the HDTV will display a full image for 720x480p out.

This all has nothing to do with the "upscaling" discussion above.

[TBoneit]

Ok, Then I've missed the point of upscaling? I could have sworn I tried what you suggest 16:9 and it still had bars on both sides. Maybe I need to try again

Upscaling is to take 480i or 480p and upconvert it to 720p or 180i(p) which would also mean filling the screen. I'm not sure how much better it could make the picture look. I guess I'll have to either pull everything out and rewire or get anothe 2 hdmi cables and a switch to see. I'm happy with the way DVDs look. I just really want the WS DVDs to fill the screen and not be postagestamped(?) in a 4:3 box on the screen.

For WS Sd I can zoom from the HD DVR to fill the screen and it looks ok to these tired ole eyes at the end of the day

Sorry to have drifted off topic. I misunderstood what upscaling was.

[manono]

Hi-

Upscaling is to take 480i or 480p and upconvert it to 720p or 180i(p) which would also mean filling the screen.

It should fill the screen in any event, whether the player does the upscaling or the TV does it. As they said, yours is a different issue. Maybe the player isn't set for 16:9 output, the TV is somehow set up wrongly, or the DVD is widescreen 4:3 (non-anamorphic).

[jman98]

Ok, Then I've missed the point of upscaling? I could have sworn I tried what you suggest 16:9 and it still had bars on both sides. Maybe I need to try again

Upscaling is to take 480i or 480p and upconvert it to 720p or 180i(p) which would also mean filling the screen. I'm not sure how much better it could make the picture look. I guess I'll have to either pull everything out and rewire or get anothe 2 hdmi cables and a switch to see. I'm happy with the way DVDs look. I just really want the WS DVDs to fill the screen and not be postagestamped(?) in a 4:3 box on the screen.

For WS Sd I can zoom from the HD DVR to fill the screen and it looks ok to these tired ole eyes at the end of the day

Sorry to have drifted off topic. I misunderstood what upscaling was.

There are only 2 resolutions for DVDs - 4:3 and 16:9. Standard definition displays will ALWAYS have black bars for anything encoded at 16:9. 16:9 can be several different resolutions - 16:9, 2.35:1 (or something similar if I'm slightly off on this one) are 2 I know about. If you do the math, you will see that 2.35:1 does NOT equal 16:9 and 2.35:1 movies will ALWAYS have black bars at the top and bottom, even on 16:9 displays. However, movies with true 16:9 aspect ratios will completely fill up the screen on a 16:9 display.

If you just want to see everything in 16:9, you need to set your HDTV to display in this format. I don't recommend it because it will make standard definition TV, VCD and VHS all look like crap when they stretch, but some people get obsessed with having EVERYTHING in 16:9 or they feel cheated. My advice is that you get much much better results just switching your TV between 4:3 for stuff like VHS, VCD, standard definition TV and 4:3 DVDs and 16:9 for the true widescreen stuff, but to each his own.

[edDV]

Ok, Then I've missed the point of upscaling? I could have sworn I tried what you suggest 16:9 and it still had bars on both sides. Maybe I need to try again

Maybe you should start a new topic and identify your DVD player and TV by model number. Yours is an aspect ratio and settings issue and somewhat off topic here.

[TBoneit]

I'll have to play around and that seems the most likely thing to do. Back on this in a couple of days with a new topic. Now where did I put that anamorphic DVD?

[thisistrevor]

Ok, so this is again slightly off topic and I apologize for that, but you seem to be getting this anyway..

My issue is with stretching. My xbox will play dvds on my HDTV but wether their widescreen or not, it will invariably stretch the image to fill the screen. I don't mind the bars at the sids, or the top and bottom nearly as much as I mind when everybody's faces look all extra wide and all the type looks all extra wide. I would much rather have a player that places the bars on the side if the dvd is 4:3, fill the screen if its 16:9, or place whatever bars need to be placed wherever they need to be placed to preserve the image in its unstretched state.

Do the dvd upconverters do this? This would be the only reason I have for buying one and I want to know before I spend the money that it will work. If not, is there a box that I can by that will do what I want?

[manono]

Sounds like you have either the X-Box or the TV set up wrong. The X-Box should be set up for output to a 16:9 TV set. I don't know about the X-Box, but most DVD players by default are set up to output for 4:3 televisions. I don't own an X-Box, so maybe someone else can give you the specifics of how to set it up correctly if you can't figure it out yourself. And there's always the chance you have the TV set up wrong. But if you get the correct aspect ratio with TV programs (4:3 programs play with pillar bars), then it's probably not the problem.

[thisistrevor]

The interesting thing is, is that through the netflix and zune streaming services on the xbox, everything plays fine. it sets up the pillar bars for 4:3, which I like, and plays HD content to fill the screen. All video games run fine. The xbox knows its running to a 16:9 1080p screen. It just chooses to stretch when it plays dvds. I think its a preferential thing. Where, some people would rather have it stretch if it means filling the screen. But this stretching bugs the $#@% out of me. Theres no preferences or settings on the xbox that have anything to do with paying dvds.

I guess I'm just wondering to hear from people who own an upconverter, if their dvds play with the bars or if it stretches.

[manono]

I think its a preferential thing.

I don't. There's only a right way and a wrong way, and stretching a 1.33:1 video to fill a 1.78:1 screen is wrong. It's either a defect in the X-Box or, again, you've set it up wrong in some way. Maybe an X-Box owner will see this and chime in.

I guess I'm just wondering to hear from people who own an upconverter, if their dvds play with the bars or if it stretches.

I do, an Oppo DVD player, and of course I have it play everything in the correct aspect ratio - nothing is stretched and pillar bars are added to 1.33:1 videos on DVD and output to a widescreen TV set. There are a few DVD players that also stretch 1.33:1 material to fill a widescreen TV, but they're defective and obsolete now.