It's impossible to get a 100% pristine, flawless digital video file, right?

[amiableakuma]

I'm hoping somebody will just say "yes" - so that I can get on with my life.

And what I mean by the topic is that - it's impossible to create a digital file that will display on a given screen with absolutely no picture quality errors of any kind. No strange little compression artifacts, no blocking/pixelization in dark areas, no imperfections in shadow coloring/fading, etc. One that will have a pristine/flawless picture and still retain every bit of it's pixel resolution sharpness/depth. - This cannot exist when we're compressing/converting DVDs/etc to new digital files - correct? I mean, at least not really with our current technology, right?

Because, moreover, what I'm specifically talking about is that I'm ripping my DVDs, converting them with AutoGK/Virtualdub/etc - and then using the "100% quality" output video file to play on my new "video ipod" w/a 4.3 inch screen, the Creative Zen Vision:W. It's relatively new tech., supports hundreds of thousands of colors and all that jazz, - but I shouldn't be expecting pristine HD quality from a digital file on a little portable device like this, right?

See, I am sure that I have always been tech-savy enough to do most everything to get excellent, near perfect file conversions from DVDs - but obviously when I display these files on a screen/s, there still remain *little* picture flaws like compression blocking in dark colors or something. Now, I AM 100% FINE WITH THIS! The files I am getting - though they still contain a few picture defects here and there - STILL LOOK GORGEOUS for the most part on my displays.

I just NEED somebody to say to me - once and for all - that what I'm seeing IS definitely as good as it gets for the current tech...AND that no amount of extra conversion tricks or buying of new superior equipment will really solve these more minor picture imperfections much at all.

Somebody please, just say that to me - so that I stop the sleepless nights and just enjoy the great entertainment I have. Or, if you've experienced otherwise - and actually gotten "the digital file that plays pristinely on even little portable players and other low to medium-end displays" - then please, tell me how you think you did it...

Thank you...

[SingSing]

If you play BMP files at 30 fps, then you able to get a perfect digital video.

The least compressed video is DV format.

But..

Human eyes are not pefect, Sear tested their original 270 paint color, and found people can only tell 112 apart.

Human also can't tell motion apart, and that's why video are played at 60Hz or less, and not 2000 frame per second.

Digital picture and video are both distored, because vison is continous as analog and not in pixels.

[Scorpion King]

You should be able to rip a hollywood dvd and convert it straight to uncompressed avi and get a 1:1 conversions (equal quality). The uncompressed movie file will be super huge, but it will retain the quality of the original with no loss. ie looseless. A conversion using any other lossless avi codec should give the same results also like Huffyuv, etc.

The problem lies when one encoded the already encoded file (the original hollywood dvd movie). Say the original started life as a very high quality professional DV file then was encoded to 9.8Mbps Mpeg2 DVD. The file has already taken a huge quality hit. Later you come along and reencode this to like 600Kbps Divx. You are going to take another huge quality hit.

The first encode to mpeg2 created flaws in the video (blocking, noise, etc). Now that these exist any attempt to reencode the video will also need to encode all those flaws. The flaws requires a huge amount of bits to retain because of their complexity. One must add bitrate to encode the flaws a video contains, besides the bits required to retain the actual video quality, or take a quality hit.

You can easily test this. Take a video file and reencode it using the exact same settings the file already has for bitrate, framesize, codec, etc. You will see the quality hit the file takes. Each encode introduces artifacts and once these artifacts are in the video the bitrate required to retain them takes a big jump over the bitrate setting the file already has.

I think that is where the word 'looseless' came from. Divx, Xvid, etc, are not looseless. Neither is mpeg2. lol

Or something like that....

Good luck.

[deadrats]

I'm hoping somebody will just say "yes" - so that I can get on with my life.

And what I mean by the topic is that - it's impossible to create a digital file that will display on a given screen with absolutely no picture quality errors of any kind. No strange little compression artifacts, no blocking/pixelization in dark areas, no imperfections in shadow coloring/fading, etc. One that will have a pristine/flawless picture and still retain every bit of it's pixel resolution sharpness/depth. - This cannot exist when we're compressing/converting DVDs/etc to new digital files - correct? I mean, at least not really with our current technology, right?

Because, moreover, what I'm specifically talking about is that I'm ripping my DVDs, converting them with AutoGK/Virtualdub/etc - and then using the "100% quality" output video file to play on my new "video ipod" w/a 4.3 inch screen, the Creative Zen Vision:W. It's relatively new tech., supports hundreds of thousands of colors and all that jazz, - but I shouldn't be expecting pristine HD quality from a digital file on a little portable device like this, right?

See, I am sure that I have always been tech-savy enough to do most everything to get excellent, near perfect file conversions from DVDs - but obviously when I display these files on a screen/s, there still remain *little* picture flaws like compression blocking in dark colors or something. Now, I AM 100% FINE WITH THIS! The files I am getting - though they still contain a few picture defects here and there - STILL LOOK GORGEOUS for the most part on my displays.

I just NEED somebody to say to me - once and for all - that what I'm seeing IS definitely as good as it gets for the current tech...AND that no amount of extra conversion tricks or buying of new superior equipment will really solve these more minor picture imperfections much at all.

Somebody please, just say that to me - so that I stop the sleepless nights and just enjoy the great entertainment I have. Or, if you've experienced otherwise - and actually gotten "the digital file that plays pristinely on even little portable players and other low to medium-end displays" - then please, tell me how you think you did it...

Thank you...

i know i'm going to get flamed for this but the answer to your question is "yes", it's mathematically impossible to get a 100% exact digital copy of anything shot on film. the reason for this is that real life can be thought of as an analogue signal, i.e. if modeled mathematically it would resemble a curved line (for illustration purposes, i'm keeping the line a simple parabola) and the math that describes the line would be a continuous function. movies and pictures shot on film likewise have the same characteristics.

digital images on the other hand, if modeled mathematically, would be described by discontinuous functions. conceptually it's similar to using linear algebra to find the area under a curve and it works on this principle:

assume you draw a line on a chalk board and analyse it. the line, from a mathematical standpoint, is made up of infinite points and looks and behaves as a continous object (think of a piece of string). now assume you want to save some chalk and you realize that if you were to draw the line on the board using a series of dots you could get the same general shape. if the dots are viewed from up close, you can clearly see that the line is discontinuous yet if viewed from 20 feet away it looks like a single continous line. you have now digitized the parabola. what you have done is taken a shape made up of infinite points and represented it with a finite set of points. the more points you use the closer you can stand to the black board without seeing the difference, but no matter how many dots you use you can never make an exact copy of the line drawn as a continous shape.

the same thing happens with video: the movie is shot on film and the film is an exact copy of the scene being shot. the film is then scanned and each frame of film is represented with a series of dots. now, with enough dots and thanks to the limitations of our eyes, it's possible make a copy that we can't visually differentiate from the original, but it is not a 100% exact copy.

that digital copy is then used for authoring dvd's and by the time you get it in VOB form, even more dots have been lost. that's why even the highest quality DVD doesn't look as good as what you see on a big screen (the fact that we watch our dvd's on tv's and monitors that are significantly smaller and of lower resloution than the original film helps with the illusion that the dvd is of high quality).

now you try and re-encode the movie to another format, and the only way to get the file smaller is by removing even more dots and thus it's impossible to ever create a 100% perfect copy so long as you follow the above pattern.

the only way for you to get an exact copy is to literally get an exact copy of the movie on the same quality film it was shot on and that's not going to happen.

so stop driving yourself crazy and be happy with what you got.

[guns1inger]

Even the big studios, with big budgets and top equipment can't get clean transfers free of the occasional (and not so occasional) artifact and fault.

[gadgetguy]

Also consider that the player has a part in the quality of what you see. I have seen videos that look exceptional when viewed on one device show blocking and other artifacts when viewed on another, and a frame by frame analysis shows that the video is fine.

[manono]

And he's converting to AVI (XviD or DivX), and viewing on an LCD screen. One of the major drawbacks of MPEG-4 compression is the so-called "black blocks" or "dark blocks". In dark areas you get what is sometimes called the "dancing blocks" or "moving blocks" phenomenon, where macroblocks of similar but not exact color to the background move all around. It's seen to a lesser degree in solid colors of all kinds (walls, skies, etc.). If you're compressing to a low resolution for your portable device, the blocks get kind of big. And if you're viewing on an LCD, it looks worse than it does on a CRT screen.

There are steps you can take to lessen the effects (adding noise and grain, for example), and although AutoGK has add-ons that allow you to edit the .avs, it's not really the best program to use to attempt such things.

And you can also lessen the efects by calibrating your screen, if it's possible.

[edDV]

...
I just NEED somebody to say to me - once and for all - that what I'm seeing IS definitely as good as it gets for the current tech...AND that no amount of extra conversion tricks or buying of new superior equipment will really solve these more minor picture imperfections much at all.

You are nowhere close to multiplex theater quality let alone as good as it gets.

I'm doing this to put your problems into perspective.

Big studios don't even expect to get prefect transfers from film. That is why they save the film if they think it has future value.

The old film gets pulled from the vault about every 10-15 years for a restoration and transfer to the latest format of the day. Then it goes back into the vault for temperature controlled storage.

Historical transfer formats have been

- 2" Quadruplex (mono then again for color)
- 1" type C composite (followed later by D2/D3 composite)
- D1 (720x480/576) 4:2:2 YCbCr uncomprssed (followed later by compressed Digital Betacam)
- HDCAM 1080i 3:1:1
- 2k (1920x1080 uncompressed 4:4:4 transfer)
- 4k (~4000x2000 uncompressed 4:4:4 transfer)
This is about it for historical film stock

Now there is talk of 8kx4k for new film technology. Production HDTV acquisition is still at HDCAM SR level (4:4:4 1920x1080p/24 440-880 Mb/s)

[InXess]

The source you are using is not perfect to begin with. Then there are tools you use, their quality ("you get what you pay for", sounds familiar?), the process you apply, hardware and software you use, your display quality and finally your knowledge.
Without the knowledge how to use them even best tools won't give you desired results. Now keep in mind that "100% quality" only means 100% (LOL, never happens) of what this or another tool may be capable of or is designed for. 100% of Corolla speed is not equal to Ferrari's 100%. Don't take it at face value. Lastly, AutoGK has never been intended to be a real Pro tool.

[AlanHK]

with no loss. ie looseless....
I think that is where the word 'looseless' came from. Divx, Xvid, etc, are not looseless. Neither is mpeg2. lol.

LOSSLESS.

[thecoalman]

The best it is going to be is the original file, minor filtering aside everthing after that and it goes downhill. Even going from DV to a high bitrate MPEG introduces artifacts. The following example would be hard to notice unless you were looking for it even on a very large screen. DV on the left, 8000CBR Mpeg2 on the right.

[edDV]

To see what you are missing, check out these Digital Cinema 4k samples.
http://red.com/gallery-still.htm

[Cornucopia]

deadrats,

what's really strange is that the "original" film master isn't a TRUE analog of reality--it's sort of "digitized" too.

Real TIME and MOTION is continuous. Motion on is film is from discreet points of sampled time (24fps sound familiar?).
The frame also LOOKS like it's analog, but on a near-microscopic level it isn't. Film uses GRAINS of photosensitive salts sitting on the emulsion layer(s). Once you get down to that level (~2k - 8k depending upon film stock used), you'll see the grains and you can't divide a grain with 2 different images, you have an average. Thus, it's acting just like a Videocam CCD imager's pixel (just not as SQUARE, nor as BIG).
That's why, when do processing/restoration/DI work, the studios can use tools that save/remove/restore the grain. They're digitizing at (sometimes) an even smaller/finer level than the original film.
Thankfully, at least the "light" and "color" values assigned seem to be continuous.

So how does that pertain to the original topic?
...I dunno...just sounds cool.

No, really, you'll have to come to some understanding about all the elements that go into arriving at a digital file, and how it can be "compromised" already, and how it could be further compromised if you aren't careful. There are ways to convert and (re-)compress that will give you at least an acceptable output for your desired final media/display. The main points to stress are:
1. Try to resize down (not up) if you can help it
2. Compress as little as possible and as late as possible
3. Same thing with colorspace, size, and framerate conversion.
4. Once you lose quality, you can't really gain it back (not counting #5)
5. Since the HumanVisualSystem isn't perfect, you can sometimes cheat

Scott

[Cornucopia]

BTW, if you do 3D modelling/rendering/animation, you CAN render out a video file that might be called "perfect" (aka >24bit, RGB, Uncompressed, 60p, 4k). Gonna need a lot of space, though.

Scott

[lordsmurf]

There are ways to convert and (re-)compress that will give you at least an acceptable output for your desired final media/display

Yes. 8)

[Wile_E]

If you play BMP files at 30 fps, then you able to get a perfect digital video.

The least compressed video is DV format.

But..

Human eyes are not pefect, Sear tested their original 270 paint color, and found people can only tell 112 apart.

Human also can't tell motion apart, and that's why video are played at 60Hz or less, and not 2000 frame per second.

Digital picture and video are both distored, because vison is continous as analog and not in pixels.

There are many people that would disagree with your statement about Human eyes can't tell a difference from faster rates or refresh rates. We certainly don't need 2000 frames, but 48fps would be nice. In the silent era, film rates were 16fps and 18fps. Now with audio we have 24fps. They could've made 30fps or higher the standard. Two reasons they chose 24fps, was less film was needed, and it was the slowest frame rate that was semi-tolerated by human eyes to represent motion. Back then, 24fps looked fluid compared to 16 and 18fps. But many would argue that 24fps is not enough to capture fast action and life-like motion. Several companies have been trying to persuade Hollywood to go 48fps. Even with digital, hollywood is shooting a meager 24fps which i think is ridiculous. Something needs to be done about this slow frame rate.

[guns1inger]

A lot of projectors play back at 48 fps by duplicating each frame. While not adding to the motion, it does reduce flicker which helps with the illusion, and reduces fatigue in the viewer.

I would also challenge the statement about DV being one of the least compressed. It is compressed at a ratio of 5:1, which is pretty high. There are plenty of lossless codecs available now that compress reasonably well and preserve your image quality exactly. With HDD space being comparatively cheap to a few years ago, lossless codecs are no longer as impractical as they first were (although as an archiving format they are still way to big).

Also, the amount of compression is related to the method of compression, or codec in use. There are plenty of codecs capable of giving better image quality than DV at much smaller file sizes if used correctly. These tend to use temporal compression, and aren't well suited to editing or repeated compressions, but for final viewing will give you equal quality at smaller sizes for the same source.

[ZAPPER]

Just to stir the pot a little.

@amiableakuma

You are posting three different parameters

First in the title you say "pristine, flawless file" I would say yes you can get a perfect "file"

Then you want that file to play on a display perfectly in the opening statment of the post. I say no, just because once something is set in motion too many variables come into play to have something static turned dynamic retain the static perfection.

Then you are talking about conversions. Even if the new format was mathmatically perfect and all that, it is by nature no longer a pristine or equal version. It is a lesser version.

What we are shooting for is a level of accecptable quality from source to final display. Some folks would blow a gasket if there was a single white pixle that flickered on in a black scene in one frame of a two hour movie, while others are happy just to be able to identify the blobs of poor color and shapes on the screen as actors in a movie.

Sit back, relax, take a breath and enjoy the damn movie. While the pursuit of perfection is a noble cause, don't loose touch with the fact that this is all about entertainment. I can sit back and watch an old Black and White "Three Stooges" and never even consider the fact that it is Black and White. The thing is not that it is a trick of the light but rather a trick on the mind. You will almost always see what you want to see wether it is there or not.



Blah blah blah blah

Next!

[thecoalman]

While the pursuit of perfection is a noble cause, don't loose touch with the fact that this is all about entertainment.

The exception to that is if you're working with original material where creating the "perfect file" is preferable especially if it's for archival purposes. The three stooges movie can be replaced down the road with the latest and greatest copy, home movies can't.

[Scorpion King]

LOSSLESS.

[ozymango]

To see what you are missing, check out these Digital Cinema 4k samples.
http://red.com/gallery-still.htm

Wow!!!

Although the fact that I can make out the tiny mustache hairs on the actress' face may not exactly make her all that happy. :P

[Marvingj]

Very Interesting.........

[InXess]

This is most impressive. So where's the limit...? not many still cameras can match that. Just wonder who will release the first movie shot with RED...

[JohnnyMalaria]

This is most impressive. So where's the limit...? not many still cameras can match that. Just wonder who will release the first movie shot with RED...

Does RED even exist?

Look at their website - all the equipment images are CAD renderings. Not one single photograph of the beast. They even say that the photos were obtained with the RED sensor - not a complete camera.

A lot of people have pre-ordered. I hope they get their cameras soon. Latest estimates are April....

[JohnnyMalaria]

the reason for this is that real life can be thought of as an analogue signal

The analog(ue) appearance of our world is an illusion.

If you measure things on a very small scale (both in terms of space and time), then you would actually start to see the world in a very different way. The continuous notions of distances and time reveal themselves to be quantized - i.e., more akin to a digital world than an analogue one.

That's quantum mechanics. It describes a bizarre world where we can't even measure anything reliably. If you try to measure something that exists for a very short time, you can measure that time but you can't measure how much energy that thing has. Equally, if you try to measure its energy (e.g., the wavelength of light), you can't measure how long it lasts for!

And this ain't no physicists' weird stuff. The bizarre goings on impact our everyday world. When you pulse a laser, as the pulse gets shorter, the wavelength of the laser becomes uncertain. There are some lasers that are pulsed so fast that they appear white.

So, in principle, you could digitize the world - but there is inherent uncertainty in what you end up with.

[edDV]

This is most impressive. So where's the limit...? not many still cameras can match that. Just wonder who will release the first movie shot with RED...

There is a fair amount of 4k film transfer going on. More to come.

I linked the red site because they had some good 4k samples.

[ozymango]

The analog(ue) appearance of our world is an illusion.

Hey! No religion or politics in the forums, man!



And whatever happened to those cameras that allow you to see the aura around people?

[deadrats]

the reason for this is that real life can be thought of as an analogue signal

The analog(ue) appearance of our world is an illusion.

If you measure things on a very small scale (both in terms of space and time), then you would actually start to see the world in a very different way. The continuous notions of distances and time reveal themselves to be quantized - i.e., more akin to a digital world than an analogue one.

That's quantum mechanics. It describes a bizarre world where we can't even measure anything reliably. If you try to measure something that exists for a very short time, you can measure that time but you can't measure how much energy that thing has. Equally, if you try to measure its energy (e.g., the wavelength of light), you can't measure how long it lasts for!

And this ain't no physicists' weird stuff. The bizarre goings on impact our everyday world. When you pulse a laser, as the pulse gets shorter, the wavelength of the laser becomes uncertain. There are some lasers that are pulsed so fast that they appear white.

So, in principle, you could digitize the world - but there is inherent uncertainty in what you end up with.

not entirely true (i have a rather strong physics background). if you wish to invoke quantum mechanics you need to also address the particle/wave duality property of all particles, depending on how they are viewed, particles can be described as discrete states (i.e. in a digital sense) or as waves (i.e. in an analogue fashion).

furthermore, if we turn our attention to forces, we see that they follow an analogue pattern, i.e. the functions that describe them are continous.

quantum mechanics offers some interesting theories, but i have yet to find anyone (and i used to pester my physics professors constantly) that can explain to me why it is that forces are described by continous functions if they are caused by and effect particles that behave as descrete points, in other words, why should force(s) be a continous commodity and not behave in a fashion described by discontinous functions.

make no mistake about it, real life is analogue in nature, gravity, light, forces, energy, matter, they are all wave like in nature, and it's only when you look at them in frozen moments of time that they can be thought of as digital.

conceptually it's as if you had a curved line that represented real life and took the 1st derivative and saw that it corresponded to a single point on the line and thus concluded that life in general must be "digital" in nature.

basically it's a mis-interpretation of what the math is telling us.

[deadrats]

And whatever happened to those cameras that allow you to see the aura around people?

the "aura" is a real phenomena (you can see it under certain conditions by looking at your hand's reflection as it's held close to a highly polished mirror).

what it actually is makes for an interesting discussion, but it's my feeling that it is either thermal energy our bodies produce and give off, light being bent by the bio-electric fields our bodies generate or possibly a limitation of our eyes ability to focus on two dis-similar surfaces simultaneously.

[Cornucopia]

the "aura" is a real phenomena (you can see it under certain conditions by looking at your hand's reflection as it's held close to a highly polished mirror)...

Are you sure you aren't talking about the double reflection/diffraction produced by a standard back-silvered mirror? This would go away if you just got a front-silvered mirror.

(man, is this so far OT!)

Scott