Comparing results of encoding

[nlec]

I just re-read the sticky about PQ and feature analysis. Is the method of frame-grabbing with virtualdubmod and comparing the pictures the preferred way to judge the quality of a DVD encoding? Are there any objective measurements for judging the actual moving picture? Or does it suffice to analyze still frames?

I ask because I just completed 4 burns to DVD of a home video. I transferred a 2-hour Video8 tape using a Sony Digital8 camcorder to DVD using 4 methods:

cam -> DVDR via S-Video
cam -> DVDR via Firewire
cam -> PC via Firewire -> DVD via MainConcept MPEG2 encoder (CBR)
cam -> PC via Firewire -> DVD via MainConcept MPEG2 encoder (1-pass VBR)

The direct DVDR captures I did in SP mode. Additionally there are 2 other tests I could do, CBR and VBR encoding of the video as captured by my VIVO nVidia card using Huffyuv. I'm putting that off due to (a) disk space shortage, and (b) four DVDs are already overwhelming. If I am satisfied with one of the results so far I won't bother trying the PC capture method.

I haven't looked at any of the burned DVDs yet, but I'm wondering if I should proceed by grabbing frames from each of the 4 discs and comparing them.

[trhouse]

Analyzing stills of stills with a reference source still can yield certain information such as how well a system can reproduce contrast, black level, resolution, etc. but it does not give complete answers when motion is involved. For example, a still of a still allows using virtualdubmod to select an I frame for comparison purposes to the reference since all the frames are of the same image ( P and B frames are interpolated frames).

It is difficult to make a motion recording where the same frame types ( I, P, and B ) match the same image so comparing the same image will likely mean the comparison will be between differents types of frames. Unfortunately, the max file size on this site is 2 MB which limits what can be done in terms of showing motion clips.

[edDV]

Except for levels issues most of the comparison difference results from motion or noise that is detected as motion. An encoder will compress inframe (I frame) and then use lack of motion to compress further over the GOP. If no "lack of motion" areas are found the encoder is forced to compress the I frames more to achieve desired average bitrate.

Best case for MPeg compression is a locked camera looking at an object in motion against a stationary low noise background. This permits dismissing the background except in the I frame thus limiting saved difference data in the P and B frames.

Worse case for MPeg compression would be a hand held camera (X, Y and rotational motion for all pixels). Add to that zooms or pans and you have a case where no "lack of motion" is detected. When that happens, the encoder must either move pixel blocks based on guesses at motion or return to the I frame and compress it further. All encoders fail at the extreme case. A good encoder will find "lack of motion" in extreme cases or be better than the competition at estimation of object movement.

Noisy sources (like VHS) can fool many encoders to detect motion where none exists. Noise discrimination is a primary encoder attribute if VHS source is being considered. Noise has greater effect in darker portions of the picture. Therefore a good test is detection of object motion under low light. Add camera shake to separate the champs.

This isn't discussed much but logic would say the "best" encoder for low noise pro cams with pro operators (e.g. camera handling skills, Steadicam or fluid head tripods) would be one with high motion sensitivity. Likewise, the "best" encoder for VHS is likely to be the one with greatest noise immunity.

[nlec]

Very informative, thanks. Even better, it actually makes sense so I think my understanding is increasing.

Taken with an old video8 camcorder and no tripod, most of the video is shaky, which is normal for home video I take it. There are minutes of footage where people are dancing at an outdoor wedding and the camera was sitting on a table pointed at the action. I'll focus initially on that part of the video.

There are also many minutes shot from a ferry boat on an overcast day. I must have been bracing against the railing because there is very little shaking. Not a lot of motion, except for passing ships (relatively slow), but the choppy water and light glinting off the waves could be interpreted as "noise".

I just read some other threads about camcorder output over Firewire and I think I may have reversed the recommended TBC and DNR settings. Looks like I need to buy a few more rewritable DVD blanks so I don't burn through all my DVD+R stock.

I suppose I should put different MPEG2 encodings of particular scenes on one DVD instead of burning the whole 2 hours to disc, each with its own encoder settings. OTOH if the result is even "just OK" I can give the copies to people I know would like them, so it's not a total waste.

I'm trying to determine a workflow for processing all my old tapes. Doing MPEG2 encoding on a PC is excruciatingly slow compared to going direct to the DVDR, so I will have to separate the old tapes into 2 categories. One set I will burn to DVD and be done with it. Another set I will archive the DV footage for future use. So far the job has been educational and fun. I want to set up a workflow that doesn't become boring and tedious. If it gets tedious I will hate these old tapes instead of loving them and wanting to preserve them for the future.

[edDV]

Here's more reading specific to Video8.
https://forum.videohelp.com/topic329312.html?highlight=video8

Your problem breaks down into three main issues.

1. Capture to the hard drive (see above).

2. Software filtering (optional).

3. MPeg2 encoding for DVD.

Consider #2 an advanced topic with pro/con.

For #3 the answer is to encode at high bitrates (e.g. >8000-9500 Kb/s) and CBR. Encoder differences mostly disappear at those rates. Avoiding VBR eliminates poor motion decisions in high noise. Use a quality encoder. They don't really cost more. Avoid crap budget encoders.

[vhelp]

Ahh peoples..

It is difficult to make a motion recording where the same frame types
( I, P, and B ) match the same image so comparing the same image will likely
mean the comparison will be between differents types of frames.

Exactly.

There is no way to set a dvd recorder unit to begin on an I-frame and at a given
(user designated) position. Once the dvd encodes a given video source, the final
I/P/B is defined according to the mpeg.

So, at best, you can try all four dvd recorder test and when completed, bring all
the dvd to the HDD via any popular dvd-rip tool, and find a scene you are interested
in and note that scene's (image) I-frame position. Then, do the same for the rest of
the dvd recordings.

-- Original DV/8MM Source --- Scene A1, at pos 1520
You like the image at pos 1520 on the VirtualDub-MPEG2 's time line. So now, you
want to analyize and compare all your dvd rips: A; B; C; and D; to it. You want to
find out which one closely resembles that image with the least amount of quality loss.
This can be in terms of Pixelations; Macro blocks; Color banding or bleeding; Loss
of sharpness or contrast; etc., etc., etc. I guess only you can tell what is quality
loss or something else worth living with, etc.

By default, Scene A1 is an I-Frame. In MPEG, there are I, P, and B frames. The one
you are interested in mostly, is the I-Frame. In VirtualDub-MPEG2, when you open
an MPG or VOB file, and you scan the time line (cursor << and >> keys) and look
down at the bottom of the status line, you will see where it gives you each the
frames of I, P, or B information. If its an P or B, you know its not an I

What you are interested in is finding in each of the dvd rips the image that matches
Scene A1 and note if *that* image is an I-Frame. If it is, than you can go ahead
and analyize that frame, etc. But if any of them are not, than its up to you to
decide to analyize it against Scene A1 or not. Its up to you. But you really want
to have I-Frames in your video/image analysis, though its not always possible.
You see, if you get an P or B frame, what has to happen is those frame have to
be put together from other frames*.

As an example of what you can expect, with respect to what was stated above,
regarding the I, P, and B frames and testing/analyizing them, in the other posts
and mine, I've provided several scenarios to help you visualize and understand
what is actually necessary to have as accurate a test as possible.

Analyize and Compare each dvd rip to -> Scene A1

You said that you have four completed dvd's, and now you want to analyize them.
So, we lable each, A; B; C; and D; as seen, below. The result could be:

Scenario #1
--> dvd rip A -- Scene A1 = Frame Position 1020 = P frame
--> dvd rip B -- Scene A1 = Frame Position 0110 = B frame
--> dvd rip C -- Scene A1 = Frame Position 2200 = I frame
--> dvd rip D -- Scene A1 = Frame Position 2000 = I frame

Scenario #1 only has two (C and D) I-Frames matching Scene A1.


Scenario #2
--> dvd rip A -- Scene A1 = Frame Position 1020 = P frame
--> dvd rip B -- Scene A1 = Frame Position 0110 = B frame
--> dvd rip C -- Scene A1 = Frame Position 2200 = B frame
--> dvd rip D -- Scene A1 = Frame Position 2000 = P frame

Scenario #2 these rip's position did not fall on the I-Frame matching Scene A1.


Scenario #3
--> dvd rip A -- Scene A1 = Frame Position 1020 = I frame
--> dvd rip B -- Scene A1 = Frame Position 0110 = I frame
--> dvd rip C -- Scene A1 = Frame Position 2200 = I frame
--> dvd rip D -- Scene A1 = Frame Position 2000 = I frame

Scenario #3, all these (A, B, C and D) fall on an I-Frame and match's Scene A1.


* Ok. So i'm no expert at this, but.. In MPEG, these frames are not complete. They may
have to go through several Reverse/Forward to rebuild a frame. But an I-Frame is a
complete frame, of which P and B are partially built from. Please excuse my crude
terminology of things.

Keep this in mind.. just because you may not have an I-Frame in your test source, it
doesn't mean you can't still use it. If the MPEG was encoded well, the analysis may prove
worthy, anyway.

-vhelp 4296

[nlec]

I've been following that thread. Time to jump into it, perhaps. I started this thread in the DVD Recorder forum since that was the starting point of my project, but I realize there are aspects of it better suited to the other forums.

[edDV]

If you don't want motion compression, you would transfer to Huffyuv or DV tape and stop there. If you want a DVD, you should minimize further compression for hand held camcorder source.

[Nelson37]

Having done literally hundreds of such tests, I will offer you this -

There is no formula, set of data, or evaluation program that will give you the correct answer, at least once you get all methods fairly close, and they should be. Can I tell you which Apple Pie out of several, all of good quality, that YOU will prefer? The one I like best may not be your favorite.

One encoder may do well with solid colors, while another may not. The same encoder may do poorly with black/white contrast, while another looks fine. Yet a third encoder may perform in between the previous two in these areas, but do a better job on rapid motion. Or perhaps a worse job.

Some people like a sharper image and can live with some artifacts, some folks hate artifacts of any kind and would prefer to soften slightly to remove them. Sharp with no artifacts would be ideal, but damn near impossible to achieve. Although not entirely.

A major problem with comparing stills is that video Moves, Stills by definition do not. You can get some good info by doing this but you must understand the info you are NOT getting.

I used a series of short, 10-second to 3-minute clips with specific features which made them difficult to encode. I then put 10 files on a disk and viewed several times, picking out the 2 or 3 best, and repeated the process. One helpful thing which happened by accident was using Blind Testing, IOW I did not know how the file was made during viewing. This prevents pre-conceived bias from being a factor. I used fairly long filenames and the DVD player only showed the first 8 characters.

There were several surprises discovered and much valuable data learned. My original testing was designed to show what the various settings in TMPGENC actually did, these were adjusted individually in increments and then in various combinations. ALL of them.

One interesting discovery was that certain types of files, otherwise valid, caused my player to choke on any file played after this "evil" file. Thought I had made several bad files until a couple of new burns with different arrangement revealed that supposed "bad" files actually did play, it was only the "evil" file which broke the player for any following files. Took me a while to verify this, never heard of such a thing before or since. But I tested this 6 ways from sunday and am rock-solid certain it did occur.