Comparing DNR on or off on 8mm video to digital conversion, help please

[videon00b]

Hello, I'm converting some 8mm tapes to digital.

My workflow is Sony TRV-85 --> IO Data USB 2.0 capture device --> AmarecTV on computer using Lagarith codec.

On my Sony camera I have TBC ON, DNR ON right now.

My goal is to simply capture the highest quality video. I dont plan on doing much editing for now, but wouldn't be opposed to it in the future once I get these tapes captured. From what I've read, DNR will get rid of noise, at the expense of reducing valuable details, (as if standard definition needed that )

I've read conflicting opinions here on whether or not DNR should be used on a camera 20 years old now that we're in the digital age, with better digital editing software but I lack the experience and eye to tell whether or not I should keep it on or off. That's why I have a couple samples for you guys to take a look at..

So, with my goals in mind (of capping best resolution) , would you turn DNR on or off?

Sample 1---

DNR ON:
https://mega.nz/#!haBHjSwI!gcOjXSwfY2ZusVdwtX4A1yE1mJQODoqkTxNLlg64HB0

DNR OFF:
https://mega.nz/#!VLZT0DyR!YQ3X9GGM-3zUVUKFdDQ2G15gA1j1e-31lRtvXV9RA-8

Sample 2--

DNR ON:
https://mega.nz/#!gDBH2DoL!0-q6r47dy3ykT4q79OB92yBSOskFpxkz2z04ohyUtW8

DNR OFF:
https://mega.nz/#!oP4lBZia!ZzqasmvLsCyrYREbEf2FO7rXdenmpQ-ByRcgiROhoww

(I know my samples aren't legal 16-235, manually changing the procamp messed with my colors and saturation a little much for my liking, and I can always convert it to 16-235 at a later time if needed)

[videon00b]

These tapes are only about 18 years old, I could do a comparison on a tape thats 30 years old if that would be more helpful.

[JVRaines]

I lack the experience and eye to tell whether or not I should keep it on or off.

I doubt that. You don't need to please some machine or a bunch of strangers on the internet.
Which one looks best to you?

[videon00b]

I lack the experience and eye to tell whether or not I should keep it on or off.

I doubt that. You don't need to please some machine or a bunch of strangers on the internet.
Which one looks best to you?

They both look pretty much the same to me. But the tape is relatively new so DNR might not be doing much. But until 2 days ago I thought Digital Noise Reduction meant it was helping with the audio, not visual noise

[jagabo]

It's a very weak noise reduction filter. You can barely see the difference. But it does appear to be causing some color bleeding. I'd leave it off and filter later in software.

And don't capture as RGB, capture as YUY2, UYVY, or some other YUV 4:2:2 subsampling. You've crushed super darks and super brights that could have been recovered.

[videon00b]

It's a very weak noise reduction filter. You can barely see the difference. But it does appear to be causing some color bleeding. I'd leave it off and filter later in software.

And don't capture as RGB, capture as YUY2, UYVY, or some other YUV 4:2:2 subsampling. You've crushed super darks and super brights that could have been recovered.

What kind of software would you filter it with? I'll capture with DNR off then.

As for my videos being in RGB I forgot to check direct stream copy in virtualdub when I was trimming the samples, but Im capturing in YUY2 lagarith. Thanks.

Anyone know what the Drop [0, +15, -0] in the lower right corner means?

[jagabo]

I filter with AviSynth. But it has a steep learning curve. A sample with moderate denoising is attached. You can go stronger or weaker, of course.

Regarding the Drop: I suspect it means no frames were detected as dropped (missed from the source device), 15 frames were inserted, and no frames were deleted -- to maintain the requested frame rate. Your capture should be 29.97 fps, not 30.27. Something isn't quite right.

[videon00b]

I filter with AviSynth. But it has a steep learning curve. A sample with moderate denoising is attached. You can go stronger or weaker, of course.

Regarding the Drop: I suspect it means no frames were detected as dropped (missed from the source device), 15 frames were inserted, and no frames were deleted -- to maintain the requested frame rate. Your capture should be 29.97 fps, not 30.27. Something isn't quite right.

Wow that sample looks really good. It really looks digital. I can see how the back wall all looks like the same color instead of changing around. Did you de-interlace that video?

I have the framerate set to 29.97 - sometimes the cap FPS goes up or down slightly, but I never notice a change in video rate. The bottom reads 30.27, 29.38, 30.30, 29.35 etc on and off but the audio doesnt lose its place with the video, and it always says 0 drops. So it at least seems ok.

Random question- why does my video camera have messed up time numbers? On this tape it starts at -36 and ends at 1:26. So its just over 2 hours. Where does the negative number come from, why doesnt it just start at 0:00?

Also one of these tapes is supposedly only 30 minutes long. At least, it only plays for 30 minutes, but how does my camera know theres not more video footage somewhere in there if it doesnt go through it? Thanks.

[jagabo]

I used Yadif() to smart bob deinterlace, and TemporalDegrain() to reduce noise. Don't know about your camera.

[videon00b]

I used Yadif() to smart bob deinterlace, and TemporalDegrain() to reduce noise. Don't know about your camera.

Ah, so thats why it looks different. Do you think I should deinterlace my video? Won't that make my files larger in the end? I do plan on viewing on the computer, so progressive probably would be best for a computer. But I've heard mixed reviews. Its not something I plan on doing right now anyway as Im still capturing.

My camera is a sony trv-85. I'll rewind the tape all the way back, then start playing it and capturing. Most tapes are 2 hours, but on three of the tapes so far, the videos stop around 30 minutes long, and the outer LCD display will show the time marker not progressing. Though if I hit fast forward it will start fast forwarding through unused tape. But how do I know its unused? It probably isnt used, but Im concerned that it might have something on them and would miss it. Kind of a question that someone can only answer if theyve had this tape player or experience with that issue.

[videon00b]

I was also wondering why is it recommended that the original video camera be used to capture the tape on? Not just the same model but the exact physical copy? I've read that on multiple sites and here on the forums. Does anyone have a theory why this is?

[jagabo]

Using the same camcorder means the heads are more likely to be (mis)aligned the same as when the video was recorded. A different camcorder won't be aligned the same.

The best deinterlacer for most material is QTGMC() in AviSynth -- but it's pretty slow. I used Yadif() for your sample since it's faster. For a real encoding I would use QTGMC() (better than any other software deinterlacer or hardware deinterlacer) and encode at 59.94 fps to keep smooth motion. Some encoders are frame rate aware -- they know that when frames are zipping past at 60 fps they don't need to be as clean as when they are shown at slower frame rates. So they use a little less bitrate for faster frame rates. That combined with the fact the changes between frames are smaller at higher frame rates means that high frame rate videos don't necessary need much more bitrate than lower frame rates. Keep your original video and you can always make a better video if you find a better deinterlacer in the future.

[videon00b]

Using the same camcorder means the heads are more likely to be (mis)aligned the same as when the video was recorded. A different camcorder won't be aligned the same.

Ah. So what does that mean for me? Am I missing some fields or something? Can you really tell a difference between the two (original camera tape was recorded on vs duplicate model camera tape wasnt recorded on) afterwards? I could get the original camcorder but got another one (same model) on ebay so as not to wear the first one out, and because its not mine. But if it means better quality on digital then I'm all for it.

The best deinterlacer for most material is QTGMC() in AviSynth -- but it's pretty slow. I used Yadif() for your sample since it's faster. For a real encoding I would use QTGMC() (better than any other software deinterlacer or hardware deinterlacer) and encode at 59.94 fps to keep smooth motion. Some encoders are frame rate aware -- they know that when frames are zipping past at 60 fps they don't need to be as clean as when they are shown at slower frame rates. So they use a little less bitrate for faster frame rates. That combined with the fact the changes between frames are smaller at higher frame rates means that high frame rate videos don't necessary need much more bitrate than lower frame rates. Keep your original video and you can always make a better video if you find a better deinterlacer in the future.

Cool, yeah that sounds good. I definitely plan on hanging on to the originals. Ive read some of your previous posts on deinterlacing where you mention it makes videos lose some quality and can add artifacts. These videos look so darn good to me as they are though. I'm really glad I'm doing this.

I have a question about this sample (only 15 seconds long) - what do you think is causing this disruption? Is it a dirty tape head? Or a bad section of the tape? The scene just before this was shot 2 months before, so it was probably left in the tape without being rewound. Thanks a lot.
https://mega.nz/#!EOpjTT6a!yw9_7tq8SD485t9ZdilsTSV-yN9O0NXELZigT5qDvkM

[jagabo]

Using the same camcorder means the heads are more likely to be (mis)aligned the same as when the video was recorded. A different camcorder won't be aligned the same.

Ah. So what does that mean for me?

Misalignment results in noise and time base problems. Your first clip wasn't really showing any of that.

I have a question about this sample (only 15 seconds long) - what do you think is causing this disruption? Is it a dirty tape head? Or a bad section of the tape? The scene just before this was shot 2 months before, so it was probably left in the tape without being rewound. Thanks a lot.
https://mega.nz/#!EOpjTT6a!yw9_7tq8SD485t9ZdilsTSV-yN9O0NXELZigT5qDvkM

The camcorder (TBC?) is repeating a scan line (multiple times) when it loses horizontal sync. That could be dirty heads (and weak sync pulses on the tape) or alignment issues. The bottom field is much worse than the top field.

[videon00b]

Using the same camcorder means the heads are more likely to be (mis)aligned the same as when the video was recorded. A different camcorder won't be aligned the same.

Ah. So what does that mean for me?

Misalignment results in noise and time base problems. Your first clip wasn't really showing any of that.

I have a question about this sample (only 15 seconds long) - what do you think is causing this disruption? Is it a dirty tape head? Or a bad section of the tape? The scene just before this was shot 2 months before, so it was probably left in the tape without being rewound. Thanks a lot.
https://mega.nz/#!EOpjTT6a!yw9_7tq8SD485t9ZdilsTSV-yN9O0NXELZigT5qDvkM

The camcorder (TBC?) is repeating a scan line (multiple times) when it loses horizontal sync. That could be dirty heads (and weak sync pulses on the tape) or alignment issues. The bottom field is much worse than the top field.

Yeah I haven't gotten hardly any sections of tape like that. Only the one I just linked, and one other time about 10 tapes ago. Each only lasted about 10 - 15 seconds and weren't very important sections of tape. Not too bad considering I've digitized over ~20 hours and only 30 seconds were poor quality. I'll see if I can get ahold of the original camcorder and then test that section of tape on it. I have a feeling it will be mostly the same though.

Should I maybe clean my tape head with head cleaning tape? I've heard its a last resort but I got this camera off ebay so I don't know what sort of condition its in, might be really dirty. Thanks

[videon00b]

Actually, now that I think about it, having the other camera probably won't do any good. The camera most of these tapes were shot on was lost in the mid 90s, the trv-85 was bought in 98/99 and 80-90 percent of these tapes are from 1988-1998. So only about 10 percent of these tapes are accessible on the original camera they were shot on.

[jagabo]

Cleaning the heads is worth a try. For the bad shot in post #13, one of the fields is much worse than the other. If that's the best cap you can get (or other shots like it) and you really need it, It's possible to use only the better field. Here are the two fields side by side. You can see that the left (top field) is much cleaner than the right (bottom). It's may also be possible to select the better field or the other on a frame-by-frame basis, interpolate motion when only one field is bad, etc.

[videon00b]

Cleaning the heads is worth a try. For the bad shot in post #13, one of the fields is much worse than the other. If that's the best cap you can get (or other shots like it) and you really need it, It's possible to use only the better field. Here are the two fields side by side. You can see that the left (top field) is much cleaner than the right (bottom). It's may also be possible to select the better field or the other on a frame-by-frame basis, interpolate motion when only one field is bad, etc.

Wow thats cool. Any theories on why one field is much worse than the other? The fields are every other line that together make up interlaced video right? Would selecting the good field make for a progressive scan video section? I'm not too bothered by it but I'll fix it if I can easily, would you recomend the method you posted in this thread? https://forum.videohelp.com/threads/351277-How-to-split-video-into-separate-fields

I just cleaned the head with some cleaning tape. The tape sections look basically the same, not better or worse. The head is used for reading the tape and for recording onto it right?

I just did another tape yesterday that had a few sections of bad tape. Just another second or two each. Though this time they were completely blue screened.
https://mega.nz/#!hOBzgSDJ!dLNiHdFfE9M3Up-zftZ1NQ6SJMOeRA6RVXkcLcexOCU

https://mega.nz/#!JD5QlSqK!Sdm_xio8sDsPo7SNQLo7SdIfMH3z0DFgcqrVFKHANKs

I guess its harder to tell whats wrong with this one, since you cant even see the fields (or anything else). They were both on the same tape, about 25 years old. I had just cleaned the heads today before I captured this. What do you think this is? I'm just curious, thanks a lot.

[jagabo]

You could try a different camera to play the tape.

To eliminate one field in that sample I used the following AviSynth sequence:

AssumeTFF()
SeparateFields() # separate the fields: top, bottom, top, bottom, top, bottom
SelectEven() # select only the top fields
nnedi3(dh=true) # double the frame height with advanced edge smoothing

You can also replace short sequences of bad frames with frames motion interpolated from the good frames surrounding them.

AviSource("amarec(20171228-2226)NEW SEGMENT.00.avi") 
AssumeTFF()
ConvertToYV12(interlaced=true)
SeparateFields().SelectEven().nnedi3(dh=true)

ReplaceFramesMC(1) # replace frame 1 with motion interpolated between frames 0 and 2
ReplaceFramesMC(5,2) # replace 2 frames, 5 and 6, with motion interpolated between frames 4 and 7
ReplaceFramesMC(25,2)
ReplaceFramesMC(30,2)
ReplaceFramesMC(48,1)
ReplaceFramesMC(91)
ReplaceFramesMC(93,4)
ReplaceFramesMC(194,2)
ReplaceFramesMC(203,2)
ReplaceFramesMC(238)
ReplaceFramesMC(249,3)
ReplaceFramesMC(256,3)
ReplaceFramesMC(269,3)
ReplaceFramesMC(277)
ReplaceFramesMC(283)
ReplaceFramesMC(286,3)
ReplaceFramesMC(296)
ReplaceFramesMC(299)
ReplaceFramesMC(301,2)
ReplaceFramesMC(304,4)
ReplaceFramesMC(309,2)
ReplaceFramesMC(312,2)
ReplaceFramesMC(315,3)
ReplaceFramesMC(320,5)
ReplaceFramesMC(327)
ReplaceFramesMC(329)
ReplaceFramesMC(363,4)

Sample, with the original on the left, fixed on the right, attached.

There's no way to recreate the long missing segments in those other clips. You might as well just cut them out, crossfade, etc.

[videon00b]

You could try a different camera to play the tape.

To eliminate one field in that sample I used the following AviSynth sequence:

AssumeTFF()
SeparateFields() # separate the fields: top, bottom, top, bottom, top, bottom
SelectEven() # select only the top fields
nnedi3(dh=true) # double the frame height with advanced edge smoothing

You can also replace short sequences of bad frames with frames motion interpolated from the good frames surrounding them.

AviSource("amarec(20171228-2226)NEW SEGMENT.00.avi") 
AssumeTFF()
ConvertToYV12(interlaced=true)
SeparateFields().SelectEven().nnedi3(dh=true)

ReplaceFramesMC(1) # replace frame 1 with motion interpolated between frames 0 and 2
ReplaceFramesMC(5,2) # replace 2 frames, 5 and 6, with motion interpolated between frames 4 and 7
ReplaceFramesMC(25,2)
ReplaceFramesMC(30,2)
ReplaceFramesMC(48,1)
ReplaceFramesMC(91)
ReplaceFramesMC(93,4)
ReplaceFramesMC(194,2)
ReplaceFramesMC(203,2)
ReplaceFramesMC(238)
ReplaceFramesMC(249,3)
ReplaceFramesMC(256,3)
ReplaceFramesMC(269,3)
ReplaceFramesMC(277)
ReplaceFramesMC(283)
ReplaceFramesMC(286,3)
ReplaceFramesMC(296)
ReplaceFramesMC(299)
ReplaceFramesMC(301,2)
ReplaceFramesMC(304,4)
ReplaceFramesMC(309,2)
ReplaceFramesMC(312,2)
ReplaceFramesMC(315,3)
ReplaceFramesMC(320,5)
ReplaceFramesMC(327)
ReplaceFramesMC(329)
ReplaceFramesMC(363,4)

Sample, with the original on the left, fixed on the right, attached.

There's no way to recreate the long missing segments in those other clips. You might as well just cut them out, crossfade, etc.

Cool! That's pretty amazing what you did with that segment of tape. I've done about 30 more tapes since last week and have seen a few instances of segments like this. Do you think age could be what causes these distortions?

I'll play the tapes in another camera, and see if it helps at all.

When I tried the first code you sent me, AVSPmod comes back with "Script error: there is no function named 'nnedi3'" Do I need to use avisynth in notepad do you think, or is AVSPmod not equipped with the extensive library avisynth is?

Thanks a lot. I'm past halfway through converting these tapes and am getting somewhat burnt out. As another poster said in one of my threads a few months ago, fast forward to get the length, then let it play through. That was good advice, as I started out watching through the whole tape, now I'm just trying to finish the converting project as painlessly as possible

[lordsmurf]

You can also replace short sequences of bad frames with frames motion interpolated from the good frames surrounding them.

That's always seemed to be so tedious to me, unless a video was just 2-3 minutes long, and quality was intended for something published. For personal needs, I'd just leave warts. The hours needed to fix seconds seems insane.

[manono]

The hours needed to fix seconds seems insane.

I regularly do it for 2.5 hour-long old movies on VHS tape. Yes, it's tedious and requires intense concentration. Some of us are just gluttons for punishment, I guess.

[lordsmurf]

I think you've found a new punishment.

"If you kids don't behave, I'll make you analyze replacement frames in Avisynth!"
"NNNOOOO!!!!!"

Heh heh.

[videon00b]

It's a very weak noise reduction filter. You can barely see the difference. But it does appear to be causing some color bleeding. I'd leave it off and filter later in software.

And don't capture as RGB, capture as YUY2, UYVY, or some other YUV 4:2:2 subsampling. You've crushed super darks and super brights that could have been recovered.

I didnt notice it before, but now I've made a huge error and believe I see what you meant.

I left the battery outside of the camera, and since the camera is 20 years old I'm sure the CMOS battery is dead too. I assumed the camera would hold my DNR settings so I havent checked it in a while, thats where my mistake lays. Not sure how many tapes have been shot like this, guess its time to go through them to look for color bleed and re-digitize.


https://mega.nz/#!9HAGhCjY!TXUGwMAGWqLK5scE05M1tqaZgGrW6pS-ib0xVzKlUwU
(not in YUY2 because i didnt click direct stream copy)

Notice the grey bricks on the left - theres a redness to them matching the porch deck that comes and goes - thats color bleeding right? I was so close to the finish line too, hah.

[jagabo]

Notice the grey bricks on the left - theres a redness to them matching the porch deck that comes and goes - thats color bleeding right? I was so close to the finish line too, hah.

Some of the bricks are red-ish but that's not color bleeding. Later in the clip there's a round red object on the table in the background. Notice how the red color sloughs off to the right of the object -- that's color bleeding. You can also see it to the right of the blue jeans against the red porch. Consumer analog video tape has very low color resolution. You always get color bleeding like this. Here's a image of the Y (greyscale), U and V (color) channels:



View it full size and see how blurry the U and V channels are. The contrast of the chroma channels has been increased to make them more visible.

[videon00b]

Notice the grey bricks on the left - theres a redness to them matching the porch deck that comes and goes - thats color bleeding right? I was so close to the finish line too, hah.

Some of the bricks are red-ish but that's not color bleeding. Later in the clip there's a round red object on the table in the background. Notice how the red color sloughs off to the right of the object -- that's color bleeding. You can also see it to the right of the blue jeans against the red porch. Consumer analog video tape has very low color resolution. You always get color bleeding like this. Here's a image of the Y (greyscale), U and V (color) channels:


[Attachment 44579 - Click to enlarge]

View it full size and see how blurry the U and V channels are. The contrast of the chroma channels has been increased to make them more visible.

Ah, yes I think I see what you mean. Thanks. I guess those bricks are naturally red-ish. I think the bleeding you mentioned is more noticeable in this scene because of how much the camera shakes around.

Do me a favor please - do you notice that color bleeding occurring in this sample (with DNR off)? You did say that this amount of color bleeding always happens..
https://mega.nz/#!FHAXVIJb!zLRvNEFsJhtv3ow7o-QQYuFhqEDjnuQHKNyOTDb-nBE

So is there a definitive method to spot color bleeding? (my inexperienced eyes cant really tell a difference..)But there's no point in doing lossless capture if I have a crumby DNR filter on, I'll re-digitize them all if I have to, but I would like to know of a method so I could see which ones have DNR on and which ones are off. So are certain scenes or colors better to look for bleeding than others?

As much as I'd like to just finish up soon, my haste set me much further back. Do you think its worth re-doing?

[jagabo]

Your video has pretty muted colors. Bleeding colors are more noticeable with more saturated colors. If you crank up the saturation and enlarge the frame you can see it better.

Before (bobbed, cropped, and enlarged 4x):


Notice as you move from the blue at the center to red at the right how the color changes from blue to purple to red. You can see the chroma channels on the right are very blurry.

After sharpening chroma (too much, to make it more obvious):


The transition from blue to red is much sharper and cleaner. The chroma channels are much sharper (though there are some artifacts).

The bleeding is relatively minor in this clip. It is much more prominent on second or third generation tapes.

A DNR filter should not have significant effect on color bleeding. It will only reduce noise. Here's the sharpened chroma image with the addition of very strong noise reduction:

[themaster1]

try this in your script so you can better identify how much noise there is
Y = GreyScale()
U = UtoY()
V = VtoY()
StackHorizontal(Y, StackVertical(U,V))
Histogram(mode="luma")