Tips for restoration of DVD with dot crawl, bad exposure...

[fcm]

I have been diving headfirst into AviSynth in order to try and restore the footage made of my mothers's funeral service.
Although recorded recently, the footage leaves much to be desired. Lighting is bad, little detail and noise that appears to be dotcrawl (at least that is what I think it is).
I have searched the internet and experimented a lot using filters like LUTDecrawl, Bifrost, Color curve correctors, QTGMC deinterlace. However only with little success.
I was hesitant in posting this on this forum as it is, obviously, a very personal video. But I really want to make the best of the footage that I have and seeing some of the results in other posts on this forum made me realise that some expert help will probably do a lot more than I could hope to accomplish in any foreseeable amount of time.


The source material I have is a PAL DVD.
MediaInfo of a small demuxed sample shows:

Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Duration : 3s 840ms
Bit rate mode : Variable
Bit rate : 3 714 Kbps
Maximum bit rate : 9 263 Kbps
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 4:3
Frame rate : 25.000 fps
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.358
Time code of first frame : 00:03:35:01
Stream size : 1.70 MiB (100%)
Color primaries : BT.601 PAL
Transfer characteristics : BT.470 System B, BT.470 System G
Matrix coefficients : BT.601



I have attached a short clip (M2V) of the video.

[sanlyn]

Dot crawl is very difficult to remove without giving up something in return, which means something of an over-filtered look. The attached mkv sample would benefit from the addition of very slight grain, as with the AddGrainC plugin. For a scene with stronger dot crawl, set MCTemporalDenoise or QTGMC presets to a stronger mode. I chose to save more detail with the sample script. Also addressed some DCT ringing, mild halos, and invalid contrast levels (clipped brights).

This is a very slow script. Those crawling dots are strong little devils.

MPEG2Source("drive:\path\to\video\cropped.d2v")

AssumeTFF().QTGMC(preset="medium") 
SmoothLevels(16,1.0,255,16,225,chroma=200,limiter=0,tvrange=true,dither=100,protect=6)
# -------- hit dot crawl, herringbone, RF hash, etc.
Spline16Resize(360,576) 
MergeChroma(last,McTemporalDenoise(settings="medium")) 
SPline36Resize(720,576) # restore frame size
MergeChroma(awarpsharp2(depth=30))
DeHalo_ALpha()
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

Tip: 3700 kbps is a stingy MPEG bitrate for valuable family archives. Makes denosing more difficult as well.

[poisondeathray]

If this sample is representative of the entire video ? More specifically, is this the same shot with a static camera ? Or is it a mulitcamera shoot?

If so, then you can limit the damage by using a static mask . Using dotcrawl filters are among the most damaging things you can do to video. For something this important , IMO, you really want to preserve as much detail as possible . (Eitherway, you can still use animated masks (e.g. rotoscoping) in other programs , but the principle is the same)

Other tricks if this is a static shoot - you can mask out or replace static areas with a single good still frame . For example , there is no movement in the top 1/2 of the frame. Dot crawl and various artifacts are only annoying to watch when they change between frames. When they don't change you generally don't even notice them . Even when there is movement , or objects moving in front - there are ways around that (rotoscoping, blocking mattes) - it's just that a static shot makes things easier

If you have the original picture that was projected, you can do a replacement that's not as blown out and show more detail. This falls under the category of "compositing" and it's quite a big topic . (And even if the camera moves, you can use various programs to match move the composite) . Sure, you can use various levels filters, highlight recovery techniques to improve what you have, but nothing will look as good as a composited image that is adjusted to match

[sanlyn]

In the brighter part of the image where the center lights are yellow burned-out detail: that detail is gone forever. Lowering luma levels (some of which I did anyway) simply revealed a bright-end "peak" or "spike" that indicates very strong bright clipping. As this appears to be an encoding from a tape original, it might be possible to rescue a bit more detail but one would need the original source, not a low-bitrate encode that has been thru colorspace changes and other processing.

[fcm]

Thanks for your replies!


@sanlyn: The mkv from your script already looks much better than any of my attempts, thank you very much. Looking at both videos side by side I can see what you mean by 'giving up something in return', i.e. details.Regarding the original source, I have tried to get that. However, the A/V guy from the crematory said that there is no tape or DV. The video from the camera is directly fed into an A/V sytem which stores it (and in the process compresses it). The only thing they can do is burn it to a DVD. They contacted the company which delivered their sytem to ask if the stored file could be copied, but this was also not possible. That company also said they were aware of the fact that the quality of the recording is not optimal, but said this is caused by the fact that the crematory uses an analog recorder.
Searching the internet regarding dot crawl makes me think this is not entirely true, the cabling is probably composite which causes the dot crawl. Long story short, the DVD recording is all I have.


@poisondeathray:
The sample is indeed representative of the entire video. It is all shot from a static viewpoint. Additionally, I do have the projected picture (appr. halfway through the service a slideshow with pictures is projected, which is the only non-static part in the upper half of the screen). Compositing this into the video is indeed the general idea, but I still have to look into how to do this in perspective and adjusting it to match the rest of the video.
What would you recommend regarding masking tools for AviSynth? I came across an earlier post from you in the thread
"Help with Dot Crawl or whatever this is..." in which an edgemask was used to the same effect. I haven't yet tried that but it looks promising.

[poisondeathray]

@poisondeathray:
The sample is indeed representative of the entire video. It is all shot from a static viewpoint. Additionally, I do have the projected picture (appr. halfway through the service a slideshow with pictures is projected, which is the only non-static part in the upper half of the screen). Compositing this into the video is indeed the general idea, but I still have to look into how to do this in perspective and adjusting it to match the rest of the video.

What would you recommend regarding masking tools for AviSynth? I came across an earlier post from you in the thread
"Help with Dot Crawl or whatever this is..." in which an edgemask was used to the same effect. I haven't yet tried that but it looks promising.

Static makes everything easier. You don't have to fiddle around with masktools if the distribution of the dot crawl artifacts is the same throughout the video (If it's a static video they generally are - they generally appear in areas of high saturation and brightness where there is Y/C crosstalk. If that's true you can just make a mask it in a photo editor e.g. gimp , photoshop . You can think of masks as a way of compositing different layers or applying different filters to different parts of the picture. I'll post a quick example later if I have time

It might be easier to do the compositing in other dedicated compositing programs, but there are some perspective plugins in avisynth (from v.c mohan and david horman), and vdub, and ffdshow . If you have stills (photos) from the ceremony and backwall, you can replace those blown out areas as well (and of course make adjustments to match, which is really the "art" of compositing) . I'm assuming you have the slideshow as well ?

[sanlyn]

Yes, sounds complicated getting into the original recording. As for "detail", I didn't apply a sharpener, but there are some subtle tricks one can play. One is to slightly lower black levels, the other is to use subtle sharpeners like FineSharp. Then add a very light touch of some fine grain. This of course doesn't "create" detail, but it gives the impression of doing so.

Masking/tracking techniques per poisondeathray will certainly preserve more of the original. The software for that isn't exactly cheap, such as AfterEffects I think Vegas Pro can also do that. It's the sort of project that would take time, but for those who value their history it would certainly be worth it.

I hate dot crawl. I already said that, right? Can't say it enough.

[fcm]

I agree on the dot crawl hating bit But i am getting really optimistic reading both your replies!!


I am familiar with Photoshop and have worked with masks on images before, that sounds do-able. Using sanlyn's script I can use that to counter the dot crawl only on the 'infected' areas.
I do have stills from the setting, perhaps even from the direction of the video camera. If I could use those to replace the walls and repair the clipped areas...Wow!.
Those stills are of much larger resolution than the video, would it then be best to upscale the resolution of the video, or would the difference in detail stand out. There's probably a trade-off.


It looks like I can reach a really great end result if I spend my time on it.
In a month's time, on the 5th of December, we celebrate Sinterklaas in the Netherlands. (It's similar to Santa Claus, if you don't know it (even if you do) you should look on YouTube for David Sedaris's Six to eight black men, it's hilarious!)
I would like to give the restored video to my father that day, so I've got my work cut out for me...


I'll start by creating a mask for the dot crawl removal and using this mask on the script of sanlyn.
Then deinterlace, sharpen, resize (or should it be some other way around?).
Finally: find out if I possess any of those compositing "arts"...

[poisondeathray]

I am familiar with Photoshop and have worked with masks on images before, that sounds do-able. Using sanlyn's script I can use that to counter the dot crawl only on the 'infected' areas.
I do have stills from the setting, perhaps even from the direction of the video camera. If I could use those to replace the walls and repair the clipped areas...Wow!.
Those stills are of much larger resolution than the video, would it then be best to upscale the resolution of the video, or would the difference in detail stand out. There's probably a trade-off.

Since you're familiar with photoshop, masks, layers, you're already half way there

You don't need any of the motion tracking stuff or expensive software for simple static masks - because this is static shoot, no camera movement. For example you can do the mask in gimp/photoshop, and composite /layer it in avisynth. Only if you want to do a precise job and roto out moving objects / people that you might need to use other software, but that's a lot of manual work even with motion tracking and dedicated software. Or you can chose something in between, maybe rough mattes are good enough for your goals. Depends how much time you want to spend on it.

I hate dot crawl with a passion too. The treatment for it is among the worst things you can do to video. Getting rid of it is easy. It's retaining details and not turning the picture to mush that's the hard part

In order to do a composite properly you usually look at the "lowest common denominator " . You usually "degrade" the high res photos to match the rest of the video, otherwise the composited in layers will stick out like a sore thumb. You have to ask yourself what your goals are: if you want it to look "realistic" - as if it was shot by maybe better lighting, better camera, or if it's ok to look "fake" like it's obviously stuck on . If it looks too clean and perfect it will be an obvious mismatch. The goal of compositing is usually to make things blend in realistically. This includes many things like multiple layers and blending modes, relighting, grain matching, adding noise. But you might have different goals in this project

I'll start by creating a mask for the dot crawl removal and using this mask on the script of sanlyn.
Then deinterlace, sharpen, resize (or should it be some other way around?).
Finally: find out if I possess any of those compositing "arts"...

What is your final formal goal ? Is it a DVD ?

[sanlyn]

The masking idea gets more interesting, since I'm learning some of that myself in AE. So I'll keep tracking this thread. Meanwhile, the holidays could slip upon us sooner than expected. You might need a 'Plan B'. Attached are reworks for PC-only and for DVD. The original has too much wasted light on the foreground audience, with people near the podium nearly washed out and the podium itself over lighted. Script is modified from earlier; runs slower, but looks more like the original, calms slight flicker in the upper part of the video and avoids banding/block noise along the top half. Increasing bitrate to something more appropriate makes a big difference:

MPEG2Source(vidpath+"cropped.d2v")
AssumeTFF().QTGMC(preset="medium")
ColorYUV(cont_y=-10)
SmoothLevels(14,0.88,255,16,230,chroma=200,limiter=0,tvrange=false,dither=100,protect=8)
# -------- for dot crawl/moire
Spline36Resize(360,576) 
MergeChroma(last,McTemporalDenoise(settings="medium")) 
Spline64Resize(720,576) # restore frame size
RemoveGrain()
LSFMod(strength=75)
MergeChroma(awarpsharp(depth=30))
DeHalo_Alpha()
SmoothTweak(saturation=1.15)
AddGrainC(0.5, 1.5)
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()
# -------- DEFLICKER 
ConvertToRGB32(matrix="Rec601",interlaced=true)
vdpath="Drive:\path\to\VirtualDub\plugins\"
LoadVirtualDubPlugin(vdpath+"Deflick.vdf", "DeFlick",1)
DeFlick(0, 6, 1, 0, 0)
Crop(4,0,-8,0).AddBorders(6,0,6,0)
ConvertToYV12(interlaced=true)
return last

[fcm]

Well, it's been a while since my last post here, sorry for that. Just returned last weekend from a two-week holiday .
This week I've been fiddling around with the script you provided, sanlyn, thank you very much!
Using photoshop I created a mask which targets the dot crawl infested areas, while leaving most of the rest of the image intact (at least, that is the general idea).
The script which I am using now (shamelessly ripped from yours, sanlyn):

#SetMTMode(5,8)

MPEG2Source("D:\Avid projects\Crematie mama\DVD Yarden\VIDEO_TS\Teststuk nr3.d2v")
AssumeTFF().QTGMC(preset="medium")
ColorYUV(cont_y=-10)
SmoothLevels(14,0.88,255,16,230,chroma=200,limiter=0,tvrange=false,dither=100,protect=8)
main=last

#SetMTMode(2)

# process dot crawl

last=main
Spline36Resize(360,576) 
MergeChroma(last,McTemporalDenoise(settings="high")) 
Spline64Resize(720,576) # restore frame size
RemoveGrain()
LSFMod(strength=75)
MergeChroma(awarpsharp(depth=30))
DeHalo_Alpha_mt()
SmoothTweak(saturation=1.15)
AddGrainC(0.5, 1.5)
heavy_filtered=last

#process rest

last=main
RemoveGrain()
LSFMod(defaults="slow")
MergeChroma(awarpsharp(depth=30))
DeHalo_Alpha_mt()
SmoothTweak(saturation=1.15)
AddGrainC(0.5, 1.5)
low_filtered=last

#create & apply mask

#SetMTMode(5,8)
ImageSource(file = "D:\Avid projects\Crematie mama\D2V bestanden\Dot crawl mask.bmp",end = 335,fps = 50)
#SetMTMode(2,0)
ConvertToYV12(interlaced=false)
mt_expand
mymask=last
#mymask=mymask.Blur(1.0).ColorYUV(gain_y=50)

mt_merge(low_filtered, heavy_filtered, mymask)
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

#SetMTMode(5,8)

# -------- DEFLICKER 

ConvertToRGB32(matrix="Rec601",interlaced=true)
vdpath="D:\Programs\Videobewerking\VirtualDub-1.9.11\plugins\"
LoadVirtualDubPlugin(vdpath+"Deflick.vdf", "DeFlick",1)
DeFlick(0, 6, 1, 0, 0)
Crop(4,0,-8,0).AddBorders(6,0,6,0)
ConvertToYV12(interlaced=true)

return last

I have attached the sourcefile for the mask.

The above script gives an interlaced 25fps version. I am thinking of ultimately making two versions: a DVD version and a deinterlaced version, resized to 900x720.

I am running into some issues, though.
First of all I can't get the MT mode functional, if I try VirtualDub crashes right at the start.
If I can get some more speed in the process it would be nice. Right now I'm getting a whopping 1.15 fps. At 25 fps output video and a video length of 1 hour the processing of the whole video would take appr. 22 hrs to finish. If that's the case, so be it, but perhaps some more speed can be squeezed out of the final script.
Secondly, I encoded to x264 so I could post a sample of the above script here. But for some reason the result looks very poor in VLCplayer, like it is oversharpened. When I save as uncompressed data it plays fine. At first I blamed the encoder (Virtualdub/MeGUI). Only today (after two evenings of frustration) I found out that it is VLCplayer doing it, when I play in KMPlayer it looks fine.....? I find it strange that it is only the encoded version that is affected and not the uncompressed one.
I have attached the sample video file.

I am hoping you guys can give some feedback on the result so far. I have mostly just copied from several scripts I found, but am unfamiliar with what most of the settings do. I fiddle around with them to see the effects on the result, but that becomes quite time-consuming.

On a sidenote: Have you watched the David Sedaris video?

Regards

[poisondeathray]

It looks over filtered to me. Personally, I would lower the sharpening and some denoising

The mask usually doesn't need to be that fine or particulate. You might miss areas on the mask because the nature of dotcrawl - it tends to move about a few pixels from one field to the next. With a stationary mask, you need to cover all those areas . Even with mt_expand, there are "grey areas" that the mask misses . If you want to see the effect of mt_expand on the mask you can set u=128, v=128 . mt_expand(u=128, v=128). I would have just drawn a few boxes, and left larger areas unaffected . For example, the curtains are largely unaffected, but have quite a bit of involvement in the mask. Same with the brick wall. Assuming the curtains don't move later, you can completely mask that section off and replace it with a still frame.

[poisondeathray]

RE: screen replacement

As mentioned above, since this is static camera shot on a flat planar surface you don't need any motion tracking or advanced tools - you can do that part in avisynth with various perspective plugins, like v.c. mohan's "perspective", or david horman's "quad", or even ffdshow's or vdub's perspective correction filter

I'll give an example using "quad" . Using avspmod, you can use the preview (f5) to get the x,y coordinates of each vertex (just mouse over). If you haven't used avspmod yet, you should. It's very useful for avisynth
http://forum.doom9.org/showthread.php?p=1592394#post1592394

In this example, I used a semi-transparent green overlay to see and tweak the coordinates. You would replace that with your actual photo or slideshow, but you would have to use quad function on it again .


https://forum.videohelp.com/attachment.php?attachmentid=21379&stc=1&d=1385138423

orig=MPEG2Source(...)

BlankClip(orig, color=    $FFFFFF) #white clip for mask layer
ConvertToRGB32()
quad(371, 35, 504, 29, 370,148, 503,148,    0,0, 720,0, 0,576 , 720,576)
q=last

BlankClip(orig, color=    $008000 ) #green clip for test layer
ConvertToRGB32()
#quad(371, 35, 504, 29, 370,148, 503,148,    0,0, 720,0, 0,576 , 720,576) #don't need it for the test overlay
green=last

overlay(orig, green, mask=q, opacity=0.6)

You can adjust to your tastes; for example you might want to use the entire screen instead, depending on the AR of the projected source material (Keeping in mind this is 4:3 PAL, and not square pixel source material). You might want to "feather" the mask edges to make it blend in more , or use antialiasing filter on the mask so you don't get as jagged "stair stepping" from the quad/perspective plugin

e.g

q
ConvertToRGB32()
Santiag(1,1)
Blur(1,1)

If you're going for more authentic realism, you have to do other manipulations to the photo and slide show. Also you would normally do other "compositing" things like put the roof reflection back in. So as the slide show progresses the reflection of each different slide is show on the roof. You do that by mirroring that layer and flipping it, adjusting the opacity




Replacing the other background elements can be more tricky depending on your source photos because of the curvature in the back wall. If your source photos are "not ideal", what is normally done are camera projection techniques . This is quite advanced and you need specialized 3d software . But since there is no camera movement, you might be able to get away with just 2d manipulations like warping and corner pinning

[sanlyn]

Sorry I missed the last few posts. Pretty busy.

The script which I am using now (shamelessly ripped from yours, sanlyn):

Shameless ripping allowed. How do you think I get my script ideas?

The above script gives an interlaced 25fps version. I am thinking of ultimately making two versions: a DVD version and a deinterlaced version, resized to 900x720.

You can't use 900x720 for DVD. That frame size, played as-is, is not a 4:3 image. If you mean that you want 720p for HD/h264, the frame size would be a 960x720 image in a 1280x720 frame (with side borders). That HD format plays only at 1:1 pixel aspect ratio and uses 16x9 images. The BluRay/AVCHD spec for 1280x720 calls for 50p FPS playback, which you would have if you deinterlace 25FPS interlaced video. You would need a fairly high HD bitrate at 1280x720 to maintain quality. But BluRay/AVCHD and h264 also have standard definition 4:3 and 16:9 modes. Here are the specs:
DVD (PAL/NTSC): https://www.videohelp.com/dvd#tech
BluRay/AVCHD(SD and HD): https://www.videohelp.com/hd#tech

First of all I can't get the MT mode functional, if I try VirtualDub crashes right at the start.

I can't get MT to work either. Neither can many other users. I guess some members here can prescribe solutions, but I've looked up all of them until I was blue in the face. It's also a fact that many plugins just don't work in MT mode.

Complex scripts often run 2X to 4X faster if you split the script into two processes, which many people do. Save the output from script 1 and use it for input to script 2. With many plugins you'll use less memory swapping/stacking, which is where the slowdown is.

Secondly, I encoded to x264 so I could post a sample of the above script here. But for some reason the result looks very poor in VLCplayer...

Most users have a lot of problems with VLC Player. I hardly use it. MPC-BE is my mainstay, although it also has a "processed" look sometimes but at least it's not wiggling and slurring all over the place like VLC. The idea that all media players look alike is a myth. They don't.

Denoisers + sharpeners can have that over filtered look, especially with difficult video. You can do lots of tweaking, as with QTGMC (set the sharpener lower, for example. Default is 1.0. Try 0.8 or 0.7). LSFMod is a sharpener with so many settings it can drive you batty, but if you look at the html you'll see settings for edge-only, avoiding clay-face, etc. Adding some fine grain also helps to avoid the overdone look (AddGrainC). GrainFactory3 is another one, often used to simulate the look of film. Be careful: its effects are "thicker" than AddGrainC. You do that near the end, and after any sharpeners. Another plugin that helps to avoid the overdone or posterized look is GradFun2DBMOD, often use to combat banding. It also adds a little grain, depending on how you set its "thr" parameter. I believe the default is 1. You can set that up to 3, but it'll soften at that point and add too much grain. 1.0 to 1.8 is a workable range. Use these grainers/debanders at the very end.[/QUOTE]

Some good points from poisondeathray. I'll have to try some of that with After Effects, which I'm still learning. But that's another story, and apparently a lifelong quest....