Need help cleaning up a couple dark videos [Bonus $]

[tkf]

Hi All,


I have two videos that were shot in the evening, and came out very dark. One is less than a minute, the other is about 20 minutes. I tried to clean them out myself using Virtual Dub and tutorials, however this is going to require someone with a little more talent. I have uploaded the one minute clip to dropbox, available for download here:https://dl.dropboxusercontent.com/u/98647932/pn2.avi


I am hoping someone can make them a little more bearable, if you have any luck with the first one the second is about the same.


The other issue I have is the time stamp seems to have been removed on import (miniDV imported through DVIO) is there any way to replace it?


These are tapes I need for work, and I am willing to provide a modest donation ($80) to whoever can help me out, however I am on a short timeline so the sooner the better!


Also I realize there is a crazy amount of camera shaking going on here, and a few glitches in the tape, not much I can do about that so as far as I know it will just have to stay.


If you have any questions feel free to send me a PM or post here.


Thanks.

Side note - I accidentally posted this thread in the editing forum last week (which I am learning was the wrong subforum), I'm not sure if there is a way to delete it, I hope I'm not breaking any forum rules with the double post!

[hech54]

I hope I'm not breaking any forum rules with the double post!

You are.

[redwudz]

tkf, please do not double post. I've deleted the other topic. If you want to move a topic to another forum, PM a Mod.

Moderator redwudz

[tkf]

tkf, please do not double post. I've deleted the other topic. If you want to move a topic to another forum, PM a Mod.

Moderator redwudz

Thanks. Sorry about that one.

[Mephesto]

Quick attempt:

[poisondeathray]

The other issue I have is the time stamp seems to have been removed on import (miniDV imported through DVIO) is there any way to replace it?

If you know how to use avisynth, you can use DVInfo() to "hardcode" the record time stamp

[sanlyn]

Quick attempt:

Cool. Did you do that with all 1659 frames in the sample?.

I think the purple comes from incorrectly converting the original colorspace.

[Mephesto]

No just the first 50, I haven't downloaded the complete file yet. I am working in RGB mode because it's annoying fixing brightness, contrast, gamma and color with Aviscripts but the purple looks better than gray or brown IMO.

OP, you saw the sample, is it good enough?

[sanlyn]

You'll avoid discoloration by first converting to YV12 or YUY2 in Avisynth. Then you'll see that there's so little video data, it looks as if it was shot in b&w. Maybe it was. I haven't had a lot of time with the download yet, but what's "down there" is mostly noise. The jpg looks smoother, but a lot of denoising kills what "detail" is there, which is precious little. There is such a thing as dark video, and such a thing as underexposure, but here is a case of "no data". That seems odd considering how much real light falls on the objects now and then.

[Mephesto]

HERE is your detail, sanlyn:

Btw, neat video only works with RGB anyway so conversion will still happen.

[sanlyn]

That's what I mean. The "detail" is all noise. If you raise the darkest levels to RGb16 or brighter, all of that noise is about 40 or 50 RGB points ahead of it. Most of the detail, what there is of it, is spred over a spectrum only about 40 RGB value in extent. The bright lights are higher, of course.

[tkf]

No just the first 50, I haven't downloaded the complete file yet. I am working in RGB mode because it's annoying fixing brightness, contrast, gamma and color with Aviscripts but the purple looks better than gray or brown IMO.

OP, you saw the sample, is it good enough?

That is definitely a vast improvement, and likely exactly what I am hoping for. Would you mind hitting one of the frames in the middle of the video, particularly one of the frames where you can see the individual getting into his car before driving away? that is the part I am most concerned with on this one.

[jagabo]

Adjustments that bring out details in one shot totally blow them out in another. For example, adjustments to this frame to bring out details of the man's face:



Make it impossible identify the vehicle in this frame:



So to do this properly is a lot more work than you think. You have to identify what details you want from which frame.

Here's a brighter version that something of a compromise for all shots. I also applied deinterlacing and a heavy noise reduction filter so it's not as sharp as it could be.

[Mephesto]

Here's the best I could do.
http://www.sendspace.com/file/oc8uqc

[vhelp]

i saw this a few days ago and played around with it. i deinterlaced it but kept it at 720x480, 60fps but still didn't like it.
the video is just too dark.

the other problem, which definately ain't helping, is video shake, and zooming in and out.
* shaking: adds to the complexity of any analysis work
* zooming: beyond the lens max (aka, going into digital zoom) adds pixelation since it has to add pixels. most camers
today have a line in-between normal lens magnifcatin/zoom vs going past that and having to "digitally" add pixels, which is far worse.
* no proper user settings for night time video

here is my suggestion:
1. tonight, go out side to the same place, possibly with a friend
2. let him do the walking in and out of that place, and then let him drive off a few times
3. while, you situate the camera on a tripod and make proper video shoots
4. fine tune that setup and then reshoot that scene again.

when you pan (move) the camera left/right, do it smoothly without jerks. then, maybe next time, you'll get it right and with better results.

[MaverickTse]

With timestamp (24MB MP4): http://sdrv.ms/1ju25y3
(The video actually do not have the dotted box around the timestamp)

[Attachment 21720 - Click to enlarge]
Tool: AviUtl
Process:
NNEDI3 (deinterlace)
Luminance x2
Brightness up a little bit
Denoise (spatial and temporal)
+Time counter

[tkf]

Wow getting some awesome responses here!

I will get through everything today and get back to you guys!

Edit - I just took a quick look at these and I am blown away at how much detail you guys were able to pull out of this clip! I will get the other clip posted up today (its a much bigger file) and see who wants to take on the task!


Thanks so much everyone, I cant believe how well some of those came out.

[RWH]

Couldn't do much with the video.

However, I was able to extract a latent print off the doorknob with crazy glue and silly putty. I have sent the information to INTERPOL, the suspect will be in custody soon. Good work team.

Mephesto, I hope you share how you did that sometime. I'm not qualified to judge, but I think your video is impressive.

[Mephesto]

RWH, messed around with brightness contrast and gamma while closely monitoring the histogram until it looked right. I made sure there were no peaks on a frame without the street lights. There are a few such tools that do this automatically for heavily-afflicted videos like these with a narrow contrast range. One of them is a histogram equalizer for Virtualdub with a dithering option to fill out the gaps in the histogram but I didn't use this for this video because it made everything look so bland and unnatural. It's useful for many videos but some need to be manually adjusted like this one.

Now it's my turn to ask a question, how did Maverick restore that timestamp? Is there metadata in the AVI or did you make that up?

[poisondeathray]

Now it's my turn to ask a question, how did Maverick restore that timestamp? Is there metadata in the AVI or did you make that up?

He used AVIUtil , but I suspect the OP wants the original record date timestamp (the metadata in the AVI) , not the running time SMPTE timecode for the clip

Use DVInfo() for the original timestamp, ShowSMPTE() for the running timecode in hh:mm: sec:frames, or ShowTime() for the hh:mm: sec:ms notation . Each has adjustments for x,y to adjust the position, fonts, sizes, colors etc...

Top Left : DVInfo()
Bottom Left: ShowTime()
Bottom Right: ShowSMPTE()

[Mephesto]

I can't open it with avisource in avisynth yet opening raw in Vdub works just fine.

[poisondeathray]

I can't open it with avisource in avisynth yet opening raw in Vdub works just fine.

with AVISource you probably need a VFW DV decoder eg. cedocida

alternatively , you can use ffms2 eg. FFVideoSource()

vdub works fine opening natively because it has an internal dv decoder

[Mephesto]

Oh wait a minute it's not a codec issue, it only happens when I had DVInfo() in the script. Avisource without it works fine.
What's the latest version of DVInfo anyway? All the links are dead. The one I have is from 2010.

[Budman1]

If you want Video Information without DVInfo, just use these 4 lines from AVISynth: http://avisynth.org.ru/docs/english/corefilters/showframes.htm

Subtitle("Ancient Mars was once covered in water -- and MAVEN will find out where it went _ Fox News.flv", font="Arial", size=24, text_color=$ff0000, align=3)
ShowFrameNumber(scroll=true, x=10, y=27, font="Arial", size=24, text_color=$ff0000)
ShowTime(x=72, y=44, font="Arial", size=24, text_color=$ff0000)
ShowSMPTE(fps=23.976, x=68, y=61, font="Arial", size=24, text_color=$ff0000)

Will display as shown:



Launcher is my program (Work in Progress) on the right for FLV repair and syncing.

[poisondeathray]

Oh wait a minute it's not a codec issue, it only happens when I had DVInfo() in the script. Avisource without it works fine.
What's the latest version of DVInfo anyway? All the links are dead. The one I have is from 2010.

I don't know the version I have is from 2005. It's not as if the "DV" format has changed in the last 10 years...

Maybe the newer versions add better compatibility or new features , not sure . This is an avisynth 2.5.x plugin but it works ok in 2.6.x .


EDIT: here is the post for the 2010 version, but like you said, DL link is dead

Maybe a little late - but here is a small update:
- really switching off the output
- fixed silly bug when working with some big type-2 AVIs
- moved the storage of the strings to class scope

http://forum.doom9.org/showthread.php?p=1404876#post1404876

[Mephesto]

The 2005 version didn't work any better either.

Budman, I'm not interested in making numbers up, I wanna actually extract from the metadata.

[poisondeathray]

What error message are you getting ?

Did you call it like this:

file = "S:\PATH\pn2.avi"
AviSource(file)
DVInfo(file, "rec_time")

[Mephesto]

Oh my bad, no. I called it more like this:

file=avisource("M:\pn2.avi")
DVInfo(file, "rec_time")

So I have to call the file twice basically?

[Budman1]

Sorry for the misunderstanding... I reran the file and got an error on avisource also so tried it with directshowsource.
Maybe that will work for you. The script I used also included the original so I could have the frame and time from
start but it also showed the dvinfo as well. For some reason I have to load the plugin:

loadPlugin("C:\Users\Bud\Desktop\Flash_Date_Time_S yncTabbed\bin\Debug\Dvinfo.dll")
file = "C:\Users\Bud\Desktop\pn2\pn2.avi"
directshowsource(file)
DVInfo(file, "rec_time")
Subtitle(file, font="Arial", size=24, text_color=$ff0000, align=3)
ShowFrameNumber(scroll=true, x=576, y=27, font="Arial", size=24, text_color=$ff0000)
ShowTime(x=638, y=44, font="Arial", size=24, text_color=$ff0000)
ShowSMPTE(fps=29.97, x=634, y=61, font="Arial", size=24, text_color=$ff0000)

Hope that works for you as well.