Repairing large .avi file for missing metadata or indexing

[johnmeyer]

I know you tried to open the file in VirtualDub and it didn't open. However, I've had this exact problem on many occasions, and I've always been able to fix it using VirtualDub. It usually provides some sort of error message saying that the file is corrupted, but that it can rebuild the index. It takes awhile to do this, and you then get an error message that seeking might be really, really slow, which it is. You then use a lossless codec (or a good intermediate codec like Cineform) to re-encode the file.

Perhaps you should try a different version of VirtualDub. I still run XP 32-bit for most of my work, and have a relatively old version of VD.

[Rapid99]

I know you tried to open the file in VirtualDub and it didn't open. However, I've had this exact problem on many occasions, and I've always been able to fix it using VirtualDub. It usually provides some sort of error message saying that the file is corrupted, but that it can rebuild the index. It takes awhile to do this, and you then get an error message that seeking might be really, really slow, which it is. You then use a lossless codec (or a good intermediate codec like Cineform) to re-encode the file.

Perhaps you should try a different version of VirtualDub. I still run XP 32-bit for most of my work, and have a relatively old version of VD.

Yeah I never got such a message x.x
It "loads" 2 frames of nothing and behaves as if all is well.

It's not re-wrapping if the filesize is larger. I meant with the AVI loaded directly

If you feed it an avs script, it will be uncompressed ,since avisynth frameserves uncompressed audio & video. But if that's working just use a lossless compression format eg. ut video codec. It works well, very stable, fast, and compatible with programs like premiere

If you manage to get it working - there are some potential levels issues with fraps, depending on if you are using the "lossless RGB" version, or the normal YUV version. There are potential issues with decoders, as the YUV version will be decoded full range by open source decoders like LAV

Oh... hm.
Well, if I load the video directly, without using the avisynth .avs files, nothing happens o.o it brings up a blank window in Vdub with 2 frames of nothing. And if I try to rewrap that, it makes a 10kb file of nothing... but, if I load the video with the .avs file, I can load everything in...

Last night, I exported a few 60 second samples with different codecs, so I could compare screengrabs of them all from various timepoints, and then compare them to 100% uncompressed.
What I ended up with, was:

Original, no compression: 20Gb file size, looked best (obviously)
HuffYUV: 5.5Gb file size, looked most like the original, but with extremely slight contrast differences
X-Vid MPEG4: ~2Gb file size, looked almost exactly like the HuffYUV sample in terms of coloring and quality... however, I noticed in some dark areas (shadows in corners and such), there was almost what appeared to be dithering(?) issues. The dark colors didn't blend smoothly, they kind of clumped together in areas.
Lagarith: ~3.5Gb Quality looked solid, similar to HuffYUV... except the colors almost washed out a little. Everything was a little brighter. For example, the grass had a nice solid contrast to it originally, while this codec seemed to turn up the Yellow value a little.

I originally wanted to use HuffYUV... except each time I tried to save it, about 20 minutes in, it would give me the same Error crash in Vdub as I had earlier, with the LAVSplitter... except, instead of the LAVSplitter being the issue, it was attributed to "avutil-lav-52"... I retried 3 times and kept getting the same thing. So, I moved on, and used Lagarith instead, figuring I could just correct the color in post using After Effects. I let that save overnight.


So, here is what i've got: The 60 second sample played fine, no issue. Looked and worked just like a regular .avi file, thumbnail and all.
The Lagarith one worked with no issues, it appeared. 1:18:00 file, 60fps, 370GB in size... but it didn't generate a thumbnail. I opened the file, and it played as expected... except now the video was lagging behind the audio again v.v and trying to skip around seemed to make the media player freeze up, though it was possible to do so.
The issue there, is when I tried bringing it into premiere... it gave me the hour+ of audio... but no video data. I must have been tired because I completely forgot that Premiere doesn't support Lagarith... *sigh* lol


SO... i'm going to give that ut video codec a shot.

Any other ideas for the table are open, of course. I'll just give that a shot for now, though.

[poisondeathray]

ut video is probably the "best" lossless option, because it works nicely with AE, Premiere (it's the lossless codec of choice even recommended on the Adobe forums) and it's decoding speed is very fast compared to lagarith. That might be why you are lagging or stuttering trying to play back 1080p60 lagarith. Magic YUV might be slightly faster , but it's realtively new, stability not as throughly tested by thousands of users yet

The color /levels issues might be because of fraps YUV and full range decoding issue I mentioned earlier . The fraps decoder decodes at normal range so everything looks right. LAV/ opensouce decoders decode at full range , so you either have to bring them back to normal range, or convert to rgb with a full range matrix . Slight shifts in color might be from Rec601 vs. Rec709 conversions . If you record in fraps RGB, none of these issues will occur

[Rapid99]

ut video is probably the "best" lossless option, because it works nicely with AE, Premiere (it's the lossless codec of choice even recommended on the Adobe forums) and it's decoding speed is very fast compared to lagarith. That might be why you are lagging or stuttering trying to play back 1080p60 lagarith. Magic YUV might be slightly faster , but it's realtively new, stability not as throughly tested by thousands of users yet

The color /levels issues might be because of fraps YUV and full range decoding issue I mentioned earlier . The fraps decoder decodes at normal range so everything looks right. LAV/ opensouce decoders decode at full range , so you either have to bring them back to normal range, or convert to rgb with a full range matrix . Slight shifts in color might be from Rec601 vs. Rec709 conversions . If you record in fraps RGB, none of these issues will occur

Oh wow. Yeah, I just did my sample export and I can't find a single difference between the original and the ut video RGB sample, other than it being less than half the file size.
(When I look for differences, I like to open each sample in MPCHC; CTRL+G and skip to an exact time / frame, fullscreen, screenshot, put it into Photoshop.. and then do the same with the same frame on the other sample... then layer them, and show/hide the top layer, with the compressed layer underneath... looking for any changes in color or anything... I can't find a single spot where the pixels change or blend differently, it's actually rather impressive)

[Rapid99]

Alright. So. The ut codec export just finished... and it seems to have worked... sort of o.o

When I play the video, all is well, up until about 34 minutes, 38 seconds. The game comes to a loading screen, the video freezes (progress bar keeps going) and audio stops as well.

Of course there is the possibility that maybe for some reason, the footage after this point is borked. But I would like to make completely sure.
If it is still there, I would be more than happy to just re-export the second half as it's own .avi and splice them together later.
(Oh, and the footage that I do have works fine in Premiere, so, that is all good to go)

Is there a way for me to use avisynth to open a specific range of frames from the video, without starting from the beginning? Like, instead of frames 1 - 280,800... could I have it just load frames 126,000 (34:30) - 280,000?

Thanks

[poisondeathray]

Trim(126000,280000) will return that frame range

However, Directshowsource() isn't necessarily frame accurate (should be ok with fraps, since it's I-frame only) , but still I would cut a larger portion around the area of interest just in case, you can always edit out the overlapped portions

[Rapid99]

Trim(126000,280000) will return that frame range

However, Directshowsource() isn't necessarily frame accurate (should be ok with fraps, since it's I-frame only) , but still I would cut a larger portion around the area of interest just in case, you can always edit out the overlapped portions

Hmph.
When using the Trim value, the further into the video I trim, the longer Vdub takes before it even lets me preview the footage. Starting at frame 1,000 or so takes a few seconds to pull together for preview... starting at 10,000 takes 2 or 3 minutes... skipping to frame 120,000 took a good 15 - 20 minutes to even let me start previewing... and I got to watch that preview for maybe 10 seconds before it crashed Vdub, with the "avutil-lav-52" error from earlier.

x.x

[poisondeathray]

I didn't expect everything to go "smoothly" for a damaged file . But I don't have any more ideas.....

[Rapid99]

I didn't expect everything to go "smoothly" for a damaged file . But I don't have any more ideas.....

It's all good. You've given more than plenty of help, and i'm definitely appreciative.
If I can't fix the second half by fiddling with it, i'll just make due with what i've got.
Thanks