need help with corrupted frames

[tom_nito]

I used MakeMKV to rip Venture Bros episodes from a Blu-Ray, but there's odd corruption in some of them. It seems to be a problem with how the disc was manufactured because I've read about other people having the same issue. Some of the corruption I only see in certain video players or after encoding, while some of the other corruption either appears either as skipped frames in some players or as visible corruption in others.

Anyways, I think I've managed to fix most of it with weird tricks, but I'm unable to fix a few frames. I was wondering if anyone knew of tricks to restore some of the frames. I'm not very proficient with AviSynth, but I tried BadFrames plugin, and it created a kind of ghostly illusion of movement. I'd like to improve on that. I've attached the sequence of six frames as BMP files. The first and last frame are fine with four corrupted frames in between. Two of the frames are completely corrupted (0028x, 0030x) while the other two are mildly corrupted (0029x, 0031x). I was hoping to at least somehow clean up the two mildly corrupted frames and then using badframes to do what i can with the other two.

[poisondeathray]

But it plays fine as a physical disc in a hardware BD player ? That's what another post said about the BD. If so, that indicates a decrypting issue with makemkv. Try something else like anydvdhd

[manono]

... but I tried BadFrames plugin, and it created a kind of ghostly illusion of movement.

That's what the BadFrames filter does - creates new frames by a weighted blending of the good frames on either sides of the bad frame(s). I use it a lot myself, but only with static corrupted frames. For what you want, ReplaceFramesMC is better. I think jagabo modded it from something similar.

In your case where the 2,3,4,5 frames of the six are corrupted, something like this might work:

ReplaceFramesMC(1,4)

This is the filter:

[tom_nito]

But it plays fine as a physical disc in a hardware BD player ? That's what another post said about the BD. If so, that indicates a decrypting issue with makemkv. Try something else like anydvdhd

Some scenes play fine on hardware players depending on the player I use. If I play on a regular player connected to a tv it might play fine, if I play it on my computer with Blu-ray playing software, it might play fine or look corrupted depending on the player.

For the scenes that would play fine under certain conditions, I found that splitting the video right before the corruption, reencoding it, then merging again would fix it. For the scene of unavoidable corruption, I was lucky that the exact same scene plays on the season 4 disc, so I just used that scene as a replacement even though there's a color difference, it's not noticeable.

That particular scene and this one look corrupted no matter what video playing software I use on my computer. And while I don't see corruption for those two problematic scenes if I watch them on a hardware player connected to the tv, I do see that those sections stutter as if it's just freezing and skipping over the bad frames, which is an affect I could achieve by just replacing the corrupted frames with the two surrounding frames.

I might still try AnyDVD HD just to be sure. Does it have any restrictions?

[tom_nito]

I tried AnyDVD HD and I still get the exact same corruption.

[poisondeathray]

If it plays fine on a proper hardware BD player (yes, a regular BD player connected to a TV), there is nothing wrong with the disc.

I just looked at a review and it's VC-1 . So likely you have either one or both of a decrypting and a decoding issue. Not very many choices of decrypting, and if both exhibit the same issue at the exact same spot ... more likely a decoding issue

I add decoding now, because VC-1 is notorious for corruption almost exactly like your screenshots with open source decoders. The Windows DMO codec is stable, and DGDecodeNV (not free, and requires a Nvidia card) is stable . The windows DMO codec comes with windows can be accessed in avisynth with a directshow graph. There should be threads either here or doom9 describing how this is done . Basically you use DirectShowSource("video.grf", fps=23.976 , audio=false) , and in the graph in graphstudio / graphedit you connect to the windows DMO codec with the renderer pin left open (not connected)

[poisondeathray]

Or if you want to cut a section around that point that is corrupted (e.g. with mkvtoolnix) , and upload that sample, I can take a look to rule out / (or in) if it's a decoder issue right away so you don't have to waste time chasing that route (or if it really is the problem...then not waste time trying to "fix" things that don't need to be fixed) . The sample needs to be a few seconds longer on either side (needs to include a few GOP's)

[tom_nito]

... but I tried BadFrames plugin, and it created a kind of ghostly illusion of movement.

That's what the BadFrames filter does - creates new frames by a weighted blending of the good frames on either sides of the bad frame(s). I use it a lot myself, but only with static corrupted frames. For what you want, ReplaceFramesMC is better. I think jagabo modded it from something similar.

In your case where the 2,3,4,5 frames of the six are corrupted, something like this might work:

ReplaceFramesMC(1,4)

This is the filter:

I need some guidance on how to use this. I can't get it to run with VirtualDub. I made a text file with .avs extension with the following:

import("[path]\ReplaceFramesMC.avs") 
AviSource("[path]\video.avi", false)
ReplaceFramesMC(28,31)

Am I doing something wrong or missing something? I'm an avisynth noob and just started trying it last week.

[manono]

You're beginning with frame 28 and want to interpolate 31 frames in a row? If there's much movement at all the results probably won't look pretty, maybe something like what BadFrames might create, or worse.

Anyway, ReplaceFramesMC needs other filters to work. MVTools2, I think. Also, whenever a script doesn't work in VDub, post the error message as well as the script itself.

[tom_nito]

Or if you want to cut a section around that point that is corrupted (e.g. with mkvtoolnix) , and upload that sample, I can take a look to rule out / (or in) if it's a decoder issue right away so you don't have to waste time chasing that route (or if it really is the problem...then not waste time trying to "fix" things that don't need to be fixed) . The sample needs to be a few seconds longer on either side (needs to include a few GOP's)

How is this? There's a second single corrupted frame too that you'll see, but that one seems easy to fix.

It looks like it is VC1.

[tom_nito]

You're beginning with frame 28 and want to interpolate 31 frames in a row? If there's much movement at all the results probably won't look pretty, maybe something like what BadFrames might create, or worse.

Anyway, ReplaceFramesMC needs other filters to work. MVTools2, I think. Also, whenever a script doesn't work in VDub, post the error message as well as the script itself.

I think I meant to choose frames 27 and 30 if that matters because I was trying to choose the equivalent frames of (1, 4) in my test video.

I'm using VirtualDub.exe and trying to open the .avs file that includes the code I attached before and it just says unable to open file. That's what my attempt at a badframes script did as well until I finally figured out I had to load the .dll for it. I've tried the same for the replaceframes one and added this line to my previous code:

LoadPlugin("[path]\mvtools2.dll")

But I still get the same error. I tried what looks like an updated version of mvtools2.dll and mvtools.dll as well. I can't find much information on google about ReplaceFramesMC or what other plugin requirements it might have.

[poisondeathray]

1) This sample is indeed corrupted. But if this disc plays fine in a regular BD player connected to a TV, it is 100% a decrypting issue. Unfortunately not very many decrypting options are available...

Different decoders have different error concealment algorithms. e.g. compare the attached 29, 31 to your version. (Your screenshots used the wrong matrix rec601, it should have used rec709, for HD, but I matched 601 for the ones I uploaded) . The errors are in the exact same location, but appear "closer" in terms of color / content instead of almost random colors. It might be less work to fix if you were attempting manual repair before interpolating . If you just blurred edges through a mask it might be closer to being acceptable

2) ReplaceFramesMC just blends the frames (ghosting) in this situation, which is expected. Certain types of content like cartoons don't interpolate very well. In this sample just simple blending with convertfps does essentially the same thing

3) For your script , post your full script verbatim and the actual error messages that you are getting. I hope your [path] is the actual path

e.g.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\mvtools2.dll")

Do you have an .avi video ? Otherwise AVISource() won't open anything. You need to use an appropriate source filter for your video. For MKV, you might use FFVideoSource or L-Smash (but you need to download the plugins separately)

[manono]

Do you have an .avi video ? Otherwise AVISource() won't open anything.

That was my thought, too. And even if it is some sort of an AVI, it might not be one acceptable to VDub.

So, if it's an AVI, tom_nito, what kind of video is inside? If you don't know, open the video in MediaInfo and find out and either let us know or post the text file available.

Try opening the script with just the single source line:

AviSource("[path]\video.avi", false)

If it doesn't open, then it's not an acceptable AVI. And either fill in the full path, or, if the video is in the same folder as the script:

AviSource("video.avi", false)###substituting the real name of the video

[tom_nito]

1) This sample is indeed corrupted. But if this disc plays fine in a regular BD player connected to a TV, it is 100% a decrypting issue. Unfortunately not very many decrypting options are available...

Different decoders have different error concealment algorithms. e.g. compare the attached 29, 31 to your version. (Your screenshots used the wrong matrix rec601, it should have used rec709, for HD, but I matched 601 for the ones I uploaded) . The errors are in the exact same location, but appear "closer" in terms of color / content instead of almost random colors. It might be less work to fix if you were attempting manual repair before interpolating . If you just blurred edges through a mask it might be closer to being acceptable

2) ReplaceFramesMC just blends the frames (ghosting) in this situation, which is expected. Certain types of content like cartoons don't interpolate very well. In this sample just simple blending with convertfps does essentially the same thing

3) For your script , post your full script verbatim and the actual error messages that you are getting. I hope your [path] is the actual path

e.g.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\mvtools2.dll")

Do you have an .avi video ? Otherwise AVISource() won't open anything. You need to use an appropriate source filter for your video. For MKV, you might use FFVideoSource or L-Smash (but you need to download the plugins separately)

There's definitely some stuttering going on with a regular blu-ray player as if it doesn't want to play those corrupted frames and just skips over them. I don't remember if I tried going step by step through the frames on that player.

I've messed around with different video players on my computer and I get less corruption on some renderers. I forget which one, but I think I got similar results somehow to your screenshot by messing with renderers in MPC-HC. It seemed like the corrupted frames would skip oddly if I went through it frame by frame and the corrupted areas would get filled in by previous frames instead of rainbowy puke. But I can't get that sort of output in VirtualDub.

Yes, my video is an avi that I created with VirtualDub from the original MakeMKV file. It looks like it's RV24 codec.

I think I got ReplaceFramesMC working. I think using ffmpegsource2 with an mkv file got it to work, although I also moved LoadPlugins to the top for my script, I think only ffms2.dll needed to be first.

Does that mean VirtualDub is creating image sequences in the wrong matrix? Or is the mkv wrong? How do I get rec709 images and check what the color matrix is for images and videos? And how do I get output like yours in VirtualDub? I can get it looking somewhat similar in other video players by messing with renderers, but I haven't gotten it to work in VirtualDub.

[manono]

Yes, my video is an avi that I created with VirtualDub from the original MakeMKV file. It looks like it's RV24 codec.

I didn't suggest it wasn't a real AVI. I suggested the video within might not be usable with AviSource. Which seems to be the case. However, RV24, whatever that is, isn't one of the Blu-Ray video codecs.

[poisondeathray]

Does that mean VirtualDub is creating image sequences in the wrong matrix? Or is the mkv wrong? How do I get rec709 images and check what the color matrix is for images and videos? And how do I get output like yours in VirtualDub? I can get it looking somewhat similar in other video players by messing with renderers, but I haven't gotten it to work in VirtualDub.

Short explanation: yes vdub is changing the colorspace, but only when taking screenshots. For the actual video if you use video=> fast compression without any filters, vdub will pass the original YUV video

When you take a screenshot like png, bmp, the YUV video is converted to RGB colorspace/colormodel. By convention "SD" is supposed to use Rec601, "HD" is supposed to use Rec709 . Vdub always converts with Rec601. The colors will look slightly shifted if you use the wrong matrix for YUV<=>RGB conversions. Instead, you can control the RGB conversion in the script when taking screenshots (but not forgetting to comment the line out or erase it) with ConvertToRGB24(matrix="rec709")

The ones I uploaded were decoded with a Mainconcept based decoder (in Adobe) , but the official WMVideoDMO decoder I described above using graphstudio produces similar error concealment results, just slightly more off color. The libavcodec based decoders produce crazy colors such as in the 1st post. They are common ones used in popular media players , although some default one use the MS one for WMV/VC-1 file types - that's probably why you're seeing the variability in different media players. Different decoders also behave slightly differently for errors - for example, some might put a duplicate frame instead of the corrupted one

But you were on the right track - if you can "fix" up more frames, the interpolation results will be generally be better. That is, if you have fewer frames to interpolate (or less motion between them), you usually have higher accuracy. Interpolating a string of 1 or 2 frames, is usually better than 8 or 9. There is just too much missing information in the latter. Or you can look at the other way and use interpolation as a starting point for more manual clean up.

But if you can get a slightly better error concealment result to start with - that means less work for you to do manually later , no matter what workflow you choose.

RV24 is real video variant I think ? I'd try not to convert multiple times, because you can lose quality if not using a lossless codec. If you don't have access to Adobe subscription, the next best one is the official MS one described with graphstudio above. But you have to be careful to process things linearly without temporal filters or seeking around, because DirectShowSource() is not frame accurate when navigating around / scrubbing timeline. Essentially you can "lose your place" and mix up frames

[tom_nito]

I think I have a handle on rec709 now. I can see the color difference. I don't have those Adobe or Microsoft programs. The best I've gotten is MPC-HC with "System Default" DirectShow Video Output. And it seems to replace those corrupted squares with bits from nearby frames. Can you upload rec709 screenshots? Yours look like a better result than what I'm getting. And then I'll see what I can do with all of this.

[poisondeathray]

Can you upload rec709 screenshots? Yours look like a better result than what I'm getting. And then I'll see what I can do with all of this.

Did you mean from the Mainconcept/Adobe decoder ?

[tom_nito]

Those images helped a lot. I cleaned them up and luckily one of the renderers I tried gave my frames a solid beige color that I could easily cut out as a transparent background. Then I used a combination of the images plus the closest uncorrupted frame as backgrounds to fill in the transparent holes. Then it was a matter of patching them together and adjusting the colors. I used BadFrames on the two completely corrupted frames. The water is the only part I didn't mess with because it was too complicated and it looks fine in the video.

I've uploaded my cleaned up frames and the result.

[poisondeathray]

Looks better.

I noticed you fixed some of the blocks in the balloon/aircraft(?) and sky, but not as much on the "ocean"

You might be able to improve on that by "fixing" the ocean a bit more. It's slightly more tricky than usual because this sequence occurs over a fade, but basically you just cut and paste from other clean parts of frames with masks. If you examine this sequence closely, the "ocean waves" just repeat themselves in a pattern. There are clean ocean frames later on but that are "brighter", so you'd have to match the levels and color. It's easier to do in a NLE or compositing application like after effects. If you wanted the "full meal deal" repair there is actually a way by redoing the animation in layers

[tom_nito]

Now I'm having an issue with the colors not matching. ConvertToRGB24(matrix="rec709") seems to change the colors from how they look in the original video. Maybe because I might have used it when I created the image sequence for the video and I was trying to use it again when I was combining the images and using BadFrames? So I removed the ConvertToRGB24 line and the AVI that VirtualDub creates seems to match the original. But when I encode to x264 in Handbrake or Avidemux, then there's a noticeable color change. My image sequence is in BMP because I was trying to have lossless frames, if that matters.

I tried importing the image sequence directly into Avidemux and same result when I tried encoding.

[poisondeathray]

When you made the BMP sequence, how did you convert to RGB? Whenever you convert between YUV and RGB (each way), a conversion is done , that's what the matrix and rec601 or 709 are doing. Often, if you don't specify what you want, software might assume 601 like vdub. Other times, some software will do it based on dimensions, such as a certain width or height

The original video on the BD and in the MKV sample is in YUV . So you actually lose quality by converting it to RGB such as a BMP or PNG . Sometimes it's necessary. You just don't want to do it too many times. A 8bit RGB bitmap is only lossless if the source was 8bit RGB to begin with (your's isn't)

There are ways to "fix" things in YUV, for example with the colormatrix filter. Basically , to change <as if> you had used 601 vs 709 or some other matrix . But it's better to do things the right way only once. The more conversions, even colorspace conversions (like BMP) , generally the more quality loss (there are some lossless conversions and higher bitdepth conversions, but we won't talk about them here)

When you "fix" things, then go back to encoding YUV video from the image sequence, that's another RGB => YUV conversion, so you need to specify the matrix there as well. It wouldn't be ConvertToRGB(matrix="rec709"), it would actually be ConvertToYV12(matrix="rec709") because now you're coming from the RGB sequence back to YUV (such has the h264 delivery format)

Earlier, the ConvertToRGB24(matrix="rec709"), was only to preview the original video. You're not supposed to use it , unless you use that to extract images for editing

The other possibility is your previewing method is again wrong. For example, some directshow filters or even some renderers might be imposing another accessory conversion.

The way it's supposed to look in terms of colors on BD is rec709. Pop in the disc and double check on a BD player (connected to a TV). So the second set of images I loaded (post #18) are how they are supposed to look

[tom_nito]

I think I got it working by using ConvertToYV12(matrix="rec709") when creating the AVI. It looked wrong in VirtualDub with it, but looked correct without it, so that was throwing me off. But exporting and encoding to x264 looks fine if I use ConvertToYV12(matrix="rec709"). I fixed the water too so it's less blocky. I don't notice any color loss now. Everything looks good now, thanks.